Skip to content

Normalize line endings in JavaScript template messages - #1450

Open
Denny-j-mangalathu wants to merge 1 commit into
python-babel:masterfrom
Denny-j-mangalathu:fix/js-template-line-endings
Open

Denny-j-mangalathu wants to merge 1 commit into
python-babel:masterfrom
Denny-j-mangalathu:fix/js-template-line-endings

Conversation

@Denny-j-mangalathu

Copy link
Copy Markdown

A multiline JavaScript template literal evaluates to the same string whether its source file uses LF, CRLF, or CR. Babel currently preserves physical CRLF and CR in the extracted message ID, so a catalog generated from a Windows checkout can miss the runtime lookup.

Normalize physical line endings in template literals before decoding escapes. Explicit carriage-return escapes still produce carriage returns. Token positions are calculated from the original source.

See the note under ECMAScript template literal semantics.

The CRLF and CR regression cases fail on upstream master. With the fix, 88 extraction, lexer, and extraction frontend tests pass. Frontend tests used locale data from the published Babel package.

Prepared with Codex assistance.

@akx akx added area/javascript JavaScript extractor AI Assisted labels Nov 9, 2026

@chrikrah chrikrah left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Denny-j-mangalathu this can be merged from my side; CRLF and CR fail on master.

non-blocking: #1385 edits the same unquote_string line; I'll re-test whichever lands second.

# git archive 307a6ac (head), base a18709c, Python 3.13, pytest 9, babel.__file__ from the export
$ python -m pytest -q tests/messages/test_js_extract.py tests/messages/test_jslexer.py
26 passed
# jslexer.py from a18709c, tests kept
$ python -m pytest -q tests/messages/test_js_extract.py
FAILED test_extract_normalizes_template_line_endings[\r\n]
FAILED test_extract_normalizes_template_line_endings[\r]
2 failed, 18 passed
# CLDR imported with scripts/download_import_cldr.py
$ python -m pytest -q tests --ignore=tests/messages/test_setuptools_frontend.py
6836 passed, 1219 skipped, 1 xfailed
# not run: test_setuptools_frontend.py (needs the package installed)
# git 2.43.0, heads 307a6ac (#1450) and 521f0f9 (#1385)
$ git merge-tree --write-tree 307a6ac 521f0f9
CONFLICT (content): Merge conflict in babel/messages/jslexer.py
exit 1

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI Assisted area/javascript JavaScript extractor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants