Skip to content

fix(version): require a comma between version constraints - #26

Merged
jonyoder merged 1 commit into
mainfrom
measure-r-comma
Aug 10, 2026
Merged

jonyoder merged 1 commit into
mainfrom
measure-r-comma

Conversation

@jonyoder

Copy link
Copy Markdown
Collaborator

Completes the invented-comma fix from 0.3.0, which closed the tokenizer route and left this one. This is rstudio/package-manager#19466's F5 residue — the last open item from that audit.

The defect

validConstraintRegexp ended each repetition with \s*\,?, making the separator optional. Combined with the deliberately-admitted empty operator (a bare version is a valid constraint), adjacency did not merely parse — it rendered a comma the input never contained:

"==0.1dev10.3"  ->  "==0.1dev10" + bare "3"  ->  renders "==0.1dev10,3"

A fabricated constraint boundary changes which versions match, and the publisher's intent is unrecoverable from the string.

⚠️ The R path is why this sat open, so it was measured, not argued

validConstraintRegexp is shared with NewRSpecifiers, which PPM uses for R constraint parsing.

Against a real full CRAN PACKAGES file:

Constraint occurrences 58,656
Distinct expressions 2,939
Accept/reject change none — identical before and after

Independently corroborated over ~3.9M parenthesised constraint expressions extracted from production CRAN RSFs: zero with two operators, zero with an internal comma. Real CRAN ranges do exist, but are written as the same package repeated across a field (R (>= 3.6.0), R (< 3.7.0)) — handled by PPM's src/description, which never reaches NewRSpecifiers.

A null result is only meaningful if the instrument works, so the change was verified to actually reject six adjacency shapes (>= 7 < 10, >=1.0<2.0, >= 1.0 != 1.3.4.* < 2.0, …) and to leave twelve load-bearing forms untouched — R dash forms, the real CRAN 1.1-18-1 and 2016-12-12 shapes, bare versions, comma forms, trailing comma, *, ===. Without that check, "no diff" could have meant an inert regex.

Python side

Over the production PyPI snapshot, 2,804,136 distinct requirement strings: exactly 21 newly rejected, 0 newly accepted.

Every one is an operand whose version match is a proper prefix leaving a valid second token — so every one previously rendered a comma from nowhere:

==0.1dev10.3      >=9.0.0a1.0      ==1.1b02.1      >=0.2.1dev4.dev0

Seven sampled forms confirmed rejected by pypa/packaging 26.2, so this closes a divergence rather than opening one. Controls (flask (>=3.0.0), urllib3 (>=1.26,<2.0), sgx-py (>=0.2.1.dev4)) all still accepted.

Deliberately unchanged

  • The empty operator stays. A bare version is a valid constraint, and PPM's GET …/doc?version=<raw> (optional auth) passes one through. Narrowing it is separately tracked in rstudio/package-manager#18634.
  • A trailing comma is still tolerated — upstream accepts >= 1.0, too.

One narrowing at a time is what keeps the blast radius measurable.

Five tests pinned the defect

TestVersion_Check asserted >= 1.0 != 1.3.4.* < 2.0 as valid. Measured: pypa/packaging 26.2 raises InvalidSpecifier on it, and accepts the comma form. Rewritten to commas — which is what PEP 440 itself writes — with each case's intent (the conjunction) unchanged.

⚠️ The two entry points diverge on dashes

newRSpecifiers rewrites - to . before validating, so >=0-12.1 is the single constraint >=0.12.1 on the R path — while on the Python path PEP 440 reads 0-12 as the post-release 0.post12, and the trailing .1 makes it an adjacent pair. A test asserting both paths reject it would have been wrong; both halves are now pinned.

Sizing

Breaking. Narrows what parses, so per this module's policy it takes a minor bump (0.4.0), not a patch. PPM pins v0.1.1 and imports only gpp/license, so nothing reaches PPM until the cutover.

Verification

  • go test ./... — all packages pass, including internal/proptest.
  • gofmt -l . — empty.
  • Both measurements re-run from scratch rather than citing the audit's figures; the audit said "20", the current corpus measures 21.

🤖 Generated with Claude Code

Completes the invented-comma fix from 0.3.0, which closed the tokenizer route and
left this one. rstudio/package-manager#19466's F5 residue.

validConstraintRegexp ended each repetition with `\s*\,?`, making the separator
OPTIONAL. Combined with the deliberately-admitted empty operator -- a bare
version is a valid constraint -- adjacency did not merely parse, it RENDERED A
COMMA THE INPUT NEVER CONTAINED:

    "==0.1dev10.3"  ->  "==0.1dev10" + bare "3"  ->  renders "==0.1dev10,3"

A fabricated constraint boundary changes which versions match, and the
publisher's intent is unrecoverable from the string.

⚠️ THE R PATH WAS THE REASON THIS SAT OPEN, because validConstraintRegexp is
shared with NewRSpecifiers, which PPM uses for R constraint parsing. Measured
against real CRAN data rather than argued:

    58,656 constraint occurrences, 2,939 distinct expressions
    (data/cache/23_PACKAGES_1-2026-05-01.gz, a real full CRAN PACKAGES file)
    -> IDENTICAL accept/reject before and after. Zero change.

Independently corroborated over ~3.9M parenthesised constraint expressions
extracted from production CRAN RSFs: zero with two operators, zero with an
internal comma. Real CRAN ranges exist but are written as the same package
repeated across a field ("R (>= 3.6.0), R (< 3.7.0)"), which is handled by
src/description and never reaches NewRSpecifiers.

⚠️ A NULL RESULT IS ONLY MEANINGFUL IF THE INSTRUMENT WORKS, so the change was
verified to reject six adjacency shapes (">= 7 < 10", ">=1.0<2.0", ...) and to
leave twelve load-bearing forms untouched -- R dash forms, the real CRAN
"1.1-18-1" and "2016-12-12" shapes, bare versions, comma forms, trailing comma,
"*", "===". Otherwise "no diff" could have meant an inert regex.

PYTHON SIDE, over the production PyPI snapshot (2,804,136 distinct requirement
strings): exactly 21 NEWLY REJECTED, 0 newly accepted. Every one is an operand
whose version match is a proper prefix leaving a valid second token, so every one
previously rendered a comma from nowhere -- "==0.1dev10.3", ">=9.0.0a1.0",
"==1.1b02.1", ">=0.2.1dev4.dev0". Seven sampled forms confirmed REJECTED by
pypa/packaging 26.2, so this closes a divergence rather than opening one.

DELIBERATELY UNCHANGED: the empty operator stays, because a bare version is a
valid constraint and PPM's /doc?version=<raw> passes one (narrowing it is
rstudio/package-manager#18634); and a trailing comma is still tolerated, which
upstream also accepts. One narrowing at a time is what keeps this measurable.

ALSO FIXES FIVE TESTS THAT PINNED THE DEFECT. TestVersion_Check asserted
">= 1.0 != 1.3.4.* < 2.0" as valid. Measured: pypa/packaging 26.2 raises
InvalidSpecifier on it and accepts the comma form. Rewritten to commas, which is
what PEP 440 itself writes; each case's intent (the conjunction) is unchanged.

⚠️ THE TWO ENTRY POINTS DIVERGE ON DASHES and a test that conflates them is
wrong. newRSpecifiers rewrites "-" to "." BEFORE validating, so ">=0-12.1" is the
single constraint ">=0.12.1" on the R path, while on the Python path PEP 440
reads "0-12" as "0.post12" and the trailing ".1" makes it an adjacent pair. Both
halves are pinned.

Also: Go permits a repeated capture-group name, so embedding the version pattern
twice needs no name stripping. Verified rather than assumed -- it is the opposite
of Python's rule, and assuming wrong would have meant a needless helper.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@jonyoder
jonyoder merged commit f028a66 into main Aug 10, 2026
4 checks passed
@jonyoder
jonyoder deleted the measure-r-comma branch August 10, 2026 12:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant