Skip to content

fix(tier-check): count an SDK release in the 24h before a spec release as tracking it - #458

Merged
felixweinberger merged 1 commit into
modelcontextprotocol:mainfrom
maxisbey:fix/tier-check-spec-tracking-window
Aug 17, 2026
Merged

fix(tier-check): count an SDK release in the 24h before a spec release as tracking it#458
felixweinberger merged 1 commit into
modelcontextprotocol:mainfrom
maxisbey:fix/tier-check-spec-tracking-window

Conversation

@maxisbey

@maxisbey maxisbey commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

Fixes #459.

tier-check's spec-tracking check looks for the first SDK release at or after the latest spec release. SDKs that ship in lockstep publish in the hours before the spec tag, so the check reads them as "no release for this spec yet" and flips to fail (a Tier 1 blocker) 30 days later unless they happen to release again. This allows a 24h lead.

For the 2026-07-28 spec (tagged 16:47Z): python-sdk v2.0.0 went out at 13:41Z, go-sdk v1.7.0 at 13:09Z, typescript-sdk's 2.0.0 packages the evening before — all three currently show 19d gap and would turn fail on 2026-08-28; with this change they pass at 0d/−1d. csharp-sdk (21:27Z, after the tag) is unchanged, and java-sdk (nothing since June) still fails on day 31 as it should.

Replaying the rule over every stable spec release × ten SDKs (70 pairs): 56 identical, 14 change, none from pass to fail. The pattern predates this cycle — typescript-sdk 1.23.0 was published 3 minutes before the 2025-11-25 tag, and kotlin-sdk 0.4.0 shipped the morning of 2025-03-26 yet was scored as a 35-day fail against that spec.

Noticed but left alone (pre-existing, separate if wanted): SDK releases are read with per_page: 50, which typescript-sdk's per-package releases (159 so far) will eventually outrun; and pre-releases count as SDK releases.

@pkg-pr-new

pkg-pr-new Bot commented Aug 16, 2026

Copy link
Copy Markdown

Open in StackBlitz

npx https://pkg.pr.new/@modelcontextprotocol/conformance@458

commit: dafbaa8

…e as tracking it

Lockstep SDKs publish in the hours before the spec release is tagged
(for 2026-07-28: python-sdk 3h early, go-sdk 3.5h, typescript-sdk the
evening before), so "first SDK release at or after the spec" read them
as not having released and would fail them 30 days later. Allow a 24h
lead.
@maxisbey
maxisbey force-pushed the fix/tier-check-spec-tracking-window branch from b1f6f02 to dafbaa8 Compare August 16, 2026 12:45
@maxisbey maxisbey changed the title fix(tier-check): count an SDK release published just before the spec as tracking it fix(tier-check): count an SDK release in the 24h before a spec release as tracking it Aug 16, 2026
@maxisbey
maxisbey marked this pull request as ready for review August 16, 2026 12:51

@felixweinberger felixweinberger left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, why 24h - feels like it should be at any time before the spec release? If it's compliant a week before, this rule wouldn't work would it?

@maxisbey

Copy link
Copy Markdown
Contributor Author

Closing in favour of tracking the problem as an issue: #459.

@maxisbey maxisbey closed this Aug 17, 2026
@maxisbey

Copy link
Copy Markdown
Contributor Author

Reopening: this is the fix for #459.

@maxisbey maxisbey reopened this Aug 17, 2026
@felixweinberger
felixweinberger enabled auto-merge (squash) August 17, 2026 10:34
@felixweinberger
felixweinberger merged commit 74edef3 into modelcontextprotocol:main Aug 17, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tier-check spec-tracking does not detect SDK releases published shortly before the spec release

2 participants