Skip to content

Tighten ast-grep rules in CI - #2753

Merged
seanmcguire12 merged 5 commits into
mainfrom
protocol-pipeline-checks
Aug 21, 2026
Merged

seanmcguire12 merged 5 commits into
mainfrom
protocol-pipeline-checks

Conversation

@monadoid

@monadoid monadoid commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Before: a developer could add, rename, or remove a protocol field without every SDK constructing or consuming it.

    • Now: source-derived tests compare every registered request and result field with the TypeScript, Python, and Go wrappers.
  • Before: SDK-only options could drift between TypeScript Zod schemas, Python declarations and runtime models, and Go structs.

    • Now: cross-language parity tests derive and compare those fields automatically.
  • Before: public creation and browser options could be added without updating the reference and configuration docs.

    • Now: Stagehand.create fields are checked structurally, and browser and logging fields must appear in the guides.
  • Before: a registered operation could be omitted from its receiving endpoint or wired in the wrong direction.

    • Now: every operation must have exactly one receiving endpoint and matching SDK boundaries.
  • Before: the protocol-schema lint checked obsolete paramsSchema and resultSchema properties.

    • Now: it checks the actual params and result registry entries.

Validation


Summary by cubic

Tightens cross-language protocol pipeline checks and documentation completeness to prevent SDK drift. AST-based tests now enforce field construction/consumption, single-endpoint routing, and canonical schema references across TypeScript, Python, and Go, and CI runs them.

  • Field pipeline: constructs or forwards every request field and consumes every result field; supports rest-spread adapters; exempts page.screenshot.type.

  • Routing parity: each protocol operation is handled by exactly one receiving endpoint (extension or SDK); SDK outbound/inbound sets match the extension router.

  • SDK schema parity and docs: compares TS Zod schemas with Python declarations/runtime models and Go structs; Stagehand.create fields align across languages and are documented structurally; browser and logging fields appear in configuration guides; every RPC-backed TypeScript object with public methods has a reference page; exempts legacy extension_id overrides in LocalBrowserConnectOptions and BrowserbaseConnectOptions from doc mention checks.

  • Linting and CI: no-loose-json-schema-in-protocol-operations requires params/result in StagehandMethods/StagehandNotifications to reference canonical schemas; CI runs via pnpm test:unit.

  • Required migration: replace any remaining paramsSchema/resultSchema with params/result that reference the canonical schemas.

Written for commit 677e7e5. Summary will update on new commits.

Review in cubic

@mintlify

mintlify Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
stagehand 🟢 Ready View Preview Aug 17, 2026, 1:29 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@changeset-bot

changeset-bot Bot commented Aug 17, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 677e7e5

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@monadoid monadoid changed the title Add protocol pipeline checks Tighten ast-grep rules in CI Aug 17, 2026
@monadoid
monadoid marked this pull request as ready for review August 17, 2026 13:51
@monadoid
monadoid requested a review from a team as a code owner August 17, 2026 13:51

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 7 files

Architecture diagram
sequenceDiagram
    participant CI as CI Pipeline
    participant Unit as Unit Tests (vitest)
    participant Protocol as Protocol Schema (stagehand.v4.json)
    participant Registry as Schema Registry (TS)
    participant TS as TypeScript SDK
    participant PY as Python SDK
    participant GO as Go SDK
    participant Ext as Extension rpcRouter
    participant Docs as Documentation MDX

    Note over CI,Unit: NEW: Cross-language protocol validation pipeline
    
    CI->>Unit: pnpm test:unit
    Unit->>Protocol: Read protocol document
    Unit->>Registry: Read StagehandMethods registry
    
    Note over Unit,TS: Field Pipeline Verification
    Unit->>TS: Extract public RPC calls (source AST)
    Unit->>PY: Extract public RPC calls (source AST)
    Unit->>GO: Extract public RPC calls (source AST)
    TS-->>Unit: Request/result field usage
    PY-->>Unit: Request/result field usage
    GO-->>Unit: Request/result field usage
    Unit->>Unit: Verify every request field constructed & forwarded
    Unit->>Unit: Verify every result field consumed & returned

    Note over Unit,Ext: Routing Parity Verification
    Unit->>Ext: Find request.method route switch
    Ext-->>Unit: Extension inbound operations list
    Unit->>TS: Find outbound .send() calls
    Unit->>PY: Find outbound .send() calls
    Unit->>GO: Find outbound .call() calls
    TS-->>Unit: Outbound operation set
    PY-->>Unit: Outbound operation set
    GO-->>Unit: Outbound operation set
    
    alt Operations handled by both endpoints
        Unit->>Unit: Fail - duplicate routing detected
    else Every operation has exactly one receiver
        Unit->>Unit: Pass - extension and SDK boundaries match
    end

    Note over Unit,Docs: Schema Parity Verification
    Unit->>TS: Read Zod client schemas (clientSchemas.ts)
    TS-->>Unit: Canonical field names
    
    Unit->>PY: Parse TypedDict declarations (client_types.py)
    PY-->>Unit: Public input fields
    Unit->>PY: Parse Pydantic models (client_models.py)
    PY-->>Unit: Runtime model fields
    
    alt Python input != runtime model
        Unit->>Unit: Fail - declaration drift detected
    end
    
    Unit->>GO: Parse struct definitions (client_options.go, browser_factories.go)
    GO-->>Unit: Struct field names
    
    alt Go struct fields != TypeScript Zod fields
        Unit->>Unit: Fail - cross-language schema drift detected
    end

    Note over Unit,Docs: Documentation Completeness
    Unit->>Docs: Scan Stagehand.create reference sections
    Unit->>Docs: Scan browser configuration sections
    Docs-->>Unit: Documented ParamField entries
    
    alt Stagehand.create fields missing from docs
        Unit->>Unit: Fail - reference documentation incomplete
    end
    
    alt Browser/extension options undocumented
        Unit->>Unit: Fail - configuration guide incomplete
    end

    Note over Unit,CI: Lint Rule Update
    Unit->>Unit: Verify no-loose-json-schema rule targets params/result
    alt Operation uses loose schema instead of canonical registry
        Unit->>Unit: Fail - lint error reported
    else Operation references canonical $defs schema
        Unit->>Unit: Pass - schema usage valid
    end

    CI-->>CI: Pipeline completes with gating results
Loading

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread packages/docs/v4/reference/stagehand.mdx Outdated
Comment thread packages/docs/v4/configuration/browser.mdx Outdated
Comment thread rules/ast-grep/sdk-parity.test.ts Outdated
Comment thread rules/ast-grep/sdk-field-pipeline.test.ts Outdated
Comment thread rules/ast-grep/sdk-field-pipeline.test.ts Outdated
Comment thread rules/ast-grep/sdk-field-pipeline.test.ts Outdated

@akeimach akeimach left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@seanmcguire12
seanmcguire12 merged commit 6ccd616 into main Aug 21, 2026
52 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants