Skip to content

fix(auth): coordinate refresh across CLI processes - #1457

Open
jeffpignataro wants to merge 3 commits into
mainfrom
jeff/fix/auth-refresh-coordination
Open

jeffpignataro wants to merge 3 commits into
mainfrom
jeff/fix/auth-refresh-coordination

Conversation

@jeffpignataro

@jeffpignataro jeffpignataro commented Sep 16, 2026

Copy link
Copy Markdown
Collaborator

When multiple CLI processes encounter an expired bearer, they can submit the same single-use refresh token. One refresh succeeds; the others receive a rejection and report that the session expired. This can make a normal bearer expiration look like a requirement to log in again.

Coordinate refresh through one transaction shared by MCP, the Bazel credential helper, and AXL credential lookup. Waiting callers re-read the store and adopt the replacement token. A store-wide advisory lock also protects login, logout, and deployment removal from lost updates to the shared credential map. File-backed credentials use private temporary files and atomic replacement.

Lock acquisition is bounded to 60 seconds and automatic refresh to 30 seconds. Refresh rejection is distinguished from transient failures, which advise retrying and preserve the stored credentials; raw token-endpoint response bodies are not exposed.

On Unix, keyring locks use a private, ownership-checked directory keyed by the effective OS user ID, independent of HOME, TMPDIR, and XDG_RUNTIME_DIR. Windows uses the OS known-folder API.

Validation

  • cargo test -p axl-runtime --lib engine::aspect::: 132 passed.
  • cargo test -p aspect-cli --test auth_refresh: 3 passed. Separate credential-helper and AXL processes share exactly one grant; logout is not overwritten by a pending refresh; failed refreshes preserve credentials.
  • The same subprocess concurrency test fails against released v2026.37.6 with the original session expired message.
  • cargo fmt --all --check, BUILD formatting, and git diff --check passed.
  • bazel test //crates/aspect-cli:auth_refresh_test --test_output=errors: passed at 392167c3 (final test result reused from cache).
  • bazel test //crates/axl-runtime:test --test_filter=engine::aspect::credential_store:: --test_output=errors: passed at a01a773b.
  • Subprocesses with different, unusable home directories and different temp/runtime overrides contend on the same test lock. Restoring the old HOME-derived path makes this test fail. Ownership, permission, and symlink checks are covered.
  • Repository pre-commit formatting hook passed.

Scope and rollout

Tests use synthetic credentials and a loopback issuer. Live keychain integration and Frontegg session policies were not tested. This addresses the reproducible refresh race; it does not establish the cause of a particular 24-hour logout or extend provider session lifetimes.

Restart long-running MCP processes after upgrading: older binaries do not participate in the lock. Independently copied credential stores remain separate and cannot safely share a rotating refresh token.

@coderabbitai

coderabbitai Bot commented Sep 16, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: d0799f6b-3fc5-4204-bfeb-af2d54d73f8a

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@jeffpignataro
jeffpignataro marked this pull request as ready for review September 16, 2026 00:18
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 16, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-16T00:22:12.141505Z 392167c Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@aspect-workflows

aspect-workflows Bot commented Sep 16, 2026

Copy link
Copy Markdown

Aspect Workflows Tasks

📅 Tue Sep 22 18:10:23 UTC 2026

Task Results


⏱ Last updated Tue Sep 22 18:17:05 UTC 2026 · 📊 GitHub API quota 1,598/7,700 (21% used, resets in 18m)
🚀 Powered by Aspect CLI (v0.0.0-dev)  |  Aspect Build · X · LinkedIn · YouTube

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 392167c308

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/axl-runtime/src/engine/aspect/credential_store.rs Outdated
let worker_grants = grants.clone();
let worker_stop = stop.clone();
let worker_bearer = bearer.clone();
let thread = std::thread::spawn(move || {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i don't quite understand the use of thread here if this code meant to be single threaded.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The thread isn't there to serve requests in parallel. It allows the main thread to keep going while a connection is parked. I'll adjust the wording though.

🤖 details:
wait_for_grant() returns once child #1's grant reaches the issuer, and the issuer then blocks on release_rx holding that connection open. Meanwhile the main thread spawns the second helper and the AXL child, which is exactly the window the test needs: both competing processes hit the locked refresh transaction while #1's grant is still in flight, and their requests queue on the listener. Without coordination they'd submit the old refresh token and the issuer would 401 them — that's the assertion grants == 1 is protecting.

On one thread you'd need a state machine interleaving accept polling with Command::spawn to get the same interleaving. I'll move the rationale up to the spawn so it's not buried in the loop body.

…g failures

The rationale for the loopback issuer's background thread sat inside the
loop body and only covered why responses are served serially, leaving the
`thread::spawn` itself unexplained. Move it above the spawn and say what
the thread is actually for: the test drives the other half of the
exchange, holding the first grant open while it starts the competing
processes.

`Drop` also re-raised a worker panic through `join().unwrap()`. That drop
runs while unwinding from the test's own failure, so the second panic
aborted the process and the original assertion went unreported.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015XYrr1YGB4TDzf3nfP4c9K

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants