Repository navigation
feat(llm): add Codex app-server provider for ChatGPT subscription auth - #512
Merged
codelion merged 1 commit intoOct 13, 2026
Conversation
Closes algorithmicsuperintelligence#484 OpenEvolve's LLM backends require an API key, so Codex subscription users could not run custom harnesses against their subscription. Add an opt-in `codex_app_server` provider following the maintainer's guidance: - Talks to `codex app-server` over its stdio JSONL JSON-RPC transport (initialize handshake -> thread/start -> turn/start), never reads the CLI's token files; tokens are owned and refreshed by the app-server. - Auth boundary: verifies a ChatGPT-plan login via account/read before every generation; missing or API-key-mode accounts fail fast with actionable `codex login` instructions and are not retried. - Threads run non-interactively (approvalPolicy=never, read-only sandbox), so generation cannot mutate the workspace. - The default OpenAI (API-key) path is untouched; no provider configured still routes to OpenAILLM. Tests drive an in-process fake app-server over real OS pipes (no network, no real binary): auth accept/reject at the boundary, protocol ordering, interrupted/crashed/hung servers, retries, registry wiring, and config propagation. Docs: new README provider section. Signed-off-by: akushonkamen <akushonkamen@163.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #484
Summary
Add an opt-in
codex_app_serverLLM provider so Codex / ChatGPT subscription users can run OpenEvolve without an API key (as requested in #484, following @codelion's guidance to go through the Codex app-server rather than the CLI's token files).codex_app_server(openevolve/llm/codex_app_server.py): talks tocodex app-serverover its stdio JSONL JSON-RPC transport —initializehandshake →account/read(auth boundary) →thread/start→turn/start. The app-server owns and refreshes the OAuth tokens; OpenEvolve never reads the CLI's token files.account/readmust report a ChatGPT-plan login. Unauthenticated accounts (or accounts running in API-key mode) fail fast with actionablecodex logininstructions and are not retried — retrying cannot fix an auth error.approvalPolicy=neverand a read-only sandbox, so generation cannot mutate the workspace."codex_app_server"in_PROVIDER_REGISTRY(openevolve/llm/ensemble.py) and re-exported fromopenevolve/llm/__init__.py, following the existingcopilot_cliprovider pattern.codex login, thenprovider: codex_app_server), auth modes, and failure modes.Motivation
OpenEvolve's LLM backends hard-require an API key:
LLMModelConfig.api_key(openevolve/config.py:56, env-resolved in__post_init__atopenevolve/config.py:100-101) is consumed byOpenAILLM(openevolve/llm/openai.py:63) and injected into theopenai.OpenAIclient (openevolve/llm/openai.py:87-88). Users on a Codex subscription have no API key at all, so they could not run custom harnesses against their subscription. As of the issue baseline (HEAD30d9752) the repo had zero Codex integration:git grep -i codexinopenevolve/→ 0 hits.The default API-key path is untouched:
openevolve/config.pyandopenevolve/llm/openai.pyhave zero changes in this PR, and a config with no provider still routes toOpenAILLM.Scope notes
openevolve/llm/__init__.pyandopenevolve/llm/ensemble.pyalready failisort --checkonmain(verified against30d9752); this PR keeps the files' existing import arrangement rather than reordering unrelated imports.tests/test_sample_from_island_ratios.py; reproduced on cleanmainviagit stash— flaky, unrelated to this change.Testing
New fully-offline suite
tests/test_codex_app_server_llm.py(24 tests): an in-process fake app-server driven over real OS pipes, so the client code exercises its actual JSON-RPC machinery end to end — no network, no realcodexbinary. Covers: auth accept (ChatGPT account) / reject (unauthenticated) / reject (API-key mode, asserting noturn/startis ever sent), protocol ordering, interrupted / crashed / hung servers, retry behavior, registry wiring, and config propagation.Test evidence (this branch; the system has no pip/pytest, so commands ran via a uv-created
.venvinterpreter):The 2 skips are pre-existing scipy skips (
tests/test_circle_packing_best_program.py:2 skipped), and the 1 warning is a pre-existingPytestReturnNotNoneWarningintests/test_regional_endpoint.py— a file this PR does not touch.Formatting:
black --check --line-length 100(the repo's pyproject settings) clean on all 4 touched.pyfiles.Development was red → green: the initial failing tests caught three real bugs before going green — a helper renamed without updating a call site;
CodexAuthErrorwrongly being retried; notifications from the app-server being dropped while a response was awaited.Transparency note: this contribution is prepared with AI assistance (an automated coding agent), reviewed and submitted by me — same disclosure as in the issue claim comment.