Skip to content

fix(llm): use AsyncOpenAI client instead of sync client in executor - #513

Merged
codelion merged 1 commit into
algorithmicsuperintelligence:mainfrom
akushonkamen:fix/278-async-openai
Oct 13, 2026
Merged

codelion merged 1 commit into
algorithmicsuperintelligence:mainfrom
akushonkamen:fix/278-async-openai

Conversation

@akushonkamen

Copy link
Copy Markdown
Contributor

Problem

#278 points out that openevolve/llm/openai.py uses the blocking openai.OpenAI client and wraps every chat.completions.create call in loop.run_in_executor(...) (openevolve/llm/openai.py:215-224 before this change). Running the sync client on the default thread-pool executor adds a thread hop per LLM call and can deadlock in edge cases when the default executor is saturated.

Changes

  • openevolve/llm/openai.py: construct openai.AsyncOpenAI instead of openai.OpenAI, and await self.client.chat.completions.create(**params, stream=False) directly in _call_api, dropping the run_in_executor round trip. The stream=False non-streaming guarantee from Kivest LLM API returns streaming SSE by default causing 'str' object has no attribute 'choices' error #436 and the raw-SSE-string guard are preserved unchanged.
  • tests/test_openai_non_streaming.py, tests/test_reasoning_effort_config.py: patch openai.AsyncOpenAI and use AsyncMock for the awaited create call.
  • tests/test_llm_ensemble.py: use AsyncMock for the mocked create in the token-usage logging test.

Testing

  • OPENAI_API_KEY=test-key python -m pytest tests/ --ignore=tests/integration: 647 passed, 84 subtests passed (~4m10s).
  • python -m compileall openevolve/llm/openai.py: passes.
  • One pre-existing flaky failure in tests/test_coevolution.py::test_real_process_workers_and_resume was observed once when running the whole suite under load (spawns real worker processes); it passes reliably in isolation both with and without this change.

Fixes #278

Replace the blocking openai.OpenAI client with openai.AsyncOpenAI and
await chat.completions.create directly, instead of running the sync
client via run_in_executor on the default thread pool. This removes
the thread-pool round trip and the risk of deadlocks when the default
executor is saturated.

Fixes algorithmicsuperintelligence#278
@codelion
codelion merged commit a1e9d91 into algorithmicsuperintelligence:main Oct 13, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Suggestion: Use AsyncOpenAI instead of the default OpenAI package

2 participants