Repository navigation
Conversation
Replace the blocking openai.OpenAI client with openai.AsyncOpenAI and await chat.completions.create directly, instead of running the sync client via run_in_executor on the default thread pool. This removes the thread-pool round trip and the risk of deadlocks when the default executor is saturated. Fixes algorithmicsuperintelligence#278
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
#278 points out that
openevolve/llm/openai.pyuses the blockingopenai.OpenAIclient and wraps everychat.completions.createcall inloop.run_in_executor(...)(openevolve/llm/openai.py:215-224before this change). Running the sync client on the default thread-pool executor adds a thread hop per LLM call and can deadlock in edge cases when the default executor is saturated.Changes
openevolve/llm/openai.py: constructopenai.AsyncOpenAIinstead ofopenai.OpenAI, andawait self.client.chat.completions.create(**params, stream=False)directly in_call_api, dropping therun_in_executorround trip. Thestream=Falsenon-streaming guarantee from Kivest LLM API returns streaming SSE by default causing 'str' object has no attribute 'choices' error #436 and the raw-SSE-string guard are preserved unchanged.tests/test_openai_non_streaming.py,tests/test_reasoning_effort_config.py: patchopenai.AsyncOpenAIand useAsyncMockfor the awaitedcreatecall.tests/test_llm_ensemble.py: useAsyncMockfor the mockedcreatein the token-usage logging test.Testing
OPENAI_API_KEY=test-key python -m pytest tests/ --ignore=tests/integration: 647 passed, 84 subtests passed (~4m10s).python -m compileall openevolve/llm/openai.py: passes.tests/test_coevolution.py::test_real_process_workers_and_resumewas observed once when running the whole suite under load (spawns real worker processes); it passes reliably in isolation both with and without this change.Fixes #278