Skip to content

fix(llm): preserve event loop in sync cache - #2479

Open
FU-max-boop wants to merge 1 commit into
microsoft:mainfrom
FU-max-boop:fix/cache-event-loop-lifecycle
Open

fix(llm): preserve event loop in sync cache#2479
FU-max-boop wants to merge 1 commit into
microsoft:mainfrom
FU-max-boop:fix/cache-event-loop-lifecycle

Conversation

@FU-max-boop

Copy link
Copy Markdown

Description

Preserve caller-owned asyncio state in the synchronous LLM cache middleware. The middleware previously installed its private event loop as the thread default, then either left a closed loop installed after a cache miss or leaked the loop on an early cache-hit return.

Related Issues

Closes #2478.

Proposed Changes

  • Keep the middleware's private event loop isolated from the caller's default loop.
  • Close the private loop in a finally block on cache hits, cache misses, and exceptions.
  • Add regression coverage for all three lifecycle paths.
  • Add the required patch-level semversioner change record.

Checklist

  • I have tested these changes locally.
  • I have reviewed the code changes.
  • I have updated the documentation (not necessary for this internal lifecycle fix).
  • I have added appropriate unit tests.

Validation

  • uv run poe check
  • uv run poe test_unit (317 passed)
  • Baseline mutation check confirms the new regression assertion fails against current main.

Additional Notes

The asynchronous cache middleware is unchanged because it already runs in the caller's active event loop and does not create or install a private loop.

@FU-max-boop
FU-max-boop requested a review from a team as a code owner August 11, 2026 05:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Sync cache middleware replaces and leaks caller event loops

1 participant