[📋 ADR]: 17685 — The network async/event API

[titusfortner]

Decision record

docs/decisions/17685-network-handler-behavior.md (#17685)

Binding implementation

• Java — PR
• Python — [py] Align driver.network handlers with ADR 17685 #18051
• Ruby — PR
• .NET — PR
• JavaScript — PR

Notes

none yet

[roberalani26]

Reviewing docs/decisions/17685-network-handler-behavior.md, I am concerned about how we plan to reconcile the event loop integration with existing synchronous blocking calls in the Python implementation. Should we implement a custom context manager to handle the asyncio loop lifecycle, or do you expect the end-user to manually inject an existing loop instance into the network handler constructor? Without a clear standard for event loop ownership, we risk significant race conditions when this library is used within larger frameworks like FastAPI or Quart.

[titusfortner]

The ADR doesn't propose asyncio and the Python binding as written doesn't use it. "Async" in the title refers to actions taken on event subscriptions over the BiDi socket independent of the main thread. Python's BiDi transport is thread-based (a daemon websocket thread plus threads to manage dispatched events), and the ADR deliberately leaves concurrency mechanics to each binding. There is no loop to inject or manage. Running synchronous Selenium inside FastAPI or Quart is unchanged by this decision: you can wrap calls in asyncio.to_thread as you would today.

[pricklymusic]

Given the current thread-based BiDi transport, we should clarify if the proposed event API will expose a thread-safe queue for these subscription events to bridge them into the main execution context. How do we ensure that user-defined callbacks triggered by the daemon websocket thread do not create race conditions when interacting with objects that lack internal locking mechanisms?

[titusfortner]

This ADR defines the handler’s observable behavior, not its transport or thread-affinity. A request that blocks navigation must be settled before navigate can return. By default Selenium should require document readiness state to be Complete before a navigation call returns (not sure if this is true currently). For network events that happen after readiness state is settled, there's no way for Selenium to know what the main thread considers “complete”; callers can synchronize on a handler signal using their language’s latch/event primitive (threading.Event in Python?). Bindings must make their own dispatch and settlement safe, but cannot guarantee thread safety for user-owned objects and should not imply that arbitrary concurrent driver access is supported.

[titusfortner]

That said, we may need a separate Navigation ADR to clarify how BiDi changes behavior.

[nvborisenko]

This is very good point to raise. Remote end events and users handlers...

Let's review generic pseudo-code:

selenium.driver.on_some_event =>
{
    wait(2 secs);
    console.out(event.data);
}

Users provided handler is just a method which Selenium invokes... how.. concurrently?.. Or awaiting per event?..

I believe in most cases it doesn't matter for user.. But there is but. We don't know how user may handle events. During year (in .NET implementation) I was changing my mind many times to be fast/performant/consistent/error_proning/memory_friendly - and finished with simple statement: Selenium should provide ordered events. It is game changer, which dictates event handlers should be executed one by one (sync) per event.

Why:

• Ordering
• If user doesn't care ordering (async execution), he can opt-in-out

I am very happy how it is implemented in .NET (low-level).

I am proposing to review pseudo-use-cases to highlight cases earlier. It will also help in implementation.

[titusfortner]

Agreed, the ADR requires handlers to be processed in reverse order, not independently. But I think the concern being raised here is more about how information passes between the async handler and the main thread.

Hopefully we'll have specific code to point at soon in case there are things to clarify better.

[AutomatedTester]

There's code to point at now for the Python row: #18051 implements all eleven decisions for driver.network across the request, response and authentication families. It's Python-only and deliberately doesn't touch the other four rows.

A few things above can be answered concretely from it.

Ordering — @nvborisenko's point and @titusfortner's reply

Handlers are dispatched sequentially, last-registered first, in a single pass per event, with the chain stopping at the first handler to settle a disposition. It's a plain loop, not a fan-out:

https://github.com/SeleniumHQ/selenium/blob/b57eb6e0840/py/private/_network_handlers.py#L1131-L1142

So the "ordered events, one at a time, not independently" property holds in Python. Worth flagging for the other rows that Ruby currently runs handlers in parallel threads, which does not satisfy this and will race when two handlers try to settle the same event. That's a behavioural change for the Ruby row, not just an API change.

Navigation and settlement

By default Selenium should require document readiness state to be Complete before a navigation call returns (not sure if this is true currently).

For Python, currently: no, not via readiness state. driver.get() issues the classic WebDriver Command.GET (py/selenium/webdriver/remote/webdriver.py:526), so how long it blocks is governed by the pageLoadStrategy capability (normal / eager / none), not by a BiDi browsingContext.navigate with wait: "complete". Nothing in #18051 changes that — it intentionally stays out of navigation.

The related half of your point is load-bearing though, and implementing this surfaced it as a real defect rather than a theoretical one:

• An intercepted request that is never settled holds navigation open until the command times out. While building body collection I hit a case where network.getData for a body that will never exist (a GET) never answers at all — it doesn't error, it hangs — so the request stayed blocked and stalled the page. Fixed in the PR by guarding on bodySize, but the general shape is exactly "a request that blocks navigation must be settled before navigate can return", and nothing in the ADR forces a binding to guarantee that. Each binding can reintroduce it independently.
• Separately, failing an event unblocks the browser, which lets the user's navigation call return and immediately re-enter driver.network. Recording the handler's exception after failing races that call, making decision 7's "surfaces to the user" non-deterministic. Ordering matters and the ADR doesn't say so.

Both are the kind of thing a shared conformance test would catch once instead of five times.

So +1 on a separate Navigation ADR — but I'd suggest the settlement-vs-navigation guarantee belongs there explicitly, rather than being left implicit in this one.

Two decisions that need a call before the other rows start

1. Backwards compatibility. The ADR's Consequences say the change "is not backwards compatible", but AGENTS.md and CONTRIBUTING.md require deprecate-then-remove. [py] Align driver.network handlers with ADR 17685 #18051 resolves this by keeping every existing public method working with a DeprecationWarning, with one exception: decision 2 moves URL matching into the browser, so client-side glob expansion ("**/api/**") stops working and only warns. A warning-only migration for a behaviour change is a judgement call — if the TLC wants a full deprecation cycle instead, better to know before four more bindings copy the Python precedent.

2. Handle types. Decision 1 says add returns a handle object. Java returns long and Python returned str today. [py] Align driver.network handlers with ADR 17685 #18051 makes HandlerHandle subclass str so existing callers keep working, but Java can't do the equivalent trick as cheaply. Worth settling once.

There's also a scoping wrinkle: decision 11 scopes a handler to a window handle or a user context, but network.addIntercept has no userContexts parameter. #18051 intercepts everything and continues out-of-scope events untouched. Confirmation that this is the intended reading would help the other rows.

Process note

This issue should stay open when #18051 merges — it only covers the Python row; Java, Ruby, .NET and JavaScript are still outstanding.

(Python implementation and this summary were produced with AI assistance; I've reviewed both. The two defects above were found by running the suite against real Chrome and Firefox, not by inspection.)