Skip to content

[provider-anthropic] fallback_cooldown_seconds: 0 reports a fallback window it never opened, masking the overload error #386

Description

@Joi

fallback_cooldown_seconds accepts 0, but at 0 the overload-fallback path reports success without doing anything, and the caller then raises a RuntimeError that replaces the real overload error.

Verified against microsoft/amplifier-module-provider-anthropic@833403b (current main). Not reproduced live — this is read from the control flow.

The path

  1. fallback_cooldown_seconds: 0 survives config load. The clamp is max(0.0, ...), so 0 is a legal value, not a rejected one:

    self._fallback_cooldown_seconds = max(
        0.0, self._config_float(self.config.get("fallback_cooldown_seconds", 300.0), 300.0)
    )
  2. An overload error reaches _open_fallback_window. It builds the window, then stores it only when the cooldown is positive — but returns True unconditionally:

    if self._fallback_cooldown_seconds > 0:
        _set_fallback_window(family, window)
        self._write_shared_fallback_state(family, window)
    ...
    return True
  3. The caller in complete() reads True as "a downgrade window is open, go around again":

    if not await self._open_fallback_window(effective_model, e):
        raise
  4. Next iteration, _resolve_effective_model finds no window and returns the same model. It is already in attempted_models, and full_retry_budget_used was not touched (that branch requires not self._is_overload_fallback_error(e)), so the loop guard fires:

    raise RuntimeError(f"Overload fallback loop detected while resolving {requested_model}")

The operator set a config key to zero and got Overload fallback loop detected instead of the overload error. The API error that actually caused it is gone.

Suggested fix, roughly in order of preference:

  • Return False when self._fallback_cooldown_seconds <= 0 — no window was opened, so the caller should re-raise the original error, which is the correct outcome.
  • Or reject 0 at config load with a warning and fall back to the 300s default, if a zero cooldown is not meant to be a way of disabling fallback.
  • Or, if 0 is meant to mean "downgrade once, no persistent window", make that explicit — the current code does not implement it either way.

Found while reviewing this provider against a local fork; the fork does not carry a fix for this, since the fix depends on which of the three readings is intended.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions