Skip to content

Add ExactlyOnceEvaluator #226

Description

@ptomecek

Add an evaluator that enforces exactly-once execution semantics for CallableModels that perform external side effects (sending notifications, posting webhooks, writing to external systems, kicking off downstream jobs, etc.).

This is distinct from durable caching. Durable caching is an optimization — a missing entry simply means recompute. Exactly-once is a correctness guarantee — a missing ledger entry means "the side effect has not happened, run it now," and a present entry means "do not run it again under any circumstances." Clearing the ledger has real-world consequences and should not be casual.

Behavior

  • Maintain a durable ledger of completed (model, context) invocations.
  • On invocation, check the ledger; if present, skip execution; if absent, execute and record completion atomically.
  • Pluggable ledger backend.

Open questions

  • Return value on skip. When a call is skipped because the ledger already records completion, what does the evaluator return? Options include a stored minimal acknowledgment, a typed sentinel, or raising. Each has tradeoffs for downstream composition.
  • Atomicity. The side effect and the ledger write are not generally co-transactional. How do we minimize the window where the side effect has happened but the ledger has not yet been updated, and what's the recommended user-side pattern when this matters?
  • Relationship to durable caching. Should the two evaluators share a storage abstraction, or stay deliberately separate to keep the semantics from blurring?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions