Skip to content

Unified error handling across the agent - #55

Merged
gvdongen merged 4 commits into
mainfrom
error-handling
Nov 17, 2026
Merged

gvdongen merged 4 commits into
mainfrom
error-handling

Conversation

@gvdongen

Copy link
Copy Markdown
Contributor

Unified error handling across the agent

Before this change, the code that caught an error decided what it meant. The same TerminalError became a tool_failed result in a tool, a guardrail_failed result in beforeTool, and failed the whole run in beforeStep. To predict whether an error stopped the agent, you had to know which code would catch it.

Now the error's type decides the outcome, wherever the run throws it: in a tool, an asyncTool, a handler a tool calls, a sub-agent, a fork, a guardrail, or a hook.

Error Who retries What happens inside a run
TerminalError, including ToolError The model The nearest agent gives the error to its model: as the tool call's result, or as a note before the model's next step.
AgentTerminalError Nobody The run fails, and so does every agent run that encloses it.
Any other error Restate Restate retries the step or the invocation, and by default pauses the invocation when its retries run out.

A TerminalError goes to the model because that lets tools compose without wrapping each other's errors, and lets the model react to a failure, for example by trying another approach after a timeout. AgentTerminalError stops the run from anywhere, so stopping never depends on where the error is caught. No configuration option changes what an error means.

Cancellation is the exception to the first row. Restate's CancelledError is a TerminalError, but it propagates from every location and never reaches the model.

Keep the meaning across journaled steps and RPC

A journaled step or an RPC rebuilds every failure as a plain TerminalError. The message, status code and metadata survive, but the class does not. AgentTerminalError and ToolError therefore store their meaning in the error's metadata (agentlib.kind and agentlib.code), and the run reads it from there. isAgentTerminal(error) performs the same check for handler code that receives errors over RPC.

Error metadata requires Restate 1.6 or later. An integration test verifies that the metadata arrives intact over an RPC, both directly and through an intermediate handler.

What the model receives

A failed tool call is recorded as {status: "error", code, message}, with the ToolError's code or tool_failed. A rejection by a guardrail or an approver is recorded with status denied. The other calls in the batch keep running, so a failed call costs no extra step.

A TerminalError from anywhere else in the run reaches the model as a note, a user message from the runtime before the model's next call:

{"source": "agent-runtime", "type": "error", "origin": "beforeStep", "code": "pii_detected", "message": "Remove the account number."}

Notes are provisional: the model sees them for the rest of the run, but they are never saved to the conversation or returned in result.messages.

Where a TerminalError is thrown What the run does Uses up the step
Any part of a tool call: beforeTool, afterTool, the tool's before and after, approvals, and anything the tool calls Records the call's error result No
beforeRun, thrown or returned as a rejection Adds a note after the input; the first model call goes ahead No
The instructions function, or beforeStep thrown or returned as a rejection Skips the step's model call and adds a note Yes
The model's response, when the model can fix it: cut off, filtered, empty, or announcing tool calls without any; two handoffs or duplicate call IDs in a batch; text over maxOutputChars Discards the response and adds a note Yes
afterRun, for a completed answer Gives the model its answer back with a note Yes
beforeSteering Rejects the steering message; the run continues —
afterRun for a stopped or interrupted result, the model call itself, loading or saving the conversation, listing skills, or the onProgress handler Fails the run, because no model step can follow or the next one would fail the same way —

Breaking changes

Guardrails and hooks

  • A TerminalError in a hook reaches the model. Thrown in beforeRun, the instructions function, beforeStep, or afterRun for a completed answer, it becomes a note instead of failing the run. In beforeSteering, it rejects the steering message. To stop the run, throw AgentTerminalError.
  • Guardrail rejections reach the model. When beforeRun or beforeStep returns false or a reason, the model receives the reason with code denied, instead of the run failing with a GuardrailError. A rejected input stays in the history, followed by the note. When afterRun rejects a completed answer, the model gets its answer back with the reason and another step, so write reasons the model can act on. Lesson 08 now does this.
  • An afterRun rejection of a stopped or interrupted result still fails the run, because no model step can follow. It now throws a ToolError with code denied instead of a GuardrailError.
  • The guardrail_failed code is removed. A TerminalError from beforeTool or afterTool is recorded as tool_failed, or with the ToolError's code, like a failure anywhere else in the tool call.

Error classes

  • ToolError now extends TerminalError. A remote handler can throw it without Restate retrying the child invocation, and its code reaches the calling agent's model. Its string code moved from code to toolCode, because code is the numeric status every TerminalError has. Read error.toolCode where you read error.code from a ToolError.
  • GuardrailError, AgentLimitError and ModelProtocolError are removed. RunFailedError remains as a deprecated alias of AgentTerminalError.

Model calls

  • Model calls have no attempt limit by default. journaledModel and aiModel no longer default to three attempts, so a failing model call follows the invocation's retry policy, which pauses the invocation by default. In aiModel, a failure that retrying can't fix, such as a missing API key or a rejected request, pauses the invocation at once. When retry.maxAttempts or retry.maxDuration is set, the call fails the run once its attempts run out, and an unfixable failure fails it immediately. Set retry.maxAttempts in handlers whose callers wait for a response, such as chat handlers.
  • A response problem the model can fix gives it another step, instead of failing the run. These problems are listed in the table above. The discarded response's tokens still count toward usage.

Mistakes in code are retryable

A terminal error would permanently fail an invocation that a deploy could fix. The SDK now throws a TypeError or RangeError for these mistakes, so Restate retries the invocation until a fix is deployed:

  • a hook result that isn't a decision, invalid beforeStep messages, or an empty replacement message;
  • afterRun output that fails the output schema or exceeds maxOutputChars;
  • tool progress or display information longer than maxOutputChars.

Two definition mistakes now fail when the code runs, usually at service startup:

  • skills() throws a TypeError for a skill with an invalid name, an empty description or a duplicate name.
  • agent() throws for invalid tool names and background handoff tools.

Other changes

  • A nested call deeper than maxToolDepth fails alone with code depth_limit, and the run continues. Before, it failed the run with an AgentLimitError.
  • A skill source other than skills() no longer fails the run on an invalid skill. The run skips a skill with an invalid name, an empty description or a duplicate name, and logs a warning. Of two skills with the same name, it keeps the first, and load receives that one.
  • A failed conversation compaction at the start of a run logs a warning, and the run continues with the history it loaded, the same way a failed in-run summary falls back to truncation.
  • An AgentTerminalError with errorCode: 413 is not a context overflow. Before, the run compacted the conversation and called the model again, and in a sub-agent the parent received tool_failed. This also applied to RunFailedError on main.

Forks

This branch is rebased on the fork tool from #54, and forks follow the same rules:

  • The fork tool's own errors, fork_limit, fork_nested and fork_unavailable, are ToolErrors, so the model receives them as tool results.
  • A fork copies the parent run's provisional notes as provisional.
  • An AgentTerminalError in a fork fails the run it was started from.
  • A fork still skips beforeRun and afterRun.

Documentation

  • A new references/error-handling.md in the restate-agents skill explains the three outcomes, what the model receives from each location, error codes, RPC behavior, cancellation, and what a failed run keeps. SKILL.md and the tools, guardrails, and models references link to it.
  • The READMEs, CONTEXT.md, the other skill references and lesson 08 describe the new behavior.

Testing

  • pnpm build: compile, lint, format and typecheck pass.
  • pnpm test: 337 tests pass. The new errors.test.ts covers each location above, AgentTerminalError from every site, an AgentTerminalError inside an asyncTool on replay, from a sub-agent and from a fork, an AgentTerminalError with code 413, cancellation from every site, a TimeoutError in a tool, and depth limits. skills.test.ts covers skills() rejecting invalid and duplicate skills, and a custom source's duplicate loading the listed skill.
  • pnpm test:integration: 72 tests pass, including error metadata over an RPC and a remote handler's AgentTerminalError and ToolError reaching an agent, directly and through a relay.

🤖 Generated with Claude Code

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ One low-impact edge case inline.

Reviewed changes This review covers the unified error-handling changes across the agent runtime, model/provider adapters, tools, skills, tests, and documentation.

  • Error outcomes: TerminalError, AgentTerminalError, and retryable errors now have consistent behavior across hooks, tools, nested agents, forks, journaled steps, and RPC.
  • Model recovery: Rejections and recoverable response problems become provisional model feedback; provider retry behavior, output limits, and usage accounting are updated.
  • Validation and fallbacks: Tool batches and skill definitions receive stricter validation, invalid external skills are skipped, and compaction/depth-limit fallbacks are adjusted.
  • Coverage and documentation: The PR adds error-path and RPC tests and updates examples and the Restate Agents references.

Important

This run used your free trial credits ($9.28 of $10 left) on GPT Luna, a fast, lightweight model. Connect a provider → to run the model you choose.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using gpt-6-luna (free trial credits) | 𝕏

Comment thread packages/core/src/skills.ts Resolved

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes This incremental review covers the follow-up changes since the prior Pullfrog review.

  • Skill validation: Reserved a skill name only after that skill passes validation, allowing a later valid entry to survive an invalid duplicate.
  • Guardrail errors: Added GuardrailError as a ToolError, classifying policy failures and rejections with guardrail-specific codes.
  • Progress reporting: Added error-note events and forwarded sub-agent and fork progress through their parent tool calls.
  • Model retries: Changed the default attempt limit for journaled and AI SDK model calls to three attempts.

Important

This run used your free trial credits ($9.22 of $10 left) on GPT Luna, a fast, lightweight model. Connect a provider → to run the model you choose.

Pullfrog  | View workflow run | Using gpt-6-luna (free trial credits) | 𝕏

@gvdongen
gvdongen merged commit e65eea9 into main Nov 17, 2026
4 checks passed
@gvdongen
gvdongen deleted the error-handling branch November 17, 2026 12:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant