Skip to content

Add enabling lazy state to skills for long conversations - #57

Open
gvdongen wants to merge 3 commits into
mainfrom
document-lazy-state
Open

gvdongen wants to merge 3 commits into
mainfrom
document-lazy-state

Conversation

@gvdongen

Copy link
Copy Markdown
Contributor

Agents with long conversations should use Restate's lazy state instead of eager state.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

The repository's canonical LazyStateConversation example still eagerly preloads all state.

Approach: sound — disabling eager preloading lets each invocation fetch only the state keys it reads, which is important for chunked conversation history.

Reviewed changes I reviewed the lazy-state guidance added to the Restate agent skills.

  • Conversation guidance: Explains object-level and handler-level lazy state, selective preloading, and the tradeoff of per-key reads.
  • SDK guidance: Documents the state preload options and when to use them.
  • Checklist: Adds a reminder to disable preloading for objects that store LazyStateConversation.

Important

This run used your free trial credits ($9.18 of $10 left) on GPT Luna, a fast, lightweight model. Connect a provider → to run the model you choose.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using gpt-6-luna (free trial credits) | 𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ The incremental changes introduce no new issues.

Reviewed changes I reviewed the updated lazy-state guidance since the prior Pullfrog review.

  • Conversation guidance: Limited the preload: false recommendation to long or large histories and explained the serverless suspension tradeoff.
  • SDK guidance: Added object-level configuration guidance and kept eager preloading as the default for small state.
  • Checklist: Updated the reminder to apply lazy state selectively and linked the supporting reference.

Important

This run used your free trial credits ($9.17 of $10 left) on GPT Luna, a fast, lightweight model. Connect a provider → to run the model you choose.

Pullfrog  | Fix it ➔ | View workflow run | Using gpt-6-luna (free trial credits) | 𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Approach: sound — selective preloading keeps conversation chunks lazy while avoiding lazy reads for the metadata and small session keys.

Reviewed changes I reviewed the selective-preloading update since the prior Pullfrog review.

  • Canonical example: Added object-level preloading for conversation/meta, active, and approval, leaving conversation chunks to be fetched on demand.
  • Conversation guidance: Replaced the all-or-nothing recommendation with a key whitelist and documented metadata key customization, per-key costs, serverless suspensions, and the server-version fallback.
  • Skill guidance: Updated the checklist and SDK reference to recommend selective preloading for long chunked conversations while preserving eager preloading for short state.

Important

This run used your free trial credits ($9.16 of $10 left) on GPT Luna, a fast, lightweight model. Connect a provider → to run the model you choose.

Pullfrog  | View workflow run | Using gpt-6-luna (free trial credits) | 𝕏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant