feat(eval): ondemand simulate — replay a dataset, evaluate synchronously - #2071

Draft

jariy17 wants to merge 5 commits into

feat/eval-ondemand-simulate

Commits on Aug 19, 2026

feat(eval): batch-evaluation simulate — invokeDataset + self-running example classes
jariy17
committed
b747257

Commits on Aug 20, 2026

refactor(eval): address Harrison's review feedback
jariy17
committed
e3e3fc1

Commits on Aug 22, 2026

fix(eval): validate assertions/expected_trajectory + require {input} in payload-template
jariy17
committed
708c074
feat(eval): add `ondemand simulate` — replay a dataset, evaluate synchronously
jariy17
committed
8ff2657
test(eval): ondemand simulate uses matchGolden committed goldens (runtime-handler pattern)
jariy17
committed
94a16ac