Repository navigation
[None][feat] agent-flow: modeling-v2-onboard planner/worker workflow - #20792
Draft
WeiHaocheng wants to merge 27 commits into
Draft
WeiHaocheng wants to merge 27 commits into
WeiHaocheng wants to merge 27 commits into
Conversation
…-flow A two-role agent-flow workflow for bringing up ModelingV2 models and targets, built on the concurrent orchestration engine. - The planner writes plan.md: one section per task, tagged new_op / target / merge / module_accuracy / e2e_accuracy, plus an Execution Graph of their dependencies. The plan is validated (flat graph, known tags, multi-dependency tasks are merge tasks, a single shared final task) and errors are sent back to the planner. - Workers run the tasks in parallel through NodeScheduler, each in its own git worktree, with a system prompt composed from the task's tags, and end by writing task_result_<id>.md. - A FAILED task pauses the run; the planner revises the plan (REVISED / DONE / ABORT), the failed and changed tasks are reset, and the build resumes. There is no round limit. - Task branches are kept until the run ends so a re-run task still starts from its dependencies' code (the scheduler otherwise reclaims a dependency branch once its only dependent has failed). - --backend switches every agent between claude-code (default) and codex. Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
… perf tag The planner is told --max-parallel and asked to keep that many workers busy: independent work side by side, feature work alongside the accuracy gate, and perf tasks (alternative implementations, new kernels, fusion) in the remaining slots. The new perf tag briefs a worker to measure, adopt only a measured win, and revert otherwise. Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
Replace the plan/build/replan phases with an event loop of its own: every task whose dependencies are DONE starts at once (up to --max-parallel), and whenever workers finish the planner gets a turn while the others keep running. A turn may add tasks, rewrite or remove tasks not yet started, retry/rewrite/remove FAILED tasks, cancel RUNNING ones, FINISH (release the final task) or ABORT. A FAILED task only holds back its dependents. DONE and RUNNING tasks are frozen, including their plan.md section. An invalid turn is discarded after three fixes and reported on the next one; three stalled turns in a row stop the run. Task states and the running plan are checkpointed, so a rerun resumes and restarts running tasks on their branches. Also fix task-section lookup matching a prefix id (gsm8k vs gsm8k-final). Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
A fresh worktree has no compiled artifacts. When the brief or the shared command cache gives a way to link a built checkout's artifacts, workers use it instead of a full build_wheel per worktree. Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
Each execution-graph node may carry priority: accuracy | feature | perf, its claim on shared GPUs. Left out, perf-tagged tasks are perf, the path to the first whole-model accuracy result is accuracy, and the rest is feature. The orchestrator publishes the running plan's classes in gpu_priorities.json for a GPU pool to serve; changing a class never re-runs a task. Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
…irst gate on a target A perf task the plan puts on the way to the first whole-model gate is on the critical path, so it is accuracy too. The first gates are the e2e_accuracy tasks that build on a target task with no such gate upstream. Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
…, safe git ops - --feedback / --feedback-file leave a message in <workspace>/inbox/. The orchestrator polls the inbox, wakes the planner, shows the messages first in its turn prompt, and moves them to inbox/consumed/ only once that turn applied. Tasks that finished since the last turn are kept in .state.json, so a restart no longer drops them. - The planner fills free slots only with completion-criteria work, then the features task.yaml asks for, then perf work; unrequested verification or polishing (other models' suites, upstreaming tests) is ruled out because it holds up landing. Human messages override the plan. - Git worktree operations run one at a time and are never interrupted by a cancellation: concurrent `git worktree add` raced on .git metadata, and a cancelled add left a locked worktree that could not be removed. Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
…nstead of chasing them Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
…gent ready task Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
…t merge Once task.yaml's goals are met, the planner marks the unmerged perf tasks side: true so the run can land without another round of merging. Side tasks keep using free slots until the final task is DONE, get --side-wait-minutes to finish afterwards, and are kept as <landed branch>-unmerged/<id> branches listed in a committed UNMERGED_OPTIMIZATIONS.md. A human can still ask for side work to be merged: the planner drops side and merges it before the final task, or after it through a chain of shared landing tasks. A cancelled shared task's changes to the checkout are undone. Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
… the plan The planner writes the tasks it can define before the deep reading (the execution setup, contract reviews, references, ops with no entry at all) and ends its first turn with planning: true. Workers start on that batch while the planner keeps adding batches, one turn after another, until it writes planning: false. Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
…ner plans With slurm-environment.gpu_pool in task.yaml, the orchestrator keeps holder jobs for whole nodes submitted from the start of the run and cancels them at its end. Workers run GPU commands through gpu_run, which locks GPUs on a holder job, serves waiters by their task's priority class and runs the command as a step in the container. Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
…er session A primer session reads what every new_op and perf worker needs (the entry contract, exemplar entries and tests, how GPU tests run) while the planner plans, and writes a summary. Those workers start as forks of its session; when it is not ready within 20 minutes, failed, or a fork cannot start, they start afresh and read the summary. agent-flow's sessions can now fork an earlier session (Claude backend) and report their session id. Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
…d task, not a pool setup hook Drop gpu_pool.setup. Preparing the container environment (reusing a build, the Python environment, model paths) is now a first-batch task: a worker writes the wrapper into the workspace and records it in test_commands.md, and the other tasks run their GPU commands through it. Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
…e auto-memory Claude Code keeps one auto-memory per repository, shared by every worktree of the checkout, so a run's agents read memories of unrelated work and of earlier runs, and left their own to later runs. The CLI now disables it for every agent it starts. Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
…the first batch The first batch is written from task.yaml, config.json and the ModelingV2 README alone; the reference code, the safetensors index, the catalog and the engine source wait for the second batch. Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
…s are placed per node gpu_run places each waiter ahead of a command on a node first; a waiter that fits nowhere claims the node with the most free GPUs, so one-GPU commands can no longer starve a whole-node command. Each planner turn shows the pool, and the turn file's gpu_nodes grows it up to gpu_pool.max_nodes (default: the worker count). Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
…grows in any phase A merge task depends only on the tasks whose code it joins; module checks, accuracy gates, benchmarks and profiles run beside the next step, and a failed check gets a fix task and a new join. The planner's example plan follows the rule, and the default priorities find the first gates by the code they check rather than by gate-to-gate edges. The planner may grow the GPU pool whenever commands wait for it, in any phase and for any class of work. Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
…tops the run Cancelling a task read its jobs.json as the job registry and raised when the worker had written the file itself in another shape, ending the run. The orchestrator now logs and skips such a file, and the worker prompt says the file belongs to the run_detached wrapper. Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
A merge node's worktree merges its dependencies' branches with git merge. A TensorRT-LLM checkout with its pre-commit hooks installed runs a DCO check on every commit message, so the unsigned merge commit was refused and the run stopped. The merges now carry a sign-off. Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
…worker's work The per-node output commit is now signed off, so a node branch passes a repository's DCO check. When the run stops, each running worktree task's changes are committed to its branch before the worktree is removed, so a resumed task starts from them instead of from its last commit. Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
…mmand recipes The worker prompt's GPU pool, Slurm and detached-job blocks now say what must hold and leave the exact commands to the worker. The detached-job block no longer assumes Slurm jobs in a pool run. Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
…urm job check in the worker prompt Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
A worktree directory that git no longer counts as a worktree (its .git file is gone) is now removed as plain files instead of failing the release. The tasks a planner turn resets are saved in the run state until each one is dropped, so a run stopped in the middle of a reset finishes it when it resumes instead of leaving the task's branch, notes and jobs behind. Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
…mands it started A local detached job leads its own session; its cancel now signals the whole process group, so a GPU command it started does not keep running after the task is reset. Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
Cancelling a running task now only stops its worker's session. Its worktree, branch, notes and GPU jobs are left as they are; the task must leave the plan in the same turn and its id is never used again, so redoing the work is a task with a new id. A FAILED task removed from the plan is abandoned the same way, and every abandoned task is listed in abandoned.md for a human to clean up. The final task, which works in the checkout itself, cannot be cancelled. A retried or rewritten FAILED task carries on from its own branch; its last result is kept as an earlier attempt. The reset machinery (job cancellation, branch and worktree removal, checkout restore, and finishing an interrupted reset on resume) is gone. Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
…ne bit for bit A new implementation, kernel or fusion may accumulate in another order and differ from the baseline in the last bits. The perf worker prompt accepts that while the error stays as small as the arithmetic explains (the entry still passes its band against its reference), and the planner no longer asks perf tasks for bitwise identity. Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
1 task done
Wanli-Jiang
added a commit
to Wanli-Jiang/TensorRT-LLM
that referenced
this pull request
Oct 24, 2026
…s and merges, cancel a local job's process group Agent-flow changes the trtllm_modeling_v2 workflow builds on, the first three ported from NVIDIA#20792: - A session can start as a fork of an earlier one (SessionConfig.fork_from): the Claude Code backend resumes the earlier session with fork_session, and the Codex backend refuses with NotImplementedError. A turn's result carries the backend's session id, and the session layer keeps the last one. - Node commits and merges in git worktrees are signed off, for repositories that require DCO. - Cancelling a local job sends SIGTERM to its whole process group, so the commands it started stop with it; a pid that leads no group gets the signal alone. - The Codex backend launches $CODEX_BIN, else the codex on PATH when it is newer than the pinned runtime (a login offers a new model only to a client that knows it), and a session given no model runs the one Codex is configured with. Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
Collaborator
|
Hi, heads-up: #22286 has merged, so ModelingV2 now lives at Since this PR touches modeling_v2, please rebase it onto the latest main. You can use these migration commands to move any files you added under the old directory and update any remaining references to the old path. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
@coderabbitai summary
Description
Adds
modeling-v2-onboard, an agent-flow workflow that brings a new model up on ModelingV2 with one planner and parallel workers, and the changes that came out of running it end to end on nemotron4-nano-preview (GB300).Orchestration
The planner writes
plan.md: tasks taggednew_op/target/merge/module_accuracy/e2e_accuracy/perf, plus an Execution Graph. Workers run tasks in parallel, each in its own git worktree, with a system prompt built from the task's tags.An event loop replaces the plan / build / replan phases. A task starts as soon as its dependencies are done, and the planner takes a turn whenever workers finish while the others keep running. A failure blocks only its dependents. State is checkpointed, so a run resumes after a crash.
The planner releases a first batch of tasks before it reads the code, then adds batches turn by turn.
Scheduling follows a priority class on each task. Free worker slots and pool GPUs both go to the most urgent ready task, in this order:
accuracy: everything on the way to the first whole-model accuracy result. That means the missingnew_opentries, the module checks, assembling the target and the first gate.feature: further capabilities and their checks.perf.The class follows what a task unblocks, not its tag, so a
new_opthat the first target needs isaccuracy. The planner can change a task's class in any turn. Resumed tasks go first, and the graph order breaks ties within a class.Checks never block joins: a failed check gets a fix task and a new join.
GPU resources
slurm-environment.gpu_pool). Holder jobs each hold a whole node and are submitted at start, so Slurm queueing overlaps planning.gpu_runlocks GPUs withflockand runs each command as a step of a holder job inside the container. Waiting commands are served by the same priority classes, re-read on every poll. A few GPUs are reserved andperfwork may not take them.Worker start-up cost
new_opandperfworkers start as forks of a primer session that has already read the entry contract and exemplars.test_commands.md.Human control and scope
--feedback/--feedback-filemessages wake the planner and override the plan in any phase.issues.mdinstead of being chased.sidework that does not hold up landing. It is kept on-unmerged/<id>branches and listed inUNMERGED_OPTIMIZATIONS.md.Robustness
git worktreeoperations are serialized and cannot be interrupted.abandoned.md. The reset machinery is removed.Prompt rules
End-to-end result. nemotron4-nano-preview on GB300:
Test Coverage
tests/workflows/modeling_v2_onboard/: 111 tests, all passing. They cover plan validation, the event loop and planner turns, cancellation and abandonment, resume, priorities, the GPU pool andgpu_runplacement, side tasks, and the inbox.PR Checklist
Please review the following before submitting your PR:
PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either
api-compatibleorapi-breaking. Forapi-breaking, includeBREAKINGin the PR title.Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.
GitHub Bot Help
To see a list of available CI bot commands, please comment
/bot help.🤖 Generated with Claude Code