fix(mistral): freeze text-only visual branches and enable resume - #3964
Merged
Merged
Conversation
Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>
Contributor
Author
|
/ok to test df3476d |
athitten
approved these changes
Sep 21, 2026
athitten
left a comment
Contributor
There was a problem hiding this comment.
Thank you @yuhezhang-ai !
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
The Ministral3 SQuAD recipes adapt only the language model, but full fine-tuning still includes the unused vision tower and its sibling multimodal projector in Adam. Freeze both explicitly with the existing module-selector config and re-enable checkpoint-resume validation for both full and PEFT training.
PR #3635 already materializes missing Adam state before saving new checkpoints. Explicit freezing additionally keeps permanently inactive visual parameters out of the optimizer. PEFT already freezes these branches through its base-model policy and LoRA exclusions; the explicit selectors document and enforce the text-only recipe policy in both examples.
Changelog
freeze_config.freeze_modulestomodel.vision_towerandmodel.multi_modal_projectorin both Ministral3 SQuAD recipes.ci.checkpoint_robustness.skip_resumeexemptions and their stale workaround comments.Validation
git diff --checkpassed.resumefor both recipes and retaincross_tp_reloadfor full fine-tuning.df3476d0f5c05a1fbe407b5371b14790aef71e24: pipeline 69061370. Both recipes ran on 8 H100 GPUs with the shipped CI overrides and resume enabled.Before this PR is ready for review
Related to AMINT-9.