Repository navigation
Conversation
…eset() Signed-off-by: lxcxjxhx <lxcxjxhx@users.noreply.github.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #3858.
FP8GlobalStateManager.reset()replaces the process-global state, but a module's registration marker (fp8_meta["buffer_index_and_autocast_key"]) is stored on the module, so it survives the reset.add_fp8_tensors_to_global_buffer()only tested for the marker's presence, so a module reused after a reset never registered into the replacementglobal_amax_buffer. Its amax was still collected locally while nothing reachedreduce_and_update_fp8_tensors(), and its delayed-scaling scale stopped adapting: with weights of1/16and sixteen inputs of128, the output stayed at1.0instead of128.0on every step.The state now carries a generation that
reset()increments, and a marker is only trusted while it belongs to the current generation:Otherwise the stale marker is cleared and the module registers again with the new buffers, which is the semantics the existing comment already claimed ("exactly once per global state").
Two consequences worth naming:
buffer_generationis bookkeeping, not FP8 tensor state, so it is excluded from the pickled extra state alongside the registration marker.FP8GlobalStateManager.BOOKKEEPING_KEYSis now the single place listing those keys, used byget_extra_stateinmodule/base.pyandops/op.py. Without that, checkpoints would grow an internal key whose value is meaningless in the process that loads them.Testing
What I ran
black --line-length 100 --checkwith the version the repository pins (24.4.2) on all four touched files: unchanged. (Black 26 wants an unrelated blank line after the module docstring; that reformat is deliberately not included.)python -m py_compileon the touched files.pylint==3.3.1ontransformer_engine/pytorch/quantization.py: 9.93/10, the only message beingE0611fortransformer_engine.common, an artefact of not having the extension installed here.What I did not run
This host has no CUDA toolkit and no built
transformer_engine_torch, so the new test and the reporter's reproduction script could not be executed here. The PR is opened as a draft for that reason, and perAGENTS.mdit should stay a draft until its human author has reviewed it. A/te-cirun is what would confirm it.The test (
tests/pytorch/test_sanity.py::test_delayed_scaling_adapts_after_global_state_reset) follows the reporter's sequence with fewer moving parts: warm up at amplitude 1,reset(), assert the global buffer is empty, run one step to force re-registration, then two steps at amplitude 128. It asserts2registered entries after the reuse (the failure mode is0), that the first step after the jump still uses the stale scale, and that the second one reaches the reference mean of 128. Before the change the same sequence reports0, 0, 0entries and an output mean of1for every step, which is the table in #3858.Type of change
Checklist
blackat line length 100 and pass the touched-file lint.git commit -s).qa/L0_pytorch_unittest/test.shand theL0_*checks have been run — not possible on this host (no GPU, no built extension); this needs the/te-cirun.Additional information
Scope is limited to the registration guard and the two places that enumerate
fp8_metakeys for serialization.init_fp8_metadataand the recipe-state reuse path are untouched: an existing module keeps its own scale and amax tensors, which is what makes reuse meaningful.