Repository navigation
Conversation
Signed-off-by: Layali Rashid <lrashid@nvidia.com>
github-actions
Bot
added
the
community-contribution
for more information, see https://pre-commit.ci
Signed-off-by: Layali Rashid <lrashid@nvidia.com>
Signed-off-by: Layali Rashid <lrashid@nvidia.com>
Signed-off-by: Layali Rashid <lrashid@nvidia.com>
Signed-off-by: Layali Rashid <lrashid@nvidia.com>
Signed-off-by: Layali Rashid <lrashid@nvidia.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Quantize aligned BF16 column segments directly into MXFP8 GEMM layouts, fold optional distributed mean scaling into weight-gradient GEMMs, and skip zero gradients for unused normalization outputs.
Candidate performance on GB300: projection backward with mean scaling was 1.067x faster (8.358 → 7.836 ms). Combined with the Megatron changes, the 41-layer, 64-GPU workload gained 1.50% E2E throughput (1503.26 → 1481.11 ms/step), from repeated ABBA runs on the same GPUs. The repository ports have not been retimed E2E.
Validation: 222 consolidated GPU tests passed on GB300 after rebuilding the merged source; all public CI checks pass.