[denera]

Description

Userbuffers and the cuBLASMp buffers for comm+GEMM overlap are shared by every layer, so a tensor saved for backward that aliases one is overwritten by the next layer's forward pass. A single layer cannot observe this numerically. The layer runner now checks every tensor saved during the test model's forward and backward passes and fails if its storage overlaps the buffer returned by get_buffer(). With the cuBLASMp backend, the check runs with cuBLASMp 0.11.0 or newer, where get_buffer() views the gathered input in its workspace (cuBLASMp buffer was not exposed via API in older versions).

The multi-layer TransformerLayer cases removed in #3692 were added in #1554 to catch this, but relied on numerical tolerances to detect it. This new test checks the root cause instead of relying on numerical failures.

Type of change

• Documentation change (change only to the documentation, either a fix or a new content)
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Infra/Build change
• Code refactoring

Checklist:

• I have read and followed the contributing guidelines
• The functionality is complete
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[greptile-apps]

Confidence Score: 5/5

[Low impact] The PR appears safe to merge; no actionable issues were found.

Summary

Adds a direct check for saved tensors that share memory with communication-overlap buffers.

• The overlap-layer runner checks saved tensors against shared buffers.

Review was source-only; GPU tests were not run.

_{Reviews (1) · Last reviewed commit: "[PyTorch] Check saved tensors against co..." · Reviewed by Greptile}