[xingjiangjun]

Summary

Add native NCCL/MCCL Reduce support through the existing shared CCL framework.

This PR is a follow-up to #66, originally contributed by @GordonYang1. It rebases and refines the original implementation and aligns the examples with the existing MPI Reduce and CCL/Hybrid examples.

The diff contains eight files. MPI implementations, MPI examples, and shared example utilities are unchanged. Zero-count Reduce now returns success before argument validation, matching the existing AllGather convention.

Changes

• Implementation
  • Share Reduce logic in src/backends/ccl/common/impl/reduce.h, with thin NCCL/MCCL registration and provider forwarding.
  • Require a matching native communicator; do not fall back to OpenMPI from the native implementation.
• Examples
  • Add Native and Hybrid examples using the existing Validator and Metrics utilities.
  • Bootstrap Hybrid communicators through host Broadcast; allocate and validate receive buffers only on root, and report validation failure through the exit status.
• Validation
  • Handle zero count before HasInvalidArgs() while retaining root-specific buffer checks for nonzero counts.

Platform and Backend Affected

Platform

• CPU
• NVIDIA GPU
• Iluvatar GPU
• MetaX GPU
• Moore Threads GPU
• Cambricon MLU
• HYGON DCU
• T-Head PPU
• Ascend NPU

The shared zero-count validation change also applies to other Reduce backend/device combinations; only the checked native platforms and OpenMPI paths were exercised. NCCL/RCCL is checked for NCCL only.

Backend

• OpenMPI
• MPICH
• NCCL/RCCL
• MCCL
• CNCL
• HCCL

Performance Impact

• No performance impact
• Performance improved
• Performance regression possible

N/A — functionality change, not a performance optimization. Shared-server timings are not performance comparisons.

Known Issues & Future Work

• Existing MPI FP16/BF16 limitations remain unchanged; no new MPI half-precision support is claimed.
• Examples reuse the existing Validator and Metrics without changing their NaN handling or generic bus-bandwidth formula.
• RCCL and other devices, all datatypes/operations, and RDMA/RoCE are not fully validated.

Test Results

Environment	Checks	Result
NVIDIA A100 / NCCL	Native 1 rank, count=1; 2 ranks, count=16; Float32 SUM	PASS
NVIDIA / OpenMPI + NCCL	Hybrid 2 ranks, Float32 SUM, count=1048576; host Broadcast bootstrap	PASS
NVIDIA / NCCL	Nonzero root, root in-place at roots 0/1, non-root null receive buffer; Int32 SUM/Avg, FP16/BF16 SUM, count=4	PASS
NVIDIA	Eight zero-count/invalid-argument checks; four internal native-communicator rejection checks	PASS
NVIDIA	AllReduce, ReduceScatter, MPI-only Reduce regression	3/3 PASS
MetaX C550 / MCCL + OpenMPI	Native 1/2 ranks, count=16; Hybrid 2 ranks, count=1048576; Float32 SUM, root=0	3/3 PASS
NVIDIA + MetaX / OpenMPI 4.1.6	MPI Reduce, 2+2 ranks, Float32 SUM, root=0, count=1048576; expected/actual=10	PASS
Build / format	NCCL-only, NCCL+MPI, MCCL+MPI, two-platform MPI-only; clang-format 16.0.6; git diff --check	PASS

All reported test processes exited with code 0 and produced the expected results or error statuses. Heterogeneous communication used UCX/TCP and the existing icclrun configuration, not RDMA. Targeted examples were run directly with timeouts rather than through the full run_examples.py suite. Tests used the working-tree snapshot whose source manifest was checked against the final commit; they were performed before commit creation, not rerun under the final SHA.

Final commit: db69aad2e4ba200d2804d2047960276c207033e1.

Sanitized test logs and standalone test sources are attached. Full originals are archived locally.

Test Involved Platform

• CPU
• NVIDIA GPU
• Iluvatar GPU
• MetaX GPU
• Moore Threads GPU
• Cambricon MLU
• HYGON DCU
• T-Head PPU
• Ascend NPU

Test Involved Backend

• OpenMPI
• MPICH
• NCCL/RCCL
• MCCL
• CNCL
• HCCL

---

Checklist

Every contributor must verify every item below before requesting
review. Tick each box only after the check has actually been performed —
do not tick speculatively. If an item truly does not apply, replace the
checkbox with N/A and briefly explain why in an inline comment.

Title, Branch, and Commits

• PR title follows Conventional Commits (e.g. feat: …, fix(nccl): …).
• Branch name follows <type>/xxx-yyyy-zzzz where <type> matches the PR title's Conventional Commits type and words are joined with hyphens (see CONTRIBUTING.md §Branches).
• Each commit message follows Conventional Commits.
• Small PR is a single squashable commit; or, for a large PR, every commit is meaningful, well-formed, and independently reviewable (see CONTRIBUTING.md §Pull Requests).
• No stray merge commits from master — the branch is rebased cleanly on top of the current master.
• No fixup! / squash! / wip commits remain.

Scope and Design

• Changes are minimal — no unrelated modifications were introduced (CONTRIBUTING.md §Code/General).
• No dead code, commented-out blocks, debug prints, printf/std::cout/print(...) left behind, or TODO without an owner and issue link.
• No unrelated formatting churn that would obscure the diff.
• Public API changes (if any) are intentional, documented, and reflected in affected callers/tests.

General Code Hygiene

• The code is self-explanatory; comments were added only where the intent or rationale is non-obvious (CONTRIBUTING.md §Code/General).
• Every modified or added file ends with a single trailing newline (CONTRIBUTING.md §Code/General).
• No trailing whitespace, inconsistent indentation, or mixed formatting styles remain.
• Identifiers referenced in comments or error messages are wrapped in Markdown backticks (e.g. the `AllReduce` implementation) (CONTRIBUTING.md §Code/General).
• All comments and error messages are in English (CONTRIBUTING.md §Code/General).
• Comments and error messages are complete sentences — capitalized first letter, terminal punctuation — unless the language/framework convention says otherwise (CONTRIBUTING.md §Code/General; §Python).

C++ Specific (if C++ files changed)

• Code follows the Google C++ Style Guide strictly.
• clang-format (version 16, per .github/workflows/clang-format.yml) has been run against all modified applicable files; the diff is clean.
• No exceptions are thrown. Error paths use assert with messages that include at least __FILE__, __LINE__, and __func__ (CONTRIBUTING.md §C++).
• Error and warning message wording follows the LLVM Coding Standards (CONTRIBUTING.md §C++).
• Constructor initializer list order matches member declaration order (CONTRIBUTING.md §C++).
• Exactly one blank line between classes, between classes and functions, and between functions (CONTRIBUTING.md §C++).
• Exactly one blank line between members (functions and variables) within a class (CONTRIBUTING.md §C++).
• Exactly one blank line before and after the contents of a namespace (CONTRIBUTING.md §C++).

Python Specific (if Python files changed)

N/A — no Python files changed.

Testing

• All applicable example programs have been built and tested successfully on at least one supported heterogeneous cluster setup.

Build, CI, and Tooling

No new backend or device is introduced.

• New backends or devices have been added to auto-detection in CMakeLists.txt under if(AUTO_DETECT_DEVICES) or to if(AUTO_DETECT_BACKENDS) if applicable.
• Both CI workflows (clang-format.yml, ruff.yml) are green locally (or expected to be green on CI).

Documentation

• README.md, CONTRIBUTING.md, or inline docs updated when behavior, build flags, or developer workflow changed.
• Any user-visible breaking change is called out explicitly under "Summary" and in the commit/PR title with a ! or BREAKING CHANGE: footer.

Security and Safety

• No secrets, access tokens, internal URLs, customer data, or personal hardware identifiers have been committed.
• Third-party code is license-compatible and attributed.
• No unsafe pointer arithmetic, uninitialized reads, or missing bounds checks were introduced.

@Ziminli, could you please review this follow-up to #66? The implementation and examples follow the existing CCL framework and the guidance from the AllToAll review. Thank you.