[Ringssss]

Summary

• Register the text-only clover_moe model using the existing Qwen3.5-MoE modules, strict checkpoint mapping, FP32 recurrent state and native InfiniOps providers.
• Separate read-only block preview from causal replay/commit. Preview uses bidirectional GDN/convolution and Clover block coreset routing; unresolved masks are never committed.
• Add an optional frontier stride decoder and expert-budget/live-row reuse, a serial OpenAI chat endpoint, a portable paired benchmark client, controlled workload, tests and build instructions.
• Pin the matching InfiniCore dependency stack.

Motivation

Clover-35B-A3B-Preview was unsupported by the native Hygon stack. This runs full checkpoint inference in InfiniLM C++ on two Hygon BW 64GB DCUs. Torch is used for token-confidence selection; model forward does not use a Torch/Transformers or SGLang fallback.

Dependencies, in order:

1. feat(hygon): expose driver modules and device attributes InfiniRT#52
2. feat(hygon): add native Clover diffusion and sparse MoE operators InfiniOps#1015
3. chore: pin Hygon Clover runtime and operator support InfiniCore#1579

The dLLM-serve adaptation is the frontier/stride mechanism, with a confidence floor and immediate progress for already-confident positions. The supplied dllm-h100 study informs top-4 preview routing and TV6 live-row reuse with refresh interval five. This PR does not claim the full admission/compaction scheduler or the H100 DP/EP communication pipeline. The endpoint currently serializes requests; that boundary is explicit in the guide.

Type of Change

• feat — new model and decoding path
• test — checkpoint mapping, frontier and full-model validation
• docs — supported configuration and reproduction
• build — dependency pin / optional decoder dependencies

Test Results of Involved Models on Supported Platforms

Hygon BW/gfx936, DTK 26.04, Torch 2.10.0, Triton 3.5.0, BF16 TP2, eager static cache, 2048 context capacity:

Validation	Result
Native build	Repository xmake release build and installation passed; a separate CMake validation build also passed
Full checkpoint	All 28 safetensors files loaded, including learned preview shards; real Chinese/English/code generation
Checkpoint/runtime/host regression	24 tests and 61 subtests passed
Clover-specific tests	3 tests passed; final PR build also passed 24 additional HTTP requests with 24/24 structural/answer checks
Companion InfiniRT	13 tests passed
Companion InfiniOps focused tests	62 passed
Companion CPU/Hygon Ops smoke	121 passed, 43 inactive-provider skips, 9485 deselected
Serving	All 96 requests per variant completed successfully in the common OpenAI chat campaign
Controlled answer/structure checks	Joint 96/96; expert-only 93/96; baseline/frontier/local SGLang 90/96
Dispatch evidence	Both TP devices contain native Clover kernels; live-row kernel only in joint mode

The native baseline matched local SGLang token IDs on the four initial prompts. Other inputs can differ across the kernels' floating-point paths. These fixture checks do not establish general accuracy equivalence. The lower confidence floors 0.3/0.5 were rejected after repetition/invalid Python appeared.

This is a rendered record of measured test output, not a terminal screenshot. The original request/response logs and traces are retained with the local integration artifacts.

Benchmark / Performance Impact

Same complete model/tokenizer, BF16, TP2, greedy sampling, neutral penalties, thinking disabled, 32 mixed prompts, three measured repetitions after warmup, concurrency one. Prompt/output caps and client endpoint are shared. Throughput uses visible completion tokens retokenized with the same tokenizer, divided by summed client request time; padded diffusion usage counters are excluded.

Variant	Output tok/s	Per DCU tok/s
Native baseline	28.57	14.29
Frontier only	38.47	19.24
Expert budget/live reuse only	28.70	14.35
Joint, floor 0.7	39.92	19.96
Local SGLang Clover iterative port	5.88	2.94

Joint/native is 1.397x and joint/local-SGLang is 6.791x. Joint repetitions measured 39.85–40.02 tok/s. This is serial short-prompt throughput, not peak concurrent capacity, and compares the local iterative SGLang port rather than Clover's official self-speculative implementation. The SGLang reference runs on a separate identical TP2 pair on the same host.

examples/bench_clover.py and test/diffusion/workload.json reproduce the HTTP workload. See docs/clover-hygon.md for launch/build details. Separate profiles verify dispatch and are excluded from timing.

Notes for Reviewers

Supported now: BF16, Hygon, eager/static cache, greedy non-streaming text, causal block commit. The model's 128K maximum context, general continuous batching, graphs, multimodal input, stochastic sampling and official AR-verify speculation are not validated by this change. Unsupported endpoint options are rejected explicitly.

Review block/slot boundaries, state immutability, full-route causal replay after approximate preview, and the shared attention preview flag. The checkpoint cast bridge also rejects same-address dtype conversion before writing, restoring an existing loading regression invariant discovered during integration.

This remains a draft pending dependency review and the broader model/service integration expected by the repository. The dedicated Clover generator/server is the validated path; the generic AR examples/test_infer.py / examples/bench.py / benchmark suite does not yet drive diffusion.

CI / ChatOps

Fork Ruff CI passed. The rerun at https://github.com/Ringssss/InfiniLM/actions/runs/40928429479 passed Host behavior, Check Format, dependency identity, and matrix generation. The NVIDIA build/package job is queued for an external runner. The earlier fork-only format failure was caused by a stale fork main; syncing the fork resolved it without unrelated source changes. Local checks using the CI formatter versions also pass all 24 changed code files.

---

Checklist

Title, Branch, and Commits

• PR title follows Conventional Commits.
• Branch uses feat/ with hyphen-separated words.
• Commit follows Conventional Commits and is squashable.
• No merge/fixup/wip commits; based on current main at preparation.
• N/A: legacy issue format is not used.

Scope and Design

• Changes implement the stated model/decoder/dependency scope.
• No debug prints or unrelated code in model execution.
• Public input flags and supported boundaries are documented.
• No unrelated formatting churn.

General Code Hygiene

• English comments/messages, trailing newlines and no whitespace errors.
• Third-party provenance and licenses recorded.
• git diff --check passed.

C++ Specific

• Model code uses existing owning tensors, modules and runtime/stream resources.
• No changes to llama_legacy.
• Changed files pass scripts/format.py with the CI formatter version.

Python Specific

• Ruff and syntax checks passed.
• Docstrings and comments use English.
• No changes to auto_config.py.

Testing

• Real Hygon build, operator and full-model tests recorded above.
• Other accelerator hardware validation: unavailable locally.
• Generic examples/test_infer.py: AR entry point; dedicated Clover generator validated instead.
• Generic examples/bench.py: AR entry point; paired Clover HTTP/offline measurements supplied instead.
• Full C-Eval/MMLU sanity suite: not run; controlled fixture is not a substitute for that evaluation.
• Generic inference server: dedicated serial Clover OpenAI endpoint validated instead.

Build, CI, and Tooling

• Native Hygon release build passed.
• Manual CI workflow dispatched; external hardware jobs remain pending.

Documentation

• Build, deployment, methods, measurement and supported boundaries documented.
• N/A: no default behavior change for existing models; optimizations are opt-in.

Security and Safety

• No secrets, private URLs or personal hardware identifiers committed.
• Third-party terms and sources retained.
• Mask/state/workspace bounds and lifetime contracts covered by checks/tests.