[dongjiang1989]

Summary

This PR adds CI integration for Spark-X2.5 models (1.7B and 4B variants) to the ExecuTorch test infrastructure.

Changes:

• Added spark_x2_5_1_7b and spark_x2_5_4b model support to .ci/scripts/test_torchao_huggingface_checkpoints.sh
• Added both models to the CI test matrix in .github/workflows/trunk.yml
• Configured XNNPACK (x86_64) and TorchAO (ARM64) backend testing for both variants

The Spark-X2.5 model infrastructure (configs, weight conversion, documentation) already exists in the codebase. This PR completes the integration by adding automated CI testing.

Model specifications:

• Spark-X2.5-1.7B: 28 layers, hybrid attention architecture (3 sliding + 1 full attention)
• Spark-X2.5-4B: 36 layers, hybrid attention architecture
• Both support 8da4w quantization and extended context (up to 1M tokens)

Hugging Face models:

• XHToken/Spark-X2.5-1.7B
• XHToken/Spark-X2.5-4B

Test plan

The changes add CI test coverage for both Spark-X2.5 model variants:

1. Automated CI tests (via GitHub Actions):

   • Both models will be tested on x86_64 with XNNPACK backend
   • Both models will be tested on ARM64 with TorchAO backend
   • Tests verify model export, weight conversion, and basic inference

2. Manual testing (to verify locally):

   # Test Spark-X2.5-1.7B
   bash .ci/scripts/test_torchao_huggingface_checkpoints.sh spark_x2_5_1_7b --test_with_runner
   
   # Test Spark-X2.5-4B
   bash .ci/scripts/test_torchao_huggingface_checkpoints.sh spark_x2_5_4b --test_with_runner

3. Unit tests (already present):

   pytest examples/models/spark_x2_5/test_spark_x2_5.py -v

The CI script validates:

• Model download from Hugging Face
• Weight conversion to Meta format
• Model export with 8da4w quantization
• File size bounds (2GB for 1.7B, 5GB for 4B)
• Inference via llama_main runner

cc @larryliu0820 @mergennachin @cccclai @helunwencser @jackzhxng @digantdesai @nil-is-all

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/23525

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⚠️ 24 Awaiting Approval

As of commit 0551746 with merge base cedc66f ():

AWAITING APPROVAL - The following workflows need approval before CI can run:

• Build Aarch64 Linux CUDA Wheels (gh)
• Build Aarch64 Linux Wheels (gh)
• Build documentation (gh)
• Build Linux CUDA Wheels (gh)
• Build Linux Wheels (gh)
• Build macOS Wheels (gh)
• Build Presets (gh)
• Build Windows CUDA Wheels (gh)
• Build Windows Wheels (gh)
• Cadence Build & Test (gh)
• Lint (gh)
• periodic (gh)
• pull (gh)
• Test ARM Backend (gh)
• Test CoreML Backend (gh)
• Test Cortex-M Backend (gh)
• Test NXP Backend (gh)
• Test QNN Backend (gh)
• Test QNN Windows MSVC build (gh)
• Test Vulkan Backend (gh)
• Test WebGPU Backend (gh)
• Test XNNPACK Backend (gh)
• trunk (gh)
• Windows MSVC Build (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[nil-is-all]

@dongjiang1989 could you fix the CI failures? Do ping when ready for review.

[nil-is-all]

@dongjiang1989 this revert would have resolved the failing CI. Could you try with this and ping to trigger CI workflows?

[dongjiang1989]

@dongjiang1989 could you fix the CI failures? Do ping when ready for review.

Thx @nil-is-all rebase done.
Please recheck it, thanks

[dongjiang1989]

Hey @nil-is-all

CI is failing because /mnt/hf_cache on the runners is read-only. The existing models (qwen3, phi_4_mini, lfm2_5) are pre-cached there, but XHToken/Spark-X2.5-1.7B and XHToken/Spark-X2.5-4B are not, so hf download fails with:

OSError: [Errno 30] Read-only file system: /mnt/hf_cache/hub/models--XHToken--Spark-X2.5-*

Please add pre-cache the two Spark-X2.5 models in /mnt/hf_cache?

[nil-is-all]

Hey @nil-is-all

CI is failing because /mnt/hf_cache on the runners is read-only. The existing models (qwen3, phi_4_mini, lfm2_5) are pre-cached there, but XHToken/Spark-X2.5-1.7B and XHToken/Spark-X2.5-4B are not, so hf download fails with:

OSError: [Errno 30] Read-only file system: /mnt/hf_cache/hub/models--XHToken--Spark-X2.5-*

Please add pre-cache the two Spark-X2.5 models in /mnt/hf_cache?

@dongjiang1989 I'd suggest making the script not depend on the shared cache being pre-populated.
The script only needs /mnt/hf_cache to be readable. The failure is hf download trying to create models--XHToken--Spark-X2.5-* in a read-only mount. If you point it at a writable workspace dir for these models, the download lands somewhere writable, But I'll run CI to check if it still flags it.

[dongjiang1989]

@dongjiang1989 I'd suggest making the script not depend on the shared cache being pre-populated.

Thanks @nil-is-all Change model download dir to custom dir.
Please recheck it, 🤭

[nil-is-all]

@dongjiang1989 I'd suggest making the script not depend on the shared cache being pre-populated.

Thanks @nil-is-all Change model download dir to custom dir. Please recheck it, 🤭

Thanks, could you update branch and ping?

[dongjiang1989]

Thanks, could you update branch and ping?

@nil-is-all git rebase done. Please re-check it

[nil-is-all]

@dongjiang1989 CI failures seem relevant. Could you take a look?

[dongjiang1989]

Thanks @nil-is-all

Investigated the 5 failing jobs. Two separate issues:

1. test-qnn-optimum-model (fp32, dit) — infra flake, unrelated to this PR. The runner's /mnt/hf_cache mount was read-only (OSError: [Errno 30]), so the HF download failed. This job is green on recent main runs; it should pass on the next re-run.

2. All 4 test-torchao-huggingface-checkpoints (spark_x2_5_*) jobs — llama_main fails to load the Spark tokenizer. Root cause is two bugs in the C++ tokenizers submodule (meta-pytorch/tokenizers), reproduced end-to-end with the real XHToken/Spark-X2.5-1.7B/tokenizer.json:

• Spark ships "normalizer": {"type": "Sequence", "normalizers": []}. An empty Sequence is a valid no-op per the HF tokenizers spec, but NormalizerConfig::create() throws on it, so HFTokenizer::load() fails and the runner falls back to tiktoken/llama2c (both fail) → Failed to create llama runner.
• After fixing that, encode() still returned 0 tokens: RegexPreTokenizer treats every Regex-pattern Split as "return the matches", which silently drops all non-matching text. Spark isolates digits with \p{N}{1,3} (behavior=Isolated, invert=false); per HF semantics invert=false means matches are delimiters and text between them must be kept. For a prompt with no digits, the old code returned zero pieces. This went unnoticed because GPT-2/llama3-style regexes tile the input, where both semantics coincide.

Fix: upstream PR meta-pytorch/tokenizers#225 addresses both (empty Sequence → no-op; Regex Split with explicit behavior + invert=false → delimiter semantics). Adds unit tests.
Verified with the real Spark tokenizer: load OK, encode("Once upon a time,") → [19327 6420 259 1279 25], decode round-trips exactly.

I'll also add a small defensive change in convert_weights.py that rewrites the no-op normalizer to null (equivalent, and accepted by older tokenizer revisions), so the local/README flow works even before the pin bump. The dit job needs no code change — just a re-run.

[dongjiang1989]

@nil-is-all please re-check it