GPUStack Runner registers accelerated backends and inference services for GPUStack. This repository maintains the Python library, container recipes, and measured runner catalog.
- Register accelerated inference services for GPUStack: vLLM, SGLang, MindIE, and VoxBox.
- Select runners with the Python API by backend, service, and dependency constraints, backed by the measured package catalog.
- Build runner images from the maintained container recipes in
pack/.
Each cell shows the newest supported engine version with its runtime line, and every supported version is listed in Supported runners.
| Backend | vLLM | SGLang | MindIE | VoxBox |
|---|---|---|---|---|
| Ascend CANN | 0.30.0(rc) (9.1) |
0.5.20 (9.0) |
2.3.0 (8.5) |
|
| Iluvatar CoreX | 0.8.3 (4.2) |
|||
| NVIDIA CUDA | 0.31.0 (13.0) |
0.5.21 (13.0) |
0.0.21 (12.8) |
|
| Hygon DTK | 0.18.1 (26.04) |
0.5.10 (26.04) |
||
| T-Head HGGC | 0.23.0 (13.0) |
0.5.12 (13.0) |
||
| MetaX MACA | 0.21.0 (3.7) |
0.5.11 (3.7) |
||
| MThreads MUSA | 0.9.2 (4.1.0) |
0.5.7 (4.3.2) |
||
| AMD ROCm | 0.31.0 (7.2) |
0.5.21 (7.2) |
CANN/vLLM images use stable vLLM releases. An (rc) label identifies a prerelease vLLM-Ascend plugin.
Install the library:
pip install gpustack-runnerSelect runners with the Python API:
from gpustack_runner import list_runners
runners = list_runners(
backend="cuda",
service="vllm",
dependencies=(("lmcache", ">=0.4.6"),),
with_unknown_dependencies=False,
)See the API reference for selection options. The dependency guide explains unknown collection, absent packages, and raw installed versions.
- Packaging: service Dockerfiles, matrix selection, image builds, and final-image collection.
- Dependency metadata: installed package records and queries.
- Release automation: weekly proposals, model configuration, PR commands, and post-merge Pack.
- Project instructions: source navigation, development checks, and contribution conventions.
Contributions are welcome. Commits must carry a Signed-off-by line certifying the
Developer Certificate of Origin.
Using GPUStack Runner in production? Add your company or project to ADOPTERS.md via pull request.
Copyright (c) 2026 The GPUStack authors. Licensed under Apache License 2.0.