Remove the NuGet publishing workflow

[ru-sh]

Summary

• Delete .github/workflows/publish-nuget.yml — CI no longer pushes packages to nuget.org (Trusted Publishing/OIDC) or GitHub Packages
• Version tags (v*) now trigger only release-binaries.yml
• Reword the release-binaries.yml header comment that referenced the removed workflow
• Update DEVELOPMENT.md / DEVELOPMENT_zh-cn.md: replace the publishing-process section with a pointer to eng/verify-packages.ps1 for local pack & verify; fix stale "gates the publish workflow" / "ship with the next version tag" claims
• CI unblock (c365530): the benchmark's pip install now falls back to --break-system-packages — the self-hosted runner's system Python is externally managed (PEP 668) and rejected the plain pip install --user, failing the model-download step before the cached-model check. Unrelated to the publishing removal; fixed here so the required check can go green.

eng/verify-packages.ps1 is kept for local packing and validation. This removes the only CI artifact egress outside the MotusLabs org.

Test plan

• Workflow syntax: no workflow files changed except deletion — YAML untouched otherwise
• pr-unit-tests green on this PR (ubuntu-latest + ubuntu-24.04-arm)
• engine-comparison smoke green after the PEP 668 fallback fix
• After merge, pushing a v* tag is expected to run only the binary release workflow

🤖 Generated with Claude Code

[github-actions]

Engine comparison — TensorSharp vs llama.cpp (PR smoke)

Engine comparison benchmark — TensorSharp vs llama.cpp vs vLLM

Same GGUF files, same host, one uniform OpenAI /v1/chat/completions surface, across text / image / audio / video / single-turn / multi-turn / function-call / structured-output scenarios on the selected compute backends (ggml_cuda / ggml_vulkan / ggml_metal / ggml_cpu / cpu / ...).

Numbers are tokens/second (higher is better). — = not applicable / skipped, fail = errored at runtime, n/a = combination never attempted.

Software / hardware

Component	Version / detail
TensorSharp	git 8e9a84e, .NET 10.0.112 (backends: ggml_cuda / ggml_metal / mlx)
llama.cpp	/home/gha-gpu/tensorsharp-bench/llama.cpp/build/bin/llama-server
vLLM	endpoint http://127.0.0.1:8000 (connect-only)
GPU	NVIDIA GeForce RTX 3090, 24576 MiB
NVIDIA GeForce RTX 3090, 24576 MiB

Methodology

• Each (engine, backend, model) group launches its server once; all of that group's scenarios run against it, so per-scenario timings exclude model-load cost.
• Metrics come from the streamed response: ttft is time-to-first-token (prefill latency proxy), prefill_tps = prompt_tokens / ttft, and decode_tps = (completion_tokens - 1) / (t_last - t_first).
• DiffusionGemma denoises whole blocks (no token stream), so it is run non-streaming and its decode_tps is wall-clock tokens/second.
• Greedy sampling (temperature=0); one warmup request per server is discarded.
• The headline per-engine tables are the single-stream, MTP-off baseline; each tensor-parallel degree (--tp N) is its own column, labeled tpN. MTP on/off, tensor-parallel scaling and parallel-request scaling are reported in their own sections below.

Performance ratio — TensorSharp vs reference engines

Geomean of TensorSharp's per-scenario speedup over each reference engine on the same backend, across every scenario both engines ran (single-stream, MTP-off). A value > 1.0× means TensorSharp is faster (for decode / prefill throughput) or lower-latency (for TTFT); — = no overlapping cells. Per-scenario ratios are in each model's section below.

Model	Comparison	decode	prefill	TTFT
Gemma 4 12B it (QAT UD-Q4_K_XL, dense)	vs llama.cpp · CUDA	0.90×	0.92×	0.92×

Gemma 4 12B it (QAT UD-Q4_K_XL, dense) (gemma4-12b)

Decode throughput (tok/s)

Scenario	TensorSharp · CUDA	llama.cpp · CUDA
text_short	55.2	56.8
function_call	—	59.3
json_mode	43.7	59.0
prefill_4k	60.2	58.7

Prefill throughput (tok/s)

Scenario	TensorSharp · CUDA	llama.cpp · CUDA
text_short	1565.8	1789.0
function_call	1033.4	1291.7
json_mode	1381.2	1324.5
prefill_4k	1557.6	1561.1

Time to first token (ms, lower is better)

Scenario	TensorSharp · CUDA	llama.cpp · CUDA
text_short	1242.8	1087.7
function_call	1971.1	1577.0
json_mode	1450.9	1483.6
prefill_4k	2478.8	2473.3

Performance ratio — TensorSharp vs reference (> 1.0× = TensorSharp faster)

Decode throughput

Scenario	vs llama.cpp · CUDA
text_short	0.97×
function_call	—
json_mode	0.74×
prefill_4k	1.03×

Prefill throughput

Scenario	vs llama.cpp · CUDA
text_short	0.88×
function_call	0.80×
json_mode	1.04×
prefill_4k	1.00×

Time to first token (latency; > 1.0× = TensorSharp lower)

Scenario	vs llama.cpp · CUDA
text_short	0.88×
function_call	0.80×
json_mode	1.02×
prefill_4k	1.00×

Output quality — TensorSharp vs llama.cpp

Both engines decode the same GGUF greedily (temperature=0) on the same backend, so their outputs should agree closely. similarity is a whitespace-normalized SequenceMatcher ratio between the two outputs (1.00 = identical); low similarity, an invalid JSON object in json_mode, or a missing tool call in function_call flags an output-quality problem on one side. Prefill scenarios (8-token outputs) are excluded. Side-by-side excerpts follow the table, lowest agreement first.

Model	Backend	Scenario	similarity	verdict	tokens (TS / ref)	finish (TS / ref)	checks
gemma4-12b	CUDA	text_short	0.51	low	72 / 71	stop / stop	—
gemma4-12b	CUDA	function_call	—	—	20 / 21	tool_calls / tool_calls	tool call: TS yes / ref yes
gemma4-12b	CUDA	json_mode	1.00	high agreement	33 / 39	stop / stop	json valid: TS yes / ref yes

Mean similarity across 2 compared cells: 0.75 (min 0.51, max 1.00).

gemma4-12b · CUDA · text_short (similarity 0.51)

TensorSharp

A transformer is a neural network architecture that processes sequences by using a mechanism called "attention" instead of reading inputs one element at a time. This allows every position in a sequence to directly consult and weigh the importance of every other position simultaneously. By replacing sequential processing, it enables the model to better maintain context over long distances and train more efficiently on vast datasets.

llama.cpp

A transformer is a neural network architecture that processes sequences by using a mechanism called "attention" rather than reading inputs one element at a time. This allows every position in a sequence to directly consult and weigh the importance of every other position simultaneously. By replacing sequential processing, the transformer enables the model to better maintain context over long distances and train more efficiently.

gemma4-12b · CUDA · json_mode (similarity 1.00)

TensorSharp

{
  "name": "Mars",
  "diameter_km": 6779,
  "has_moons": true
}

llama.cpp

{
  "name": "Mars",
  "diameter_km": 6779,
  "has_moons": true
}

Image editing (stable-diffusion)

Same input image, prompt, resolution, step count, cfg and seed for every engine. Timings are each engine's own pipeline timers (TensorSharp's [pipe-timing] phases + server elapsedSeconds; sd.cpp's phase logs + generate_image total), so weight-file loading and HTTP/process overhead are excluded on both sides. total (warm) is the steady-state request on an already-running server; first request (cold) additionally pays TensorSharp's per-request DiT rebuild + graph capture on a fresh server (a CLI engine has no such distinction). Lower is better.

No image-edit cells were run (see the image_edit scenario).

MTP / NextN speculative decoding (on vs off)

Single-stream decode tok/s with MTP/NextN speculative decoding off vs on (TensorSharp only). Speedup < 1.0× means speculation cost more than it saved for that cell — expected when the fused full-model decode path is already the fast path.

No MTP on/off pairs were run (use --mtp off,on).

Tensor parallelism (multi-GPU scaling)

One model split across N GPUs in a single process — TensorSharp --tp N, llama.cpp --split-mode tensor — with each engine pinned to exactly N devices. scaling is the highest degree that ran over the lowest one (a model that does not fit a single GPU is normalized against its own smallest working degree). Single-stream, MTP-off cells only.

No tensor-parallel cells were run (use --tp 1,2,4).

MoE CPU offload (--n-cpu-moe)

Routed experts of the first N layers kept in system RAM and multiplied on the host, against the same cell fully GPU-resident. This axis buys VRAM — it is what makes a checkpoint that does not fit run at all, and nearly doubles the context the loader can size — and pays for it in throughput, so < 1.0× is the expected shape here. Single-stream, MTP-off cells only.

No MoE CPU-offload cells were run (use --n-cpu-moe off,8,all).

Parallel-request scaling (concurrency)

decode/req is the mean per-request decode tok/s; aggregate is the system-wide decode throughput (total generated tokens / the wall window during which any sequence was decoding) when N identical requests are fired at one server at once.

No parallel-request cells were run (use --concurrency 1,4,8).

Tool-call and workflow correctness

function_call emits one tool call; agentic and code_edit drive a multi-turn workflow and turns says how many of its round trips ran (a short count is a workflow the model broke — the cell's detail names the turn and the reason).

Engine · Backend · Model	Scenario	turns	correct
llamacpp · ggml_cuda · gemma4-12b	function_call	—	yes
tensorsharp · ggml_cuda · gemma4-12b	function_call	—	yes