Repository navigation
[AgentX] Refresh the complete Kimi-K3 GB300 curve / [AgentX] 更新 Kimi-K3 GB300 完整曲线 - #4208
Conversation
|
Thanks for the contribution!
中文感谢你的贡献！
|
将完整曲线更新记录关联到 PR #4208。 Signed-off-by: Xin Li <xinli@nvidia.com>
xinli-sw
added
the
non-canary-full-sweep-enabled
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
I reviewed this PR and found no bugs. The NVIDIA master config and gb300 srt-slurm recipes have designated code owners, and the new low-latency recipe is a performance choice, so one of those owners should still take a look.
What was reviewed:
- The image bump to nightly-49cb2bbf with a digest pin on both agg and disagg master entries, and the matching
frameworks.vllmpins in agg-variants.yaml and disagg-variants.yaml. - The new agg-megakernel-variants.yaml overrides against the new master search-space points. The TP16/DCP16 override sets 4 nodes / 16 GPUs to match
num-nodes: 4, and the TP8/DCP8 override uses the 2-node base. The recipe hard-codes no acceptance length. - The perf-changelog entry is appended at the tail and covers both changed keys. Its pr-link is still the
pull/XXXplaceholder.
Extended reasoning...
The change is YAML only. It bumps the Kimi-K3 GB300 FP4 AgentX image on two nvidia-master.yaml entries, pins the vLLM framework in two existing recipes, adds a new 95-line megakernel recipe with two overrides, adds four new benchmark points (two configs at two concurrencies each), and appends a changelog entry. It touches no security-sensitive surface. I deferred because CODEOWNERS assigns specific NVIDIA owners to nvidia-master.yaml and to the gb300 recipe tree, and the new low-latency serving settings are a tuning decision for those owners to weigh.
This review covers commit 189735d, which is no longer the latest commit on this pull request; later commits are not covered by it.
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=39055199395 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=39055199395 |
升级现有 GB300 聚合及分离式配置的 vLLM 镜像，并加入使用概率草稿采样的四个低延迟点。 Signed-off-by: Xin Li <xinli@nvidia.com>
将完整曲线更新记录关联到 PR #4208。 Signed-off-by: Xin Li <xinli@nvidia.com>
Register the native K3 tool-call and reasoning parsers on aggregate and decode workers so required tool calls use the XTML path instead of generic JSON constraints. 为聚合和解码工作节点配置 Kimi K3 工具调用与推理解析器，使必选工具调用使用 XTML 路径，避免错误应用通用 JSON 约束。
75564ec to
c971eaa
Compare
Pin aggregate and disaggregated recipes to ffbbaaa6, which uses dynamo-parsers 8.1.0 and includes the K3 chunk-boundary fixes from frontend-crates PR 152. Keep the native-vLLM megakernel recipes unchanged. 将聚合和分离式配方的 Dynamo 固定到 ffbbaaa6，使用包含 K3 输出分块解析修复的 dynamo-parsers 8.1.0，保持原生 vLLM megakernel 配方不变。
Preserve the current main perf changelog byte-for-byte and append the single PR 4208 entry at the tail. Keep all 15 Kimi K3 serving configurations unchanged. 合并 main 并解决冲突，逐字节保留 main 的性能变更记录，仅在末尾追加 PR 4208 的单个条目，保持全部 15 个 Kimi K3 服务配置不变。
Use probabilistic draft sampling consistently so the existing golden-AL selector chooses 3.36 for four drafts and 3.84 for seven drafts. Keep evaluation on real block verification. 统一使用概率草稿采样，使现有 golden AL 选择器对 4 个和 7 个草稿分别使用 3.36 和 3.84；评测继续使用真实 block 验证。 Co-authored-by: Codex <noreply@openai.com> Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
|
/use 39055199395 |
|
@functionstackx staged run 39055199395. This run remains available across future |
|
/reuse-sweep-run 39055199395 |
# Conflicts: # inferencex-e2e/perf-changelog.yaml
Description
Refresh Kimi-K3 GB300 with digest-pinned vLLM nightly-49cb2bbf. Keep 11 points and add four low-latency points using probabilistic DSpark (AL 3.62/3.84) for throughput and real block verification for evals. Pin Dynamo ffbbaaa6 with K3 parsers for evals via
eval-srt-recipe; throughput runs leave the parsers disabled to avoid empty responses on short, length-capped requests. SetDYN_TCP_REQUEST_TIMEOUT=120on the Dynamo frontend and workers.Testing: The previous sweep failed on empty throughput responses, which prevented a complete measurement window and power validation. The eval-only parser change targets that failure; GPU validation of this revision remains outstanding.
Dependency: Supersedes #3315’s curve. Loopback fix: #4250; publication gate: #3165. Upstream recipe; follows #4178.
中文
用固定摘要的 vLLM nightly-49cb2bbf 更新 Kimi-K3 GB300 曲线，保留 11 个已有点并新增四个低延迟点。新增点的吞吐测试使用概率 DSpark（AL 3.62/3.84），评测使用真实 block 验证。Dynamo 固定为 ffbbaaa6，通过
eval-srt-recipe仅在评测中启用 K3 parser；吞吐测试禁用 parser，避免短输出在长度上限处被截断后返回空响应。Dynamo frontend 和 worker 均设置DYN_TCP_REQUEST_TIMEOUT=120。测试： 上一轮 sweep 因吞吐请求返回空响应而失败，未形成完整测量窗口，导致功耗校验失败。此次仅在评测中启用 parser 的改动针对这一问题；本次修订仍待 GPU 验证。
依赖： 承接 #3315 的曲线；loopback 修复见 #4250，发布门禁见 #3165。本 PR 接续 #4178；上游 recipe。实现、检查和整合所用的准确模型标识不可用。
AI model disclosure
Exact model identifier unavailable; implementation, checks and consolidation.
PR checklist and change type
Type of Change
Checklist
python -m infx.matrix.changed --base origin/mainlists affected config keys, I have appended one new entry covering them to the physical end ofinferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entriesinferencex-e2e/perf-changelog.yamlentry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs):full-sweep-fail-fast(recommended),full-sweep-enabled, ornon-canary-full-sweep-enabled. Optional modifiersall-evals,evals-only, andagentx-fastrequire a primary label; the last two block reuse while applied.OWNER/MEMBER/COLLABORATOR) has commented/use <run_id>(or the legacy/reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.