Repository navigation
[PowerX] require Kimi-K3 H200 AgentX power / [PowerX] 强制校验 Kimi-K3 H200 AgentX 功耗 - #4489
edwingao28 wants to merge 5 commits into
Conversation
|
Generally most InferenceX submissions don't need to add changes to the infx package. On the off chance they do, generally the diff is small. Please prompt your agent to clean your PR to keep your diff to only necessary & minimal changes too. We have an increased code quality bar for changes to the infx package. 中文大多数 InferenceX 提交通常不需要修改 infx 包。即使确实需要，改动通常也很小。请让你的 agent 清理 PR，使改动只包含必要且最小的内容。infx 包的改动有更高的代码质量要求。 |
|
Thanks for the contribution!
中文感谢你的贡献！
|
强制校验 Kimi-K3 H200 AgentX 的 GPU 功耗。
1c867c2 to
dc3371a
Compare
edwingao28
added
the
full-sweep-fail-fast
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| dp-attn: ${{ matrix.config.dp-attn }} | ||
| duration: ${{ matrix.config.duration }} | ||
| run-eval: false | ||
| require-power: ${{ matrix.config['require-power'] == true && matrix.config['eval-only'] != true }} |
There was a problem hiding this comment.
🟡 (optional) Manual and fork-PR sweeps of the Kimi-K3 H200 AgentX keys still pass without validated GPU power. Only run-sweep.yml forwards the new matrix require-power flag. e2e-tests.yml jobs test-sweep-agentic (line 399) and test-sweep-multi-node-agentic (line 465) pass only ${{ inputs.require-power }}. Fix: in both agentic throughput jobs, OR the input with the matrix flag, as test-sweep-multi-node already does at e2e-tests.yml:354: inputs.require-power || (matrix.config['require-power'] == true && matrix.config['eval-only'] != true). [also at: inferencex-e2e/configs/CONFIGS.md:99 - Manual and Klaud e2e runs of the Kimi-K3 H200 AgentX configs can still pass without validated GPU power, even though the scenario sets require-power: true.]
Why this was flagged
The generator now puts require-power: true on agentic rows for kimik3-fp4-h200-vllm-agentic-{latency,balanced,simple} (generate.py:1160). These rows can run through .github/workflows/e2e-tests.yml as well as run-sweep.yml. That workflow handles one-off dispatches per AGENT_OPERATIONS.md:39, and trusted-external-sweep.yml:137 dispatches it for fork PRs. In e2e-tests.yml, test-sweep-agentic (line 399) and test-sweep-multi-node-agentic (line 465) set require-power: ${{ inputs.require-power }}. That input defaults to false (line 33-37), so the matrix flag is ignored and REQUIRE_POWER resolves to '0' in benchmark-tmpl.yml:100 and benchmark-multinode-tmpl.yml:146. As a result, a fork-PR or manual sweep of these keys can still finish green without validated GPU power, which is the bug the PR says it fixes. The fixed-seq-len multi-node job in the same file (line 354) already ORs in the matrix flag. This PR updated run-sweep.yml only, and CONFIGS.md says the field requires validated power for throughput without limiting that to PR-triggered sweeps.
Verification: Severity: nit. Trigger: a Kimi-K3 H200 AgentX key runs through e2e-tests.yml, either as a manual workflow_dispatch or as a fork PR dispatched by trusted-external-sweep.yml. Its test-sweep-agentic (line 399) and test-sweep-multi-node-agentic (line 465) still pass only require-power: ${{ inputs.require-power }}. The base also never enforced it on this path.
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=40175529130 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=40175529130 |
edwingao28
removed
the
full-sweep-fail-fast
Summary
Require valid power telemetry across all three Kimi-K3 H200 AgentX configurations; eval-only jobs remain exempt. Replaces #3550; tracks #3592.
Testing: Full sweep remains active. Nine saved power audits passed; the full verifier had one request timeout and two schema failures.
Blocker: Complete sweep/eval qualification.
AI model disclosure
PR checklist and change type
Type of Change
Checklist
python -m infx.matrix.changed --base origin/mainlists affected config keys, I have appended one new entry covering them to the physical end ofinferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entriesinferencex-e2e/perf-changelog.yamlentry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs):full-sweep-fail-fast(recommended),full-sweep-enabled, ornon-canary-full-sweep-enabled. Optional modifiersall-evals,evals-only, andagentx-fastrequire a primary label; the last two block reuse while applied.OWNER/MEMBER/COLLABORATOR) has commented/use <run_id>(or the legacy/reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.中文
Kimi-K3 H200 的三个 AgentX 配置必须提供有效功耗遥测，纯评测任务保持豁免。替代 #3550，跟踪 #3592。
**验证：**完整 sweep 仍在运行。已保存的九份功耗审计通过；完整 verifier 出现一次请求超时与两项 schema 失败。
**阻塞：**完整 sweep 和评测尚未通过验收。
改动类型：修复、配置与文档。gpt-6-astra 负责恢复和独立审阅；原始主 agent 与委派 agent 的完整运行时型号无法核实。各 agent 参与实现、验证与整合。