Skip to content

feat(connectors): add williamhtan/kernelsight - #835

Open
m7m2od wants to merge 4 commits into
OpenTSLM:mainfrom
m7m2od:feature/kernelsight-connector
Open

m7m2od wants to merge 4 commits into
OpenTSLM:mainfrom
m7m2od:feature/kernelsight-connector

Conversation

@m7m2od

@m7m2od m7m2od commented Nov 20, 2026 •

Copy link
Copy Markdown

Add williamhtan/kernelsight: H100 GPU hardware-counter traces with per-segment workload labels, from williamhtan/kernelsight (Apache-2.0). The connector pins the Hub commit d8cd8e804e5b619aadbb082a1fb3a8791b60a680.

I cannot add labels from a fork. Could a maintainer add area:connectors?

What it builds

  • One Record per snapshot (1,444). A snapshot is 24 counter channels over 512 equal-width bins. Each channel is one Signal on a RegularAxis taken from the snapshot's bin edges, so the period differs per snapshot (0.2 ms to 43 ms).
  • The 24 Signals sit in five Sources, one per channel group of the dataset card: pipe signature, memory access, discriminators, system bandwidth and SASS modality.
  • Each labeled segment becomes two time-interval annotations: workload_l1 (12 classes) and workload_l2 (73 classes).
  • One ClassificationTask per snapshot. The target is the authors' dominant_l1 field.
  • The dataset card has the licence, the citation and the source link.

Notes

  • Split. docs/connectors.md says to preserve official partitions. The released train/val/test split puts dtype and seed variants of one KernelBench problem in different partitions (44 of 200 problems cross it). So the connector assigns its own split with StratifiedSplitter, grouped by problem and stratified by dominant_l1, seed 42.
  • Problem. A problem is the variant without its dtype, seed and launch options: the CUTLASS raster, swizzle and b options (_rH_s8_b1, _rM, _sw4, _b1) and a microbenchmark's thread-block size (_T256). cutlass_ws_overlap runs the CUTLASS ex48 GEMM of cutlass_gemm with device timer markers, so one shape of the two motifs is one problem. A short CUTLASS name (m4096_k2048, kheavy, baseline) leaves the other sizes at a default. The 12 short names whose kernel launch times match a full shape within about 1% join the problem of that shape. No problem crosses partitions (0 of 401).
  • 3/1/1, not 8/1/1. Several classes have only 6 to 8 problems. With 8/1/1, loss, pooling and elementwise each lose the validation or the test partition. With 3/1/1, only softmax (2 problems) has no validation snapshot.
  • Released splits kept. Each record's metadata keeps the authors' memberships under release_splits (train, val, test, iid, param_ood, composed, length_ood), so the released benchmark stays reproducible.
  • Multi-hot track. In the 472 cutlass_ws_overlap snapshots, an overlapping memory_movement / memory_movement_copy run exists only in the release's multi-hot track. Each such run becomes a workload_l1 and a workload_l2 interval with occurrence metadata track=multihot.
  • No unpickling. The release stores its vocabularies and counter names as pickled object arrays. The connector keeps those names as constants and reads only numeric arrays, with allow_pickle=False. A missing, corrupt or damaged file raises TimeNetConnectorError.
  • convert reads only the bin edges and labels. The lazy loader reads the counter values and checks their shape.

Verified against the real release

timenet-build build williamhtan/kernelsight from the Hub at the pinned revision gives 1,444 records and 1,444 tasks, split 862 / 291 / 291. The records carry 45,860 segment intervals for each of workload_l1 and workload_l2, plus 590 track=multihot intervals for each. verify() passes.

Locally, make check-ci, make test, make test-connectors and make license-check pass. The connector has 35 tests. One of them reads a real snapshot of the pinned release from tests/fixtures/ (16 KB).

Part of the TimeNet Online Hackathon: onboard a dataset, then fine-tune a time-series language model on it. This replaces #831, which I closed before rebasing onto the connector API changes (#822, #823, #797, #827).

🤖 Generated with Claude Code

m7m2od and others added 4 commits November 20, 2026 22:58
KernelSight: 1,444 H100 GPU counter snapshots (24 channels x 512 bins)
with per-segment L1/L2 workload labels, from the pinned Hub release.

- One Record per snapshot, 24 Signals in five channel-group Sources,
  a per-snapshot RegularAxis from the bin edges.
- Segment L1/L2 interval annotations, plus multi-hot-only overlap runs
  (track=multihot) for the 472 cutlass_ws_overlap snapshots.
- One ClassificationTask per snapshot on the authors' dominant_l1.
- Split grouped by problem (KernelBench dtype/seed variants stay
  together), stratified by class, 3/1/1, seed 42; the released split
  memberships are kept in record metadata.
- Reads only numeric npz arrays (allow_pickle=False).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… errors in kernelsight

- Group a Variant by its Problem without launch options: the CUTLASS
  schedule suffix (_rH_s8_b1) and the microbenchmark thread-block size
  (_T256). No Problem crosses partitions now (0 of 427, was 32 CUTLASS
  shape groups with 725 snapshots).
- Add the workload_l2 annotation (memory_movement_copy) to the 590
  multi-hot-only intervals.
- Raise TimeNetConnectorError for a missing or corrupt npz or split
  file, a missing array and segment arrays of different lengths.
- Check the data shape in the lazy loader, so convert reads no values.
- Test one real snapshot of the pinned release from tests/fixtures.
- Card: bin width is 0.2 to 43 ms (not 68), the metadata key is
  release_splits, and the comment says why the split is 3/1/1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant