Repository navigation
Conversation
✅ CI checks passed
Updated for |
The connector pandalin98/enginemt-qa builds one task per labelled question-answer pair of the EngineMT-QA release on the Hugging Face Hub. A single-row question reads the Record of its row. A multi-row question reads one joined Record with the first 60 steps of each of its 10 rows on an ordinal axis. The authors' code gives its model the same input. When a flight ends inside a row, the release fills the rest of the row with zeros. A row Record stops at its last real step. In a joined Record, the padded steps are null. The train file copies 4,039 lines of the test file. Tasks that the other file repeats carry the source issue repeated_across_splits, so that a user can rebuild the paper's split. A pair with an empty answer gets no task. The annotation unlabelled_pairs lists each one. Co-authored-by: Othman Sbai <212133622+osbai@users.noreply.github.com>
karhu
force-pushed
the
feature/tnet-174-enginemt-qa-dataset-connector
branch
from
November 20, 2026 20:11
adbfb2b to
d6a0d45
Compare
… pair The released train file holds 4,039 byte copies of test lines. The paper does not mention them. Before, their tasks stayed in the train split with the source issue repeated_across_splits, so every user had to know about the flag to avoid training on test pairs. Now a labelled train pair that has the same rows, question and answer as a test pair gets no task. That leaves out 27,961 train pairs: the labelled pairs of the copied lines, and 6 pairs of a train-only line about a row that a test line also names. The annotation test_pairs_in_train lists each one. The test split keeps every labelled pair, and the train split keeps 72,335 tasks. The tasks no longer have the source_issues metadata key. After the change, no pair repeats another pair in its own split. The README now describes the rows from a read of the whole file. A flight is a median of 16 rows long, and channel 32 is constant within a flight. Most multi-row lists take about one window from each of 10 consecutive flights. 651 lists mix two runs of channel 32, which are likely two engine units.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
EngineMT-QA (Wang et al., ITFormer, ICML 2025) is a question-answering benchmark over simulated aero-engine flights from NASA's N-CMAPSS. The authors publish it on the Hugging Face Hub. One HDF5 file holds 118,921 rows of 600 steps and 33 channels. Two JSON Lines files hold the question-answer pairs. TimeNet has no connector for it.
The release has properties that its card and paper do not state:
Solution
The connector
pandalin98/enginemt-qabuilds 114,312 tasks (72,335 train, 41,977 test), one per labelled pair, under the authors' file split. It leaves out the train pairs that repeat a test pair:AnswerTaskClassificationTaskClassificationTaskAnswerTaskpadded_stepsgives the number of zero steps that the connector left out.alignment_padding. In the pinned release, no block has padding.ClassificationTaskover a label set of its option letters. The fault cause is multi-label.source_file,source_lineandqa_offsetas the authors' evaluation names them.Design decisions
unlabelled_pairslists each one. The authors' current metric raises on an empty reference.test_pairs_in_trainlists each one. The test split keeps every labelled pair, so it stays comparable with the paper. The multi-row lines of both files still draw on the same 1,900 rows, in other combinations. The README describes this and the 651 lists that mix two engine units, by our reading of channel 32.channel_00tochannel_32, because the authors' channel audit names none of them.JsonlDataReader. The domain rules (stage, rows,<ts>marker, closed questions, answer letters) and the pinned counts per file stay in the connector.h5pyis in the connector'srequirements.txtand in the dev group, aswfdbis, so thattyresolves it.Validation
make check, the pre-commit hooks on the changed files, and the connector's 42 tests andty(Python 3.13 and 3.11) in the environment built from itsrequirements.txt.This PR replaces #629.