Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
"""Connectors for datasets released by Yilin Wang (``pandalin98``) and co-authors.

Keep this regular package so pkgutil.walk_packages can discover its connector children.
"""
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# EngineMT-QA

`pandalin98/enginemt-qa` converts questions about simulated aero-engine signals from the
[ITFormer paper][paper] and [authors' code][code]. The [Hugging Face release][card] is pinned to
`ea2051a15512212976c22c1a075ec549c7d48eef`.

The Hub metadata declares MIT, but its license section is an unfilled placeholder and the paper
states no dataset license. The local card follows that metadata; confirmation from the authors
remains outstanding.

## Source and records

| File | Content |
| --- | --- |
| `time_series_data.h5` | `seq_data`: float64, shape `(118921, 600, 33)`; `data_ID`: IDs 1–118,921 |
| `train_qa.jsonl` | 13,291 lines, 101,565 question-answer pairs |
| `test_qa.jsonl` | 5,539 lines, 42,477 question-answer pairs |

The connector creates one Record for each row referenced by a labelled pair: 7,755 rows in the
pinned release. Tasks referring to the same row share its Record. Unreferenced rows, which the
authors also use for encoder pretraining, are not included.

Each Record contains 33 float64 signals named `channel_00` through `channel_32`, with 600 values
and unknown units. Values are unchanged. The 1 Hz regular axis follows the paper's description of
600-point, ten-minute segments; the HDF5 file itself states no rate. Record metadata retains the
published `row_id`.

The release supplies no channel names, engine identity, flight identity or scaling parameters.
The [authors' channel audit][audit] does not establish which of the paper's 32 channels corresponds
to each of the 33 published columns. The connector therefore retains all 33 without assigning
physical names.

## Tasks and splits

Each labelled pair becomes one task, in source order. Train and test follow the two QA files.

| Stage | TimeNet task | Inputs | Target |
| --- | --- | --- | --- |
| Understanding | `AnswerTask` | 1 Record | Published answer |
| Perception | `ClassificationTask` | 1 Record | Published option letters |
| Reasoning | `ClassificationTask` | 10 Records | Published option letter |
| Decision-Making | `AnswerTask` | 10 Records | Published answer |

Closed-question label sets preserve the printed options. Fault-cause questions allow several
letters; other closed questions require one. Published question text supplies the prompt.

The authors take the first 60 steps of each of ten Records for multi-cycle questions. Tasks keep
the complete Records and store `steps_per_cycle: 60`; consumers reproducing that input must apply
this truncation themselves. The selected IDs are not necessarily consecutive cycles.

Task metadata retains `stage`, the closed-question name where applicable, `source_file`,
`source_line`, `qa_offset`, and `source_issues`. `source_line` and `qa_offset` are zero-based to
match the authors' evaluation; error messages use one-based physical file lines.

## Missing labels and source overlap

The connector omits tasks for 1,769 empty closed answers and records their locations in the
`unlabelled_pairs` dataset annotation. An empty fault-cause answer is not interpreted as “no
fault.” The authors trained on these pairs, so this is a deliberate conversion difference.

Duplicate pairs remain in the dataset, with flags in `source_issues`:

| Flag | Meaning |
| --- | --- |
| `repeated_across_splits` | Both files contain the same rows, question and answer |
| `repeated_question` | The same file repeats those rows, question and answer |

In the pinned QA files, 27,955 of 41,977 labelled test tasks also appear in train. Scores on this
split therefore include substantial overlap. Consumers evaluating unseen pairs can exclude the
flagged test tasks and should report that choice.

Other limitations remain: component-health and fault-cause answers can disagree, generated open
answers vary in wording, and published counts and channel descriptions differ between the files,
card and paper. The connector follows the pinned files and does not repair their answers.

## Building and verification

```bash
uv run timenet-build build pandalin98/enginemt-qa --out .timenet-registry
```

The download includes an 18.8 GB signals file; the referenced rows contain about 1.2 GB of values.
Keep source files available until writing finishes. Conversion checks the HDF5 structure, QA
schema, option vocabulary and pinned QA inventory. Task rereads verify the QA line fingerprints.
Invalid sources raise `TimeNetConnectorError` with context.

The QA parser has been run on the pinned QA files and the HDF5 header inspected. A complete build
and an end-to-end scan of the signals file have not been verified. Tests use small fixtures.

[paper]: https://arxiv.org/abs/2506.20093
[code]: https://github.com/Pandalin98/ITFormer-ICML25
[card]: https://huggingface.co/datasets/pandalin98/EngineMT-QA
[audit]: https://github.com/Pandalin98/ITFormer-ICML25/blob/b80142f/docs/ISSUE_23_CHANNEL_AUDIT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
"""EngineMT-QA: aero-engine question answering over simulated N-CMAPSS cycles, in four tasks."""

from timenet_connectors.datasets.pandalin98.enginemt_qa.connector import (
CONNECTOR as CONNECTOR,
EngineMTQAConnector as EngineMTQAConnector,
)
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
"""Orchestrate EngineMT-QA source validation, records and repeatable tasks."""

from collections.abc import Mapping
from typing import ClassVar

from timenet.dataset import TimeFDataset
from timenet.errors import TimeNetConnectorError
from timenet.types import (
AnswerTask,
ClassificationTask,
)
from timenet_connectors.bases.huggingface import BaseHuggingFaceConnector, HubFileRef
from timenet_connectors.datasets.pandalin98.enginemt_qa.records import build_record, unlabelled_annotation
from timenet_connectors.datasets.pandalin98.enginemt_qa.source import (
FileCounts,
QAFile,
make_channel_loader,
read_recording_count,
scan_questions,
)
from timenet_connectors.datasets.pandalin98.enginemt_qa.tasks import iter_tasks
from timenet_connectors.datasets.pandalin98.enginemt_qa.vocabulary import QA_PARTS, ClosedQuestion, Part, Stage


class EngineMTQAConnector(BaseHuggingFaceConnector):
"""Build EngineMT-QA from the pinned files of its Hugging Face release."""

HF_REPO = "pandalin98/EngineMT-QA"
SOURCE_REVISION = "ea2051a15512212976c22c1a075ec549c7d48eef"
SOURCE_FILES: ClassVar[Mapping[str, str]] = {part.value: part.source_file for part in Part}
# Counted on the pinned files. They match the authors' README: 101,565 train and 42,477 test pairs.
SOURCE_COUNTS: ClassVar[Mapping[Part, FileCounts]] = {
Part.TRAIN: FileCounts(
lines=13_291,
pairs={
Stage.UNDERSTANDING: 41_128,
Stage.PERCEPTION: 35_987,
Stage.REASONING: 16_300,
Stage.DECISION_MAKING: 8_150,
},
unlabelled={ClosedQuestion.FAULT_CAUSE: 1_054, ClosedQuestion.HEALTH_STATE: 215},
),
Part.TEST: FileCounts(
lines=5_539,
pairs={
Stage.UNDERSTANDING: 17_240,
Stage.PERCEPTION: 15_085,
Stage.REASONING: 6_768,
Stage.DECISION_MAKING: 3_384,
},
unlabelled={ClosedQuestion.FAULT_CAUSE: 422, ClosedQuestion.HEALTH_STATE: 78},
),
}

def convert(self, raw_refs: list[HubFileRef]) -> TimeFDataset:
"""Convert the files into one lazy Record per named row, and a stream of one task per labelled pair.

Conversion checks the signals file's layout and every QA line once, and compares each QA file's
counts with the pinned ones. The values stay in the HDF5 file until written, so the files must stay
available until then. A closed pair with an empty answer is a missing label and gets no task.

Args:
raw_refs: The downloaded files, one per part.

Returns:
A dataset with the Records, the unlabelled pairs as an annotation, and a stream of every task.

Raises:
TimeNetConnectorError: If a part is missing, or a file is malformed or has other counts.
TimeNetConnectorError: If a file cannot be read.
"""
paths = {Part(ref.part): ref.path for ref in raw_refs}
if paths.keys() != set(Part):
raise TimeNetConnectorError(f"EngineMT-QA needs the parts {[part.value for part in Part]}")
signals_path = paths[Part.SIGNALS]
n_rows = read_recording_count(signals_path)
qa_files = [QAFile(paths[part], part, n_rows) for part in QA_PARTS]
scan = scan_questions(qa_files, self.SOURCE_COUNTS)
reader = make_channel_loader(signals_path)
records = {row_id: build_record(row_id, reader) for row_id in sorted(scan.named_rows)}

dataset = TimeFDataset(metadata=self.metadata())
dataset.annotate(unlabelled_annotation(scan.unlabelled))
for record in records.values():
dataset.add_record(record=record)

dataset.set_task_stream((AnswerTask, ClassificationTask), lambda: iter_tasks(qa_files, records, scan))
return dataset


CONNECTOR = EngineMTQAConnector
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# yaml-language-server: $schema=https://docs.timenet.ai/schemas/dataset-card-v1.schema.json
# Dataset Card for the pandalin98/enginemt-qa connector: the single source for this dataset's metadata.
#
# Training convention: the tasks are the authors' four stages, one task per published QA pair, split as
# the authors train and test (train_qa.jsonl, test_qa.jsonl). Closed questions are classifications over
# their option letters, as the authors' model answers and their metric scores. 1,769 closed pairs with an
# empty answer have no task. Signals are the release's z-scored values at 1 Hz, 33 unnamed channels per
# 600-step cycle. Tasks that the other split repeats name repeated_across_splits in their
# source_issues metadata.
yaml_schema_version: 1
dataset_id: pandalin98/enginemt-qa
dataset_version: 1.0.0
name: EngineMT-QA
description: >-
Aero-engine time-series question answering over simulated N-CMAPSS flight cycles: 142,273 labelled
QA pairs (100,296 train, 41,977 test) in four tasks. Understanding and Perception ask about one
600-step cycle of 33 z-scored channels; Reasoning and Decision-Making ask about 10 cycles.
Perception and Reasoning are multiple choice; Understanding and Decision-Making are open-ended
answers written with LLMs and checked by experts.
license: MIT
source_url: https://huggingface.co/datasets/pandalin98/EngineMT-QA
source_revision: ea2051a15512212976c22c1a075ec549c7d48eef
citation: >-
Wang et al. ITFormer: Bridging Time Series and Natural Language for Multi-Modal QA with
Large-Scale Multitask Dataset. ICML 2025. https://arxiv.org/abs/2506.20093
domains:
- industrial
tags:
- aero-engine
- n-cmapss
- simulated
- question-answering
- classification
- generated-supervision
- huggingface
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
"""Build EngineMT-QA records and dataset provenance from checked source facts."""

from collections.abc import Sequence

from timenet.dataset import Record, Signal, SignalChannel, Source
from timenet.json import JsonValue
from timenet.types import (
Annotation,
)
from timenet.types.shared_loading import SharedChannelLoader
from timenet_connectors.datasets.pandalin98.enginemt_qa.source import PairLocation
from timenet_connectors.datasets.pandalin98.enginemt_qa.vocabulary import (
AXIS,
CHANNELS,
N_STEPS,
SOURCE_NAME,
SPEC,
AnnotationKey,
MetadataKey,
)


def build_record(row_id: int, reader: SharedChannelLoader[int]) -> Record:
"""Return a row's Record with 33 lazy signals and its published row ID."""
signals = Signal.from_shared_loader(
loader=reader,
recording_key=row_id,
channels=tuple(SignalChannel(name=name, spec=SPEC, time_axis=AXIS, n_values=N_STEPS) for name in CHANNELS),
)
return Record(sources=(Source(name=SOURCE_NAME, signals=signals),), metadata={MetadataKey.ROW_ID: row_id})


def unlabelled_annotation(places: Sequence[PairLocation]) -> Annotation:
"""Return the source locations of published pairs that have no answer label."""
pairs: list[JsonValue] = [place.metadata for place in places]
return Annotation(
key=AnnotationKey.UNLABELLED_PAIRS,
value=len(places),
description="How many closed pairs have an empty published answer. They are missing labels and have no task.",
metadata={"pairs": pairs},
)
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# huggingface_hub downloads the source dataset from the HuggingFace Hub; 1.1 adds tqdm_class.
huggingface_hub>=1.1
# h5py reads the signals file, time_series_data.h5.
h5py>=3.10
Loading
Loading