Skip to content

feat(connectors): add Petrobras 3W oil-well events dataset - #833

Open
RenatoGallicola wants to merge 5 commits into
OpenTSLM:mainfrom
RenatoGallicola:feature/petrobras-three-w
Open

RenatoGallicola wants to merge 5 commits into
OpenTSLM:mainfrom
RenatoGallicola:feature/petrobras-three-w

Conversation

@RenatoGallicola

@RenatoGallicola RenatoGallicola commented Nov 20, 2026 •

Copy link
Copy Markdown

Description

3W is a Petrobras dataset of rare undesirable events in offshore oil wells. Version 2.0.0 holds 2,228
instances: 1,119 real ones from 40 wells, 1,089 simulated, and 20 hand-drawn. Each instance is one
Parquet file with 27 process variables at 1 Hz and a per-second label. The label marks normal
operation and the transient and steady phases of one of nine event types. TimeNet had no connector
for it, and the industrial domain has no dataset yet. The data is CC BY 4.0.

This PR adds the connector for petrobras/three-w. The source repository stores the Parquet files
in plain git, without LFS. So the connector pins each of the 2,228 files to a raw GitHub URL at
commit c86cc7c7 and checks its SHA-256 from files.csv. generate_manifest.py rebuilds that list
for a later commit.

Each instance becomes one Record. Its metadata holds the instance name, the well, the provenance
(real, simulated, or drawn), the event code, and the first timestamp. 3W states no time zone, so
the first timestamp stays text and the record has no absolute start_time. The variables become Signals in seven Sources, one for each measuring point: downhole gauge, subsea
Xmas tree, production choke, gas lift line, service line, valve states, and well status. They share
six specs by quantity, for example pressure in pascal. Each Signal keeps the 3W column as its name
and the dataset.ini description in its metadata.

ClassificationTask(
    inputs=(record,),
    targets=("Spurious Closure of DHSV",),
    label_set=EVENT_TYPES,
    split=Split.TEST,
)
TemporalLocalizationTask(
    inputs=(record,),
    targets=(TimeInterval.seconds(7355, 12504),),
    mode=LocalizationMode.EXHAUSTIVE,
    split=Split.TEST,
)
  • Every instance gets one ClassificationTask over the ten event types. An instance whose event has
    a labeled transient phase also gets one TemporalLocalizationTask for it. That gives 2,228 and
    1,142 tasks.
  • The per-second labels become event_phase interval annotations: normal, transient, or
    steady. A label that does not belong to the instance's event raises TimeNetConnectorError, and
    so do timestamps that are not one second apart.
  • Nothing is filled in. A variable that is empty for a whole instance gets no Signal, because that
    well has no such sensor. A variable with gaps keeps them as nulls, with the reason not_recorded.
    A variable that holds one value for a whole instance keeps it and gets a constant_value
    annotation, because that usually means a stuck sensor. The connector reads null counts and value
    ranges from the Parquet column statistics, so convert reads no values.
  • The release has no split. Real instances are grouped by well and stratified by event type, with
    3/1/1 folds, so no well is in two splits. Simulated and drawn instances come from no well, so they
    are always in train.
  • The card states three details. The splits are generated by TimeNet. Events 3 and 6 have real
    instances from two wells only, so validation or test can miss them. The state codes 0-8 are
    kept as released, because the dataset files do not explain them. The card also gives the source
    revision, the URL of the data license in the release, and the citation of the 2.0.0 paper.
  • A README.md beside the connector states these decisions with the numbers measured on the
    release, as the Sleep-EDF connector does.

Test plan

  • Command: uv run pytest packages/timenet-connectors/src/timenet_connectors/datasets/petrobras
    • Coverage: Converts synthetic Parquet files. Checks the Signals of an instance, the specs and
      descriptions, the dropped empty variables, the kept nulls, the constant_value annotation, the
      phase intervals, the localization targets, the split by well, the errors for a foreign label and
      for irregular timestamps, a storage round trip, and the manifest. All 11 tests pass.
  • Command: uv run timenet-build build petrobras/three-w --local-registry <dir>
    • Coverage: Downloads the real release from the pinned URLs and builds it in about 18 minutes
      (1.1 GB). describe() on the stored dataset shows 2,228 records, 2,228 classification tasks, and
      1,142 localization tasks.
  • Command: uv run scripts/check_connectors.py (make test-connectors)
    • Coverage: All 14 connectors pass.
  • Command: ruff format --check ., ruff check ., and ty check on the connector, also with
    --python-version 3.11
    • Coverage: All pass.
  • Not passed here: make test on Windows. 40 core tests fail and 29 error, and the same tests fail
    on an untouched main on this machine. This PR changes no core code.

@RenatoGallicola
RenatoGallicola marked this pull request as ready for review November 20, 2026 15:30
Main changed the connector API. A label set carries its name as an
annotation, a Hugging Face connector passes its Hub settings as class
keywords and reads a mapping from part to path, BaseConnector names the
whole download result, and Annotation takes its value by keyword.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants