Repository navigation
feat(connectors): add Petrobras 3W oil-well events dataset - #833
Open
RenatoGallicola wants to merge 5 commits into
Open
RenatoGallicola wants to merge 5 commits into
RenatoGallicola wants to merge 5 commits into
Conversation
RenatoGallicola
marked this pull request as ready for review
November 20, 2026 15:30
Main changed the connector API. A label set carries its name as an annotation, a Hugging Face connector passes its Hub settings as class keywords and reads a mapping from part to path, BaseConnector names the whole download result, and Annotation takes its value by keyword.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
3W is a Petrobras dataset of rare undesirable events in offshore oil wells. Version 2.0.0 holds 2,228
instances: 1,119 real ones from 40 wells, 1,089 simulated, and 20 hand-drawn. Each instance is one
Parquet file with 27 process variables at 1 Hz and a per-second label. The label marks normal
operation and the transient and steady phases of one of nine event types. TimeNet had no connector
for it, and the
industrialdomain has no dataset yet. The data is CC BY 4.0.This PR adds the connector for
petrobras/three-w. The source repository stores the Parquet filesin plain git, without LFS. So the connector pins each of the 2,228 files to a raw GitHub URL at
commit
c86cc7c7and checks its SHA-256 fromfiles.csv.generate_manifest.pyrebuilds that listfor a later commit.
Each instance becomes one Record. Its metadata holds the instance name, the well, the provenance
(real, simulated, or drawn), the event code, and the first timestamp. 3W states no time zone, so
the first timestamp stays text and the record has no absolute
start_time. The variables become Signals in seven Sources, one for each measuring point: downhole gauge, subseaXmas tree, production choke, gas lift line, service line, valve states, and well status. They share
six specs by quantity, for example
pressurein pascal. Each Signal keeps the 3W column as its nameand the
dataset.inidescription in its metadata.a labeled transient phase also gets one TemporalLocalizationTask for it. That gives 2,228 and
1,142 tasks.
event_phaseinterval annotations:normal,transient, orsteady. A label that does not belong to the instance's event raisesTimeNetConnectorError, andso do timestamps that are not one second apart.
well has no such sensor. A variable with gaps keeps them as nulls, with the reason
not_recorded.A variable that holds one value for a whole instance keeps it and gets a
constant_valueannotation, because that usually means a stuck sensor. The connector reads null counts and value
ranges from the Parquet column statistics, so
convertreads no values.3/1/1 folds, so no well is in two splits. Simulated and drawn instances come from no well, so they
are always in train.
instances from two wells only, so validation or test can miss them. The
statecodes 0-8 arekept as released, because the dataset files do not explain them. The card also gives the source
revision, the URL of the data license in the release, and the citation of the 2.0.0 paper.
README.mdbeside the connector states these decisions with the numbers measured on therelease, as the Sleep-EDF connector does.
Test plan
uv run pytest packages/timenet-connectors/src/timenet_connectors/datasets/petrobrasdescriptions, the dropped empty variables, the kept nulls, the
constant_valueannotation, thephase intervals, the localization targets, the split by well, the errors for a foreign label and
for irregular timestamps, a storage round trip, and the manifest. All 11 tests pass.
uv run timenet-build build petrobras/three-w --local-registry <dir>(1.1 GB).
describe()on the stored dataset shows 2,228 records, 2,228 classification tasks, and1,142 localization tasks.
uv run scripts/check_connectors.py(make test-connectors)ruff format --check .,ruff check ., andty checkon the connector, also with--python-version 3.11make teston Windows. 40 core tests fail and 29 error, and the same tests failon an untouched
mainon this machine. This PR changes no core code.