Repository navigation
Conversation
Parse the BattLeDIM 2020 SCADA and leak tables (semicolon-separated, decimal comma, 5-minute grid), the leak list of the generating configuration, the 2018 repair report and the L-Town EPANET links. Joining the two years checks the grid, the sensor lists and the agreement between the configuration, the leak tables, the repair report and the network model, and raises TimeNetConnectorError on any mismatch.
Download 13 files of Zenodo record 4017659, each pinned by sha256, and build one record of the 2018-2019 recording: 119 float64 signals (33 pressure heads, 3 flows, 1 tank level, 82 metered demands) on a 5-minute grid, with every leak as record annotations over its active interval. Two-week windows, one every 7 days, give four tasks about their last week: the types of new leaks, their start times, the mean leak outflow, and a rule-generated text that names each new or repaired leak, its size and the nearest pressure sensor by network links. Windows split 8/1/1 by calendar block across both years, because 2019 repeats most of the 2018 metered demands.
Write a two-year synthetic release in the source layout and check the record, the leak annotations, the window tasks and their text, the calendar-block split, a storage round trip, and that a gap, a missing value or a disagreement between the source files raises.
… more of the release - The text no longer states an incipient leak's peak when it falls after the window; it says the leak was still growing. Zero rounded outflow reads as "no measurable outflow yet". - Active leaks at the end of a window come from the leak list, not from rounded outflow, so the text cannot contradict its own new leaks. - The build fails if a leak start or repair falls outside every target week, if the repair report misses or repeats a 2018 repair, if a sensor is listed twice, or if a CSV cell is not a timestamp or a number. - Share one microsecond conversion and drop dead code.
…embargo overlapping windows - The leaks become dataset annotations instead of record annotations. Every task reads the record, and loaders return a record's annotations with each item, so the record now holds sensor signals only. - A window whose context week is a validation or test target week of another partition gets no tasks, so training never sees the signals of a held-out target week. - Small outflows keep their second decimal; only zero reads as no outflow. - Only p800 may keep outflow one step past its configured end, abrupt leaks must peak at their start, and malformed configuration sections raise TimeNetConnectorError. - Compute the total outflow once, keep the window geometry in one place, return a Table from the year join and read headers as utf-8-sig.
Author
|
Our team is focusing on a different dataset for the hackathon! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds
kios/battledim, a connector for BattLeDIM 2020, the Battle of the Leakage Detection and Isolation Methods (Zenodo 4017659, CC BY 4.0).Why it matters:
The connector downloads 13 of the record's files from Zenodo. Each one is pinned by sha256; the full download is about 190 MB.
Mapping
Record: one Record for the continuous recording, 2018-01-01 00:00 → 2019-12-31 23:55, 210,240 steps.
L-TOWN.inp.Leaks: each leak becomes one dataset annotation (
leak, value = pipe id), with type, start, end, peak, hole size, peak outflow and repaired status as metadata. They are deliberately not on the record: every task reads the record, and loaders such asTimeFTorchDatasetreturn a record's annotations in each item, so the record holds only sensor signals.Windows: two-week windows, one every 7 days (103 windows; 98 after the embargo below). Each gives four tasks about its last week, all scoped to the window:
ClassificationTaskabrupt,incipient}: the types of leaks starting in the target week, empty if noneTemporalLocalizationTask(exhaustive)ScalarPredictionTaskmean_leak_outflowover the target week, in m³/hAnswerTaskThe text targets come from fixed templates filled from the configuration, the leak tables and
L-TOWN.inp. No model writes them. The README quotes the templates verbatim. A text never states a fact from after its window: an incipient leak whose peak falls later is described as still growing.Decisions
Split: grouped by calendar block (four ISO weeks), 8/1/1 with
StratifiedSplitter, seed 42.Time zone: the release's naive timestamps are read as UTC; it states no zone.
Strict reading: the build fails on a gap, a missing or non-finite value, or any disagreement between:
Nothing is filled. The release passes all checks.
Source issues documented in the README
Files
No
requirements.txt: the connector uses only core dependencies (numpy, pyarrow, PyYAML viatimenet[build]). The test writes a synthetic release, so the repo ships no dataset bytes.How to load
Validation
make check(ruff format, ruff check, ty)make check-ci(all pre-commit hooks + ty on 3.11)make test-unitmake testtest_an_isolated_build_records_its_environment. It fails identically on cleanupstream/mainon our machine, because the isolated build picks a local conda interpreter that has pytest installed. Unrelated to this PR.make test-connectorskios/battledimmake license-checktimenet-build build kios/battledimon the real releaseRead-back of the built release:
Example (target week 2018-03-05):
classification→["abrupt"]temporal_localization→TimePoint(2018-03-05 15:45)mean_leak_outflow→47.578m³/hanswer→ "A sudden leak started on pipe p673 on 2018-03-05 15:45, with a hole of 22.9 mm and an outflow of up to 28.4 m3/h in this window. The nearest pressure sensor is n229, 2 links away. At the end of the window 4 leaks are active, with a total outflow of 52.8 m3/h."Licence and citation
The data is CC BY 4.0 (Zenodo record and KIOS-Research/BattLeDIM). Cite:
Limitations
Contributed by Team RAMadan during the TimeNet hackathon, Round 1 (November 2026).
Checklist
There is no PR template in the repo; this mirrors
CONTRIBUTING.mdandAGENTS.md.feature/kios-battledim)make check,make check-ci,make test-unit,make license-checkpassmake test-connectorspasses;make testpasses except one environment-dependent test that also fails on cleanmainlocally (see Validation)tygreendataset-card-v1; licence, citation and source URL setTimeNetConnectorError; nothing is filledarea:connectors: we can't add labels on this repo, so could a maintainer add it? Thanks!