Skip to content

refactor(connectors): pass annotations to the constructor in every connector - #827

Merged
masquare merged 6 commits into
refactor/huggingface-convert-filesfrom
refactor/annotations-at-construction
Nov 20, 2026
Merged

masquare merged 6 commits into
refactor/huggingface-convert-filesfrom
refactor/annotations-at-construction

Conversation

@masquare

Copy link
Copy Markdown
Contributor

Every connector now attaches annotations the same way: as an annotations=[...] list when it builds the owner. This is the cleanup sweep on top of the stack, kept apart so the PRs below do not grow.

What changes

  • TimeFDataset(metadata=..., annotations=[...]): the dataset takes annotations at construction like a Record, a Source, a Signal and a task. It attaches each one as annotate() does, in order.
  • sensorcaps, tsqa, tsaqa_benchmark, arfbench, ptb_xl and sleep_edfx pass the list to the constructor. The loops over key and value pairs are gone.
  • The docs and the connector skill teach construction as the default.
# before
dataset = TimeFDataset(metadata=self.metadata())
for key, value in (
    ("source_repository", self.HF_REPO),
    ("source_revision", self.SOURCE_REVISION),
):
    dataset.annotate(Annotation(key, value=value))

# after
dataset = TimeFDataset(
    metadata=self.metadata(),
    annotations=[
        Annotation("source_repository", value=self.HF_REPO),
        Annotation("source_revision", value=self.SOURCE_REVISION),
    ],
)

Where annotate() stays

  • A task refers to the returned occurrence (ecg_qa_cot, tsaqa_benchmark, hello_world).
  • The owner already exists, for example a Record imported from a parent dataset.
  • One annotation is attached to several owners, or a spanned annotation is attached to a Signal (sleep_edfx, hello_world).

The built datasets do not change.

Checks

  • make check and make test-connectors pass.
  • The core, registry and torch tests pass.

@masquare
masquare added this pull request to stack #826 November 19, 2026 18:04
@masquare
masquare requested a review from karhu November 19, 2026 18:04
@masquare
masquare marked this pull request as ready for review November 19, 2026 18:04

@karhu karhu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Receivers of annotation objects behave inconsistently across the stack, some copy the annotation objects, some don't. annotate(), add_annotations(), TimeFDataset(annotations=...) (this PR) and LabelSet(annotations=...) (#797) attach a copy. The Record, Source, Signal and Task constructors keep the object as given. Consequently we have two similar ways to annotate a record, but one breaks at write time:

shared = Annotation("condition", value="healthy")

# fails at write time: both records hold the same occurrence
ds.add_record(Record(sources=[...], annotations=[shared]))
ds.add_record(Record(sources=[...], annotations=[shared]))
# put_dataset -> TimeFValidationError: an occurrence of annotation
#   'condition' is attached in more than one place

# works: annotate() attaches a copy each time
ds.add_record(Record(sources=[...])).annotate(shared)
ds.add_record(Record(sources=[...])).annotate(shared)

@masquare
masquare removed this pull request from stack #826 November 20, 2026 07:25
…nnector

TimeFDataset takes annotations like the other owners. The connectors
pass annotations=[...] when they build a dataset, a Record or a task,
and call annotate() only for an owner that exists or when a task needs
the returned occurrence.
…onstructor

Record, Source, Signal and Task now copy each annotation passed as annotations=, as annotate() does. So one Annotation object can go to several owners, and the writer no longer rejects it as attached in more than one place. The reader, imports, edits and with_references keep the occurrences that already belong to the object.
…e stored ones

The reader passed the stored annotations to each constructor, which copied
them, and then put the stored tuple back. That copied every occurrence once
per read for nothing. The reader now builds the entity without annotations,
validates each stored annotation as construction does, and attaches it.
Source gets _validate_annotation like Record and Signal for that.
…ot hold

The Hub source now lives in source_url and source_revision of the
dataset metadata, so the example does not repeat it as annotations.
@masquare
masquare force-pushed the refactor/annotations-at-construction branch from 8636cdf to 9addb4e Compare November 20, 2026 15:42
@masquare
masquare added this pull request to stack #829 November 20, 2026 15:42
@masquare
masquare requested a review from karhu November 20, 2026 15:45
@masquare
masquare merged commit df1d9d6 into main Nov 20, 2026
4 checks passed
@masquare
masquare deleted the refactor/annotations-at-construction branch November 20, 2026 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants