Skip to content

feat(dataset): walk every annotation owner and list all keys in the schema - #812

Merged
masquare merged 7 commits into
feature/connector-api-ergonomicsfrom
feature/annotation-owner-walk
Nov 20, 2026
Merged

masquare merged 7 commits into
feature/connector-api-ergonomicsfrom
feature/annotation-owner-walk

Conversation

@masquare

@masquare masquare commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Add one generic walk over every object that can hold annotations, and use it everywhere the code reads annotations of the whole dataset.

What changes

  • annotatable_entities() on Task yields the entities of a task that can carry annotations. A plain task yields itself, and a task class that holds another annotatable entity yields that entity too.
  • walk_annotations() on Record, Task and the dataset yields their annotation occurrences. The dataset yields its own, then those of each Record, then those of each task.
  • Every owner gets annotation(key), which returns the first annotation with a key.
  • derive_schema and describe read every owner through the walk.

Behaviour change

The schema now also lists the annotation keys of the dataset and of tasks, not only those of Records, Sources and Signals. A key with conflicting descriptors on two owners now raises an error. For a streamed dataset, the writer adds the keys of the streamed tasks while it streams them. It raises on a conflict before the manifest is written.

This PR does not change label sets. The next PR in the stack makes a label set an annotation owner through this walk.

Checks

  • make check and make test-connectors pass.
  • The core suite passes per directory.

@masquare
masquare added this pull request to stack #813 November 18, 2026 23:46
@masquare masquare added the area:core Changes packages/timenet label Nov 18, 2026
@masquare
masquare removed this pull request from stack #813 November 19, 2026 05:19
@masquare
masquare added this pull request to stack #816 November 19, 2026 05:19
@masquare
masquare requested a review from karhu November 19, 2026 05:45
@masquare
masquare marked this pull request as ready for review November 19, 2026 05:45
@masquare
masquare force-pushed the feature/annotation-owner-walk branch from f1eff3f to 7ce78ef Compare November 19, 2026 16:17
@masquare
masquare removed this pull request from stack #816 November 19, 2026 16:18
@masquare
masquare changed the base branch from main to feature/connector-api-ergonomics November 19, 2026 16:18
@masquare
masquare added this pull request to stack #824 November 19, 2026 16:18
@masquare
masquare force-pushed the feature/annotation-owner-walk branch from 7ce78ef to aedf8ac Compare November 19, 2026 17:34
@masquare
masquare removed this pull request from stack #824 November 19, 2026 17:34
@masquare
masquare added this pull request to stack #826 November 19, 2026 17:34
@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

✅ CI checks passed

Check Status
Quick checks ✅ Passed
Tests (Python 3.13) ✅ Passed
Tests (Python 3.11) ✅ Passed

Updated for d4a95de · run #8631

@masquare
masquare force-pushed the feature/annotation-owner-walk branch from aedf8ac to 27fb02d Compare November 19, 2026 17:48

@karhu karhu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR description looks out-of-date, otherwise LGTM

…chema

Add annotation_owners() to Record, Task and the dataset. The dataset
yields itself, then each Record with its Sources and Signals, then each
task. walk_annotations() on Record, Task and the dataset goes through it,
and every owner gets annotation(key), which returns the first annotation
with a key.

derive_schema and describe read every owner through the walk, so the
schema now also lists the annotation keys of the dataset and of tasks.
The keys of streamed tasks are added while the writer streams them, and
a key with conflicting descriptors raises before the manifest is written.
TimeFDataset.annotation_owners() had no caller apart from
walk_annotations(). walk_annotations() now yields the dataset
occurrences, then those of each Record, then those of each task.
Record and Task keep annotation_owners(), which the load plan uses.
The name says what the method gives: the entities of a Record or a task
that can carry annotations.
Record.annotatable_entities() had no caller apart from
Record.walk_annotations(), so it is gone and walk_annotations() walks
the Sources and Signals itself again. Task keeps annotatable_entities(),
because a task class can hold another annotatable entity.
Nothing in the source needs a lookup by key. Callers read the
annotations directly.
Task.annotatable_entities() only yielded the task. walk_annotations()
now returns the task's annotations, and a task class that holds another
annotated object can add its occurrences there.
@masquare
masquare force-pushed the feature/annotation-owner-walk branch from 27fb02d to d4a95de Compare November 20, 2026 07:25
@masquare
masquare removed this pull request from stack #826 November 20, 2026 07:25
@masquare
masquare added this pull request to stack #829 November 20, 2026 07:25
@masquare
masquare merged commit 790a71f into main Nov 20, 2026
4 of 8 checks passed
@masquare
masquare deleted the feature/annotation-owner-walk branch November 20, 2026 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:core Changes packages/timenet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants