Skip to content

feat: EXPKITS-1525 adopt FlowData v0.7.1 for SDK 1.0 - #240

Merged
zmolnar merged 6 commits into
mainfrom
feature/EXPKITS-1525-flowdata-upgrade
Nov 16, 2026
Merged

zmolnar merged 6 commits into
mainfrom
feature/EXPKITS-1525-flowdata-upgrade

Conversation

@zmolnar

@zmolnar zmolnar commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Goal

EXPKITS-1525: Adopt FlowData v0.7.1 as the generator foundation for SDK 1.0.

Change

Before

Payload IDs depended on schema contents, coupling typed readers to exact generated snapshots.

After

  • Import checksum-verified generator sources and provenance, with OPK formatting/licence decoration.
  • Generate stable root-name payload IDs, explicit producer identity and SDK name/major compatibility across C++, Python, Rust, TypeScript and the live Python bridge.
  • Preserve attachment to a compatible running Python interpreter; align development package dependencies and isolate fixtures with the locked FlatBuffers runtime. Reuse the image wheel with checksum validation; a missing wheel retains the download fallback.
  • Replace the old payload-ID scheme without aliases or packet conversion. Payload processing semantics are unchanged in this foundation layer.
  • Match flatc's enum-first namespace lookup, including a parent enum shadowed by a nearer table. Add a generated C++ compilation regression for scalar, vector, optional and struct fields.
  • Restore the generator documentation link to the repository AGENTS instructions; document local generator adaptations.

Testing

Coverage and historical validation

Generation/drift checks and native, browser, Python and Rust regressions cover packet compatibility, producer identity and interpreter attachment. generate, check and fixture regressions pass with --network none; corrupt cached wheels are rejected. All 135 SDK-tool tests pass with normal networking. This is unreleased SDK 1.0 work; publication is separate.

Post-rebase evidence

The 2026-11-16 assembled-stack build, native/WebUI/Plumber tests, SDK/bundle checks
and documentation validation are recorded in the Testing section of #276
at aad8a7f218a2. These results supplement the earlier coverage above; they are not
standalone local reruns at every lower PR head.

Reproduction commands

From the repository root inside the supported OPK development container, with
the pinned submodules initialized and the normal model/media setup available:

NINJAFLAGS=-j2 ./scripts/build.sh debug true
meson test -C development/build-active --print-errorlogs --num-processes 2
./scripts/perception-sdk.sh check
./scripts/opksink-web.sh test
./scripts/opksink-web.sh check
PYTHONPATH=tools/plumber:generated/open_perception_kit/python/src python3 -m unittest discover -s tools/plumber/tests

From the host checkout, run ./scripts/gen-doc.sh to generate and validate the
documentation and diagrams in build output.

Links

Part of the breaking, unreleased SDK/product 1.0.0 refactor. Producers and consumers
must use compatible regenerated snapshots. Stable release packaging/publication is separate.

@zmolnar
zmolnar requested review from a team as code owners November 3, 2026 15:27
@zmolnar
zmolnar added this pull request to stack #248 November 5, 2026 06:56
@zmolnar
zmolnar force-pushed the feature/EXPKITS-1525-flowdata-upgrade branch 2 times, most recently from 79a7e85 to bb9cdec Compare November 11, 2026 10:31
Comment thread tools/perception/generate.py Resolved
Comment thread tools/flowdata-sdk/docs/README.md Resolved
@zmolnar
zmolnar force-pushed the feature/EXPKITS-1525-flowdata-upgrade branch 2 times, most recently from 4dcfc14 to 966ecf1 Compare November 12, 2026 07:08
Szendvics
Szendvics previously approved these changes Nov 12, 2026

@Szendvics Szendvics left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rechecked the fixes at 966ecf1e. Nit: The remaining enum-resolution edge case is reproducible with a synthetic schema but does not affect the current OPK schemas, so I don't recommend changing the implementation for it.

Task: EXPKITS-1525

Signed-off-by: Zoltan Molnar <zoltan.molnar@arm.com>
Task: EXPKITS-1525

Signed-off-by: Zoltan Molnar <zoltan.molnar@arm.com>
Task: EXPKITS-1525

Signed-off-by: Zoltan Molnar <zoltan.molnar@arm.com>
Task: EXPKITS-1525

Signed-off-by: Zoltan Molnar <zoltan.molnar@arm.com>
Task: EXPKITS-1525

Signed-off-by: Zoltan Molnar <zoltan.molnar@arm.com>
Task: EXPKITS-1525

Reuse the verified image wheel with isolated Python, and match flatc enum-first type lookup in the generated bridge.

Signed-off-by: Zoltan Molnar <zoltan.molnar@arm.com>
@zmolnar
zmolnar force-pushed the feature/EXPKITS-1525-flowdata-upgrade branch from 966ecf1 to 9a5985b Compare November 16, 2026 12:50

@Szendvics Szendvics left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved as is.

@zmolnar
zmolnar enabled stacked auto-merge (squash) November 16, 2026 14:03
@zmolnar
zmolnar merged commit 30c793c into main Nov 16, 2026
19 checks passed
@zmolnar
zmolnar deleted the feature/EXPKITS-1525-flowdata-upgrade branch November 16, 2026 14:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants