Skip to content

feat: EXPKITS-1530 make mask semantics and image space explicit - #264

Merged
zmolnar merged 1 commit into
feature/EXPKITS-1529-gaze-measurementsfrom
feature/EXPKITS-1530-mask-semantics
Nov 16, 2026
Merged

zmolnar merged 1 commit into
feature/EXPKITS-1529-gaze-measurementsfrom
feature/EXPKITS-1530-mask-semantics

Conversation

@zmolnar

@zmolnar zmolnar commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Goal

EXPKITS-1530: Declare mask meaning, encoding and placement across producers and consumers.

Change

Before

Strings and rendering settings implied mask meaning. Crops lacked an independent image basis, absent annotations became defaults, and source-resolution composition could erase valid fine-grid instances. Plumber omitted mask families.

After

  • Reuse typed mask roots with Uint8/Uint32LE, exact storage and explicit image regions. Declare opacity meaning, semantic namespace/void values and binary instance membership.
  • Preserve optional instance categories, confidence and boxes; validate shapes, values and parser settings before publication.
  • Adapt merged segmentation controls and compare all mask families in Plumber. Preserve valid palettes, controls, graded alpha and non-accumulating overlap opacity.
  • Compose instance masks at bounded display/device-pixel resolution using the render transform. Resize/DPR changes refresh cached rasters; very large displays retain the documented allocation limit.

Testing

Coverage and historical validation

Native parsers/pipelines, wire/bridge, Plumber and browser pixel/control regressions cover cropped grids, void/class zero, malformed storage and overlap order. The fine-grid reproduction now renders 625 foreground pixels instead of zero; paused-frame resize and DPR changes are covered. Physical GPU execution is unverified.

Post-rebase evidence

The 2026-11-16 assembled-stack build, native/WebUI/Plumber tests, SDK/bundle checks
and documentation validation are recorded in the Testing section of #276
at aad8a7f218a2. These results supplement the earlier coverage above; they are not
standalone local reruns at every lower PR head.

Reproduction commands

From the repository root inside the supported OPK development container, with
the pinned submodules initialized and the normal model/media setup available:

NINJAFLAGS=-j2 ./scripts/build.sh debug true
meson test -C development/build-active --print-errorlogs --num-processes 2
./scripts/perception-sdk.sh check
./scripts/opksink-web.sh test
./scripts/opksink-web.sh check
PYTHONPATH=tools/plumber:generated/open_perception_kit/python/src python3 -m unittest discover -s tools/plumber/tests

From the host checkout, run ./scripts/gen-doc.sh to generate and validate the
documentation and diagrams in build output.

Links

Part of the breaking, unreleased SDK/product 1.0.0 refactor. Producers and consumers
must use compatible regenerated snapshots. Stable release packaging/publication is separate.

@zmolnar
zmolnar requested review from a team as code owners November 9, 2026 09:40
@zmolnar
zmolnar added this pull request to stack #248 November 9, 2026 09:40
@zmolnar zmolnar closed this Nov 9, 2026
@zmolnar zmolnar reopened this Nov 9, 2026
@zmolnar
zmolnar force-pushed the feature/EXPKITS-1530-mask-semantics branch from 0da1605 to 7fda690 Compare November 11, 2026 10:31
@zmolnar
zmolnar force-pushed the feature/EXPKITS-1530-mask-semantics branch 2 times, most recently from 26f5587 to 506c5f5 Compare November 11, 2026 12:40
@zmolnar
zmolnar force-pushed the feature/EXPKITS-1530-mask-semantics branch from 506c5f5 to dfd0311 Compare November 11, 2026 13:43
@zmolnar
zmolnar force-pushed the feature/EXPKITS-1530-mask-semantics branch 2 times, most recently from 5eb51f5 to 51ccad2 Compare November 12, 2026 07:08

@Szendvics Szendvics left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Three Minor findings and one non-blocking Nit at 51ccad28.

Comment thread development/web/src/mask-data.js Resolved
Comment thread development/tests/mask_pipeline_test.py Resolved
Comment thread development/tests/mask_pipeline_test.py Resolved
@zmolnar
zmolnar force-pushed the feature/EXPKITS-1530-mask-semantics branch from 51ccad2 to 9a80c00 Compare November 13, 2026 11:29
Szendvics
Szendvics previously approved these changes Nov 16, 2026
Task: EXPKITS-1530

Use typed bitmap encoding, explicit opacity meaning and source image space.
Preserve optional category and score availability, validate mask producers,
and align native/browser consumers, inspection and Plumber comparison.
Regenerate the SDK and bridge; cover malformed data, crop placement and
real mask pipelines. Record related findings under EXPKITS-1542.

Signed-off-by: Zoltan Molnar <zoltan.molnar@arm.com>
@zmolnar
zmolnar force-pushed the feature/EXPKITS-1530-mask-semantics branch from 9a80c00 to 05d1724 Compare November 16, 2026 12:50

@Szendvics Szendvics left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved as is.

@zmolnar
zmolnar enabled stacked auto-merge (squash) November 16, 2026 14:03
@zmolnar
zmolnar merged commit 805b826 into main Nov 16, 2026
31 of 36 checks passed
@zmolnar
zmolnar deleted the feature/EXPKITS-1530-mask-semantics branch November 16, 2026 14:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants