Skip to content

Centralizing and simplifying transformation system in SpatialData - #1291

Draft
ajkswamy wants to merge 31 commits into
mainfrom
transformation_manager
Draft

Centralizing and simplifying transformation system in SpatialData#1291
ajkswamy wants to merge 31 commits into
mainfrom
transformation_manager

Conversation

@ajkswamy

@ajkswamy ajkswamy commented Oct 27, 2026

Copy link
Copy Markdown
Collaborator

Background and problems

Currently, transformation information in SpatialData objects are stored individually and independently in constituent spatial elements. This system makes the following complicated to different degrees:

  • I/O/roundtrip with NGFF 0.6 files
  • applying transformations defined for an element to an element linked to it
  • defining a network of transformation between elements that are beyond single hop transformations

Solution

  • Centralize all transformation information into a central multi-directional graph data structure at the root of SpatialData
  • Refactor all usage of ".attrs" of spatial elements to use information from the graph. Deprecate these usages/functions/methods
  • Add ergonomic methods to SpatialData to help specify transformation information from/to/between elements. Deprecate specifying transformation when create spatial elements from parse... methods of models
  • Remove deprecated code
  • ....

ajkswamy and others added 30 commits October 27, 2026 13:27
@ajkswamy
Transformation manager foundation (#1164)
15d8f00 
* chore: added pandas-stubs as dev dependency; helps with type checking

* chore: added local hatch.toml to gitignore

* chore: Add .kilo and plans to .gitignore

* feat: added literals and types for element types and group names

* feat: added .coverage and htmlcov to gitignore

* feat: transformation manager at the root of spatialdata object

* fix: io_zarr.py mixup between ELEMENT_TYPE_VECTOR/RASTER

* fix: avoid exposing TransformationManager in spatialdata __init__.py

* fix: element associated to cs -> element belonging to cs

* refac: transformation_manager now uses nx.MultiDiGraph

* refac: tests to match new transformation manager implementation

* fix: restored unintentionally removed functions + others fixes

* feat: custom error messages for transformation graph elements

* feat: custom exceptions for transform manager

* refac: transform manager attributes private; added missing method

* feat: more custom error and warnings + fixes

* feat: more test coverage + fixes

* feat: support for transformation management in graph with mutliple edges

* fix: fixes in tests + renaming for readability

* fix: typing Affine Transform

* feat: TransformationManager return Sequence instead of list

* feat: made TransformationManager.graph type specific over node type

* fix: removed internal attribute access warnings

* refac: rename check_if... methods to assert_... + one additional method

* fix: node type spec for TransformationManager._graph

* fix: made edge key definition from transforms more robust

* refac: method rename

* refac: TransformationManager get/remove methods don't raise error if edges are missing

* feat: new test for TransformationManager.add_transformation

* fix: better error messages and edge case handling for when transformation path is ambiguous

* feat: TransformationManager edge (key) definition made stronger

* fix: simplified access to attributes of TransformationManager

* fix: TransformationManager.unset_element is now private

* fix: throw error a path has one node

* fix: simplified checking if coordinate system has associated transforms

* fix: improved Transformation Manager code quality with better typing

* fix: improved Transformation Manager exception naming and messaging

* fix: improved Transformation Manager documentation

* feat: added tests for Transformation Manager

* fix: used explicit syntax for type definitoin

* refac: typo

* fix: removed TransformationManager import from spatialdata __init__.py

* fix: TransformationManager, simplified code using custom error
@ajkswamy
Transformation manager foundation (#1164)
89b8667 
* chore: added pandas-stubs as dev dependency; helps with type checking

* chore: added local hatch.toml to gitignore

* chore: Add .kilo and plans to .gitignore

* feat: added literals and types for element types and group names

* feat: added .coverage and htmlcov to gitignore

* feat: transformation manager at the root of spatialdata object

* fix: io_zarr.py mixup between ELEMENT_TYPE_VECTOR/RASTER

* fix: avoid exposing TransformationManager in spatialdata __init__.py

* fix: element associated to cs -> element belonging to cs

* refac: transformation_manager now uses nx.MultiDiGraph

* refac: tests to match new transformation manager implementation

* fix: restored unintentionally removed functions + others fixes

* feat: custom error messages for transformation graph elements

* feat: custom exceptions for transform manager

* refac: transform manager attributes private; added missing method

* feat: more custom error and warnings + fixes

* feat: more test coverage + fixes

* feat: support for transformation management in graph with mutliple edges

* fix: fixes in tests + renaming for readability

* fix: typing Affine Transform

* feat: TransformationManager return Sequence instead of list

* feat: made TransformationManager.graph type specific over node type

* fix: removed internal attribute access warnings

* refac: rename check_if... methods to assert_... + one additional method

* fix: node type spec for TransformationManager._graph

* fix: made edge key definition from transforms more robust

* refac: method rename

* refac: TransformationManager get/remove methods don't raise error if edges are missing

* feat: new test for TransformationManager.add_transformation

* fix: better error messages and edge case handling for when transformation path is ambiguous

* feat: TransformationManager edge (key) definition made stronger

* fix: simplified access to attributes of TransformationManager

* fix: TransformationManager.unset_element is now private

* fix: throw error a path has one node

* fix: simplified checking if coordinate system has associated transforms

* fix: improved Transformation Manager code quality with better typing

* fix: improved Transformation Manager exception naming and messaging

* fix: improved Transformation Manager documentation

* feat: added tests for Transformation Manager

* fix: used explicit syntax for type definitoin

* refac: typo

* fix: removed TransformationManager import from spatialdata __init__.py

* fix: TransformationManager, simplified code using custom error
@Tomaz-Vieira @ajkswamy
Creates the graph module based on the Ngff* classes
db849b7 
Cleans up and modernizes the NGff classes so that they have stronger
invariants and guarantee that they always have input and output.

Reading and writing to zarr is done via ome-zarr-models-py.

Uses graph module as part of io_raster.py::try_read_ngff06_multiscale
to interpret Ngff trnasformations and produce an output that could be
added to the new graph implementation.
@Tomaz-Vieira @ajkswamy
addresses some PR comments, simplifies MapAxis
dccc3f6
@Tomaz-Vieira @ajkswamy
Adds ProjectAxisEdge, makes inverse return optional
950ffd7
@Tomaz-Vieira @ajkswamy
Moves exceptions to exceptions.py, edge names default to None
0a2f374
@Tomaz-Vieira @ajkswamy
Use custom exceptions everywhere, adds Raises to docstrings
d29a2a4
@Tomaz-Vieira @ajkswamy
Fix Affine.from_affine_matrix slicing]
3618cdd
@Tomaz-Vieira @ajkswamy
Fix coords bug when reading from OMEZarrMultiscale. Some cleanup
4313c56
@Tomaz-Vieira @ajkswamy
Adds test for parsing OMEZarrMultiscale
9c2e1d3
@Tomaz-Vieira @claude @ajkswamy
Adds transformations tests, fixes some slicing bugs
f94498e 
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
@Tomaz-Vieira @ajkswamy
Adds basic .sel test to parsed multiscale
c0da25f
@Tomaz-Vieira @ajkswamy
Removes unused class
3a94382
@Tomaz-Vieira @ajkswamy
Adds more slicing tests to parsed multiscales
129fd2d
@Tomaz-Vieira @ajkswamy
Adds/fixes comments in try_parse_ngff06_multiscales
e050c10
@Tomaz-Vieira @ajkswamy
Removes to_model methods for now
5836cd0
@Tomaz-Vieira @ajkswamy
Moves AxisParsingException to exceptions.py
de5718c
@Tomaz-Vieira @ajkswamy
Moves AxisParsingException back into vert.py to prevent circular imports
d58038d
@Tomaz-Vieira @claude @ajkswamy
Fixes docstrings, list exceptions in "Raises"
55e1ba5 
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
@Tomaz-Vieira @ajkswamy
Incorporates parse_project_axis
d802aa8
@Tomaz-Vieira @claude @ajkswamy
Fixes parsing translatiion model getting bad name
c18df09 
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
@Tomaz-Vieira @claude @ajkswamy
Adds test for parsing *Edge transforms from ngff
c10c758 
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
@Tomaz-Vieira @ajkswamy
Fixes dependencies on ome-zarr and ome-zarr-models
b6d4a66
@Tomaz-Vieira @ajkswamy
Cleans up transform_points for rotation and affine
ec7f22b
@Tomaz-Vieira @ajkswamy
Fixes renamed fields in ome-zarr-models
05a28c7
@Tomaz-Vieira @ajkswamy
Use frozen dataclasses for CoordSystems and Axis. Fix MapAxisEdge
98c1b37
@Tomaz-Vieira @ajkswamy
Enforces Rotation matrix is orthonormal with det==1
834f921
@Tomaz-Vieira @ajkswamy
Bump xarray so we can use RangeIndex
6ca7e21
@Tomaz-Vieira @ajkswamy
Removes try-from_model_or_defaul for now
0c3ae64
@Tomaz-Vieira @ajkswamy
Emits warning on AffineEdge.inverse when condition is big
1951436
@codecov

codecov Bot commented Oct 27, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.97912% with 95 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.28%. Comparing base (1d5dcbe) to head (475d15b).

Files with missing lines Patch % Lines
src/spatialdata/transformations/graph/edge.py 86.24% 59 Missing ⚠️
src/spatialdata/transformations/graph/vert.py 78.75% 17 Missing ⚠️
src/spatialdata/_io/io_raster.py 74.19% 16 Missing ⚠️
...atialdata/_core/transformation_manager/__init__.py 98.63% 2 Missing ⚠️
src/spatialdata/_io/io_zarr.py 80.00% 1 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #1291      +/-   ##
==========================================
- Coverage   91.43%   91.28%   -0.15%     
==========================================
  Files          53       57       +4     
  Lines        8395     9238     +843     
==========================================
+ Hits         7676     8433     +757     
- Misses        719      805      +86
Files with missing lines Coverage Δ
src/spatialdata/__init__.py 95.65% <ø> (ø)
src/spatialdata/_core/spatialdata.py 93.47% <100.00%> (+0.01%) ⬆️
...ialdata/_core/transformation_manager/exceptions.py 100.00% <100.00%> (ø)
src/spatialdata/_types.py 100.00% <100.00%> (ø)
src/spatialdata/_utils.py 85.27% <100.00%> (ø)
src/spatialdata/transformations/__init__.py 100.00% <100.00%> (ø)
src/spatialdata/_io/io_zarr.py 88.42% <80.00%> (ø)
...atialdata/_core/transformation_manager/__init__.py 98.63% <98.63%> (ø)
src/spatialdata/_io/io_raster.py 84.61% <74.19%> (-3.05%) ⬇️
src/spatialdata/transformations/graph/vert.py 78.75% <78.75%> (ø)
... and 1 more

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ajkswamy @Tomaz-Vieira