Skip to content

Add validated CT/SEG tensor and patient-split tutorial - #116

Merged
CPBridge merged 5 commits into
ImagingDataCommons:masterfrom
denis-samatov:tutorial/ct-seg-tensors
Oct 22, 2026
Merged

CPBridge merged 5 commits into
ImagingDataCommons:masterfrom
denis-samatov:tutorial/ct-seg-tensors

Conversation

@denis-samatov

Copy link
Copy Markdown
Contributor

Add a self-contained tutorial for turning a small public C4KC-KiTS cohort into validated CT/SEG tensors and a patient-disjoint train/test manifest.

Related to #115. This is a concrete implementation for review; the proposal's scope and notebook placement have not yet been confirmed by a maintainer.

Changes

  • Select four referenced CT/SEG pairs from the pinned IDC v25 catalog, with a Hugging Face catalog option.
  • Check source SOP references and frame geometry before conversion; reject unsupported geometry without silent resampling.
  • Export float32 HU images, separate binary mask channels preserving overlaps, DICOM LPS affines, source checksums and coded segment metadata.
  • Demonstrate a reproducible patient split and a minimal PyTorch Dataset.
  • Add 15 behavior tests covering ordering, HU/affine conversion, overlapping masks, incompatible pairs, missing CT instances and patient grouping; link the tutorial from the README.

Validation

Repeated locally on 13 October 2026, Python 3.12.14:

  • python -m pytest test/src/test_ct_seg_to_tensors.py -q: 15 passed.
  • python test/src/check_notebook_standards.py: all 34 notebooks passed.
  • nbformat validation and git diff --check: passed.
  • All 14 code cells executed in order through IPython, including the install cell, catalog selection, DICOM validation/export and PyTorch consumption. Real-data inputs were reused from a local cache.
  • Four patients; 77,970,396 bytes of selected DICOM; train/test patients 3/1; final checks passed for binary masks, consistent channels and patient separation.

Execution outputs are cleared in the submitted notebook to keep the diff small; its cell sources were checked against the fully executed copy. Google Colab execution has not been verified.

Review focus

Would this fit advanced_topics, or would collections_demos be preferable? Feedback on the validation boundary and the four-patient teaching example would be useful. This convenience sample is not a model-quality benchmark; enhanced CT, irregular grids and resampled segmentations are intentionally outside scope.

AI assistance was used for implementation and local validation.

@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@CPBridge

Copy link
Copy Markdown
Collaborator

Hi @denis-samatov thanks for this proposal. @fedorov and I discussed this and we really liked the idea and think it could be very useful for IDC users, however we were concerned that the original notebook had too much validation code that distracted from the main purpose of the tutorial (loading paired CTs and SEGs), and too little explanation of the key concepts. We also felt that conflating the cohort selection and image loading parts was unhelpful.

Rather than go backwards and forwards on this, I decided to propose a simpler version that Claude created using your initial notebook, by distilling the main points, removing much of the distracting validation code, and adding some text explanation to guide the reader through the concepts used. But the concept is your own.

I pushed this to the PR already. Perhaps you could look through it and see what you think and whether this would be useful to new IDC users?

@denis-samatov

denis-samatov commented Oct 22, 2026 via email

Copy link
Copy Markdown
Contributor Author

@CPBridge

Copy link
Copy Markdown
Collaborator

Thanks @denis-samatov good points/suggestions. I applied them in the last commit

@fedorov fedorov left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks really nice, thank you for working on this @denis-samatov and @CPBridge!

@CPBridge
CPBridge merged commit 416e1a8 into ImagingDataCommons:master Oct 22, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants