Skip to content

Catch Issues Decoding PNGs - #155

Merged
jrjohnson merged 1 commit into
mainfrom
catch-decode-exceptions
Nov 9, 2026
Merged

jrjohnson merged 1 commit into
mainfrom
catch-decode-exceptions

Conversation

@jrjohnson

Copy link
Copy Markdown
Owner

These type of errors shutdown the differ and leave us with no useful report. Catch them instead and output the error info.

See https://github.com/ilios/ember-simple-charts/actions/runs/42475065372/job/129449286737?pr=1507 for an example.

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

✅ Visual Diff Approved

❌ Visual Diff Report — FAILED

288 images compared: 270 different · 18 added

Details

Differences (270)

File Diff % Notes
chromium-1280x720-baseline-screenshot-for-checkerboard-png.png 100.00% ⚠️ Dimension mismatch (1280x4703, Data length: 24079360 → 1280x4898, Data length: 25077760)
chromium-1280x720-baseline-screenshot-for-color-grid-png.png 100.00% ⚠️ Dimension mismatch (1280x4703, Data length: 24079360 → 1280x4898, Data length: 25077760)
chromium-1280x720-baseline-screenshot-for-dimension-mismatch-on-color-depth-png.png 100.00% ⚠️ Dimension mismatch (1280x4703, Data length: 24079360 → 1280x4898, Data length: 25077760)
chromium-1280x720-baseline-screenshot-for-gradient-16bit-png.png 100.00% ⚠️ Dimension mismatch (1280x4703, Data length: 24079360 → 1280x4898, Data length: 25077760)
chromium-1280x720-baseline-screenshot-for-gradient-mixed-depth-png.png 100.00% ⚠️ Dimension mismatch (1280x4703, Data length: 24079360 → 1280x4898, Data length: 25077760)
chromium-1280x720-baseline-screenshot-for-greyscale-with-alpha-png.png 100.00% ⚠️ Dimension mismatch (1280x4703, Data length: 24079360 → 1280x4898, Data length: 25077760)
chromium-1280x720-baseline-screenshot-for-landscape-png.png 100.00% ⚠️ Dimension mismatch (1280x4703, Data length: 24079360 → 1280x4898, Data length: 25077760)
chromium-1280x720-baseline-screenshot-for-night-sky2-png.png 100.00% ⚠️ Dimension mismatch (1280x4703, Data length: 24079360 → 1280x4898, Data length: 25077760)
chromium-1280x720-baseline-screenshot-for-opaque-landscape-png.png 100.00% ⚠️ Dimension mismatch (1280x4703, Data length: 24079360 → 1280x4898, Data length: 25077760)
chromium-1280x720-baseline-screenshot-for-resized-banner-png.png 100.00% ⚠️ Dimension mismatch (1280x4703, Data length: 24079360 → 1280x4898, Data length: 25077760)
chromium-1280x720-baseline-screenshot-for-webapp2-png.png 100.00% ⚠️ Dimension mismatch (1280x4703, Data length: 24079360 → 1280x4898, Data length: 25077760)
chromium-1280x720-candidate-screenshot-for-checkerboard-png.png 100.00% ⚠️ Dimension mismatch (1280x4703, Data length: 24079360 → 1280x4898, Data length: 25077760)
chromium-1280x720-candidate-screenshot-for-color-grid-png.png 100.00% ⚠️ Dimension mismatch (1280x4703, Data length: 24079360 → 1280x4898, Data length: 25077760)
chromium-1280x720-candidate-screenshot-for-dimension-mismatch-on-color-depth-png.png 100.00% ⚠️ Dimension mismatch (1280x4703, Data length: 24079360 → 1280x4898, Data length: 25077760)
chromium-1280x720-candidate-screenshot-for-gradient-16bit-png.png 100.00% ⚠️ Dimension mismatch (1280x4703, Data length: 24079360 → 1280x4898, Data length: 25077760)
chromium-1280x720-candidate-screenshot-for-gradient-mixed-depth-png.png 100.00% ⚠️ Dimension mismatch (1280x4703, Data length: 24079360 → 1280x4898, Data length: 25077760)
chromium-1280x720-candidate-screenshot-for-greyscale-with-alpha-png.png 100.00% ⚠️ Dimension mismatch (1280x4703, Data length: 24079360 → 1280x4898, Data length: 25077760)
chromium-1280x720-candidate-screenshot-for-landscape-png.png 100.00% ⚠️ Dimension mismatch (1280x4703, Data length: 24079360 → 1280x4898, Data length: 25077760)
chromium-1280x720-candidate-screenshot-for-night-sky2-png.png 100.00% ⚠️ Dimension mismatch (1280x4703, Data length: 24079360 → 1280x4898, Data length: 25077760)
chromium-1280x720-candidate-screenshot-for-opaque-landscape-png.png 100.00% ⚠️ Dimension mismatch (1280x4703, Data length: 24079360 → 1280x4898, Data length: 25077760)
chromium-1280x720-candidate-screenshot-for-resized-banner-png.png 100.00% ⚠️ Dimension mismatch (1280x4703, Data length: 24079360 → 1280x4898, Data length: 25077760)
chromium-1280x720-candidate-screenshot-for-webapp2-png.png 100.00% ⚠️ Dimension mismatch (1280x4703, Data length: 24079360 → 1280x4898, Data length: 25077760)
chromium-1280x720-new-candidate-screenshot-for-happy-chihuahua-png.png 100.00% ⚠️ Dimension mismatch (1280x4703, Data length: 24079360 → 1280x4898, Data length: 25077760)
chromium-1280x720-report.png 100.00% ⚠️ Dimension mismatch (1280x4703, Data length: 24079360 → 1280x4898, Data length: 25077760)
chromium-1280x720-visual-difference-highlighting-changed-pixels-in-checkerboard-png.png 100.00% ⚠️ Dimension mismatch (1280x4703, Data length: 24079360 → 1280x4898, Data length: 25077760)
… and 245 more

Added Files (18)

  • chromium-1280x720-baseline-screenshot-for-corrupted-candidate-png.png
  • chromium-1280x720-candidate-screenshot-for-corrupted-candidate-png.png
  • chromium-375x667-baseline-screenshot-for-corrupted-candidate-png.png
  • chromium-375x667-candidate-screenshot-for-corrupted-candidate-png.png
  • chromium-768x1024-baseline-screenshot-for-corrupted-candidate-png.png
  • chromium-768x1024-candidate-screenshot-for-corrupted-candidate-png.png
  • firefox-1280x720-baseline-screenshot-for-corrupted-candidate-png.png
  • firefox-1280x720-candidate-screenshot-for-corrupted-candidate-png.png
  • firefox-375x667-baseline-screenshot-for-corrupted-candidate-png.png
  • firefox-375x667-candidate-screenshot-for-corrupted-candidate-png.png
  • firefox-768x1024-baseline-screenshot-for-corrupted-candidate-png.png
  • firefox-768x1024-candidate-screenshot-for-corrupted-candidate-png.png
  • webkit-1280x720-baseline-screenshot-for-corrupted-candidate-png.png
  • webkit-1280x720-candidate-screenshot-for-corrupted-candidate-png.png
  • webkit-375x667-baseline-screenshot-for-corrupted-candidate-png.png
  • webkit-375x667-candidate-screenshot-for-corrupted-candidate-png.png
  • webkit-768x1024-baseline-screenshot-for-corrupted-candidate-png.png
  • webkit-768x1024-candidate-screenshot-for-corrupted-candidate-png.png

Generated by visual-differ at 2026-11-09T22:19:39.695Z

View the report.

These type of errors shutdown the differ and leave us with no useful
report. Catch them instead and output the error info.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Copilot was unable to complete this review at the selected tier due to model policy or plan restrictions.

🟢 Approval recommended

The change consistently converts PNG decode failures into structured comparison outcomes, updates both HTML/Markdown reporting to surface those details safely, and adds expanded unit plus screenshot coverage with fixtures for corrupted/truncated inputs.

No open issues 🎉

What changed in this PR

This pull request makes the visual differ resilient to corrupted/truncated/non-PNG “.png” inputs by catching PNG decode failures and surfacing them as structured, reportable results instead of crashing the run.

Changes:

  • Introduce a dedicated PNG decoding error type and propagate decode failures as structured comparison outcomes.
  • Update HTML + Markdown report generation to display PNG decoding error details and avoid rendering diff images when decoding fails.
  • Expand unit and screenshot tests and add fixtures/samples to cover corrupted/truncated PNG scenarios.

Approval Readiness Assessment

This pull request requires changes.

File Summary
test/​visual-differ.test.ts Updates expectations so invalid PNGs don’t crash the run and are reported; some assertions currently depend on exact decoder error strings.
test/​report-generator.test.ts Adds/updates coverage for HTML report rendering for PNG decoding error rows.
test/​png-file-pair-factory.test.ts Adds decode-failure coverage; some assertions currently depend on exact decoder error strings.
test/​markdown-report-generator.test.ts Adds/updates Markdown output coverage for PNG decoding error notes; includes exact-string assertions.
test/​helpers/​test-utils.ts Adds helpers/fixtures to support corrupted/truncated PNG cases and expected result shapes.
templates/​report.html Updates report UI to display PNG decoding error details.
screenshot-tests/​report.test.ts Updates Playwright coverage for the report UI; includes assertions tied to exact decoder error strings.
samples/​candidate/​corrupted-candidate.png Adds a corrupted “.png” sample to exercise decoding-failure behavior end-to-end.
package.json Formatting-only change.
lib/​visual-differ.ts Converts PNG decode failures into structured comparison results so the run can complete and be reported.
lib/​report-generator.ts Plumbs decoding-error metadata into HTML report data and suppresses diff-image rendering when decoding fails.
lib/​png-file-pair-factory.ts Wraps PNG decoding failures into a dedicated error type for higher-level handling/reporting.
lib/​markdown-report-generator.ts Adds PNG decoding error notes to Markdown output and refactors note generation.
lib/​image-comparer.ts Extends comparison result data to carry optional PNG decoding error details.
lib/​errors.ts Adds PNG decoding error type(s); naming between pngDecodingFailure vs pngDecodingError is currently inconsistent.

Review effort: Balanced

@jrjohnson
jrjohnson marked this pull request as ready for review November 9, 2026 23:14
@jrjohnson
jrjohnson merged commit 59d4845 into main Nov 9, 2026
18 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants