Skip to content

Better error handling - #150

Merged
jrjohnson merged 13 commits into
jrjohnson:mainfrom
stopfstedt:better-error-handling
Nov 6, 2026
Merged

jrjohnson merged 13 commits into
jrjohnson:mainfrom
stopfstedt:better-error-handling

Conversation

@stopfstedt

@stopfstedt stopfstedt commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator

use Errors to capture and pass failure during file pair creation and comparison instead of treating them as state in the data model.

a quick word on all these new error classes. these are mostly informed by the need to pass different contexts for each error condition to the output layer.
ideally, we would just use one class and encode the error details in its message.
but that would require more changes in the report output generators, which are outside of the scope of what i'm attempting to achieve here.

@stopfstedt
stopfstedt marked this pull request as ready for review November 5, 2026 21:01
@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown

✅ Visual Diff Report — PASSED

270 images compared: 270 identical

View the report.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Copilot was unable to complete this review at the selected tier due to model policy or plan restrictions.

🟡 Changes recommended

Approval is blocked by critical TypeScript API/declaration issues (exported error classes reference non-exported types) and unhandled IO/decode failures in PngFilePairFactory.create(), plus several tests that can pass without asserting throws, which together undermine correct builds and reliable error reporting.

13 open issues
What changed in this PR

This pull request refactors visual diff failure handling to use structured Error subclasses (with richer context) during PNG file-pair creation and image comparison, rather than modeling failure states directly in the data model. It introduces a PngFilePairFactory, threads typed errors through the compare/report pipeline, and updates tests/utilities to match the new flow.

Changes:

  • Adds a typed error hierarchy (FilePairComparisonError and subclasses) to carry baseline/candidate/diff path context and specific mismatch/format details.
  • Introduces PngFilePairFactory (plus getOutputFilePaths) to centralize PNG decoding/validation and output-path derivation.
  • Updates visual-differ and the test suite/helpers to use the new factory + thrown errors instead of prior state flags/return-shapes.

Key review feedback to address before approval:

  • Critical: lib/errors.ts exports classes with public properties typed by non-exported interfaces (DimensionMismatch, UnsupportedBitDepth, UnsupportedNumberOfChannels), which is likely to break .d.ts emit with declaration: true.
  • Critical: PngFilePairFactory.create() does not wrap IO/decode failures in the typed comparison error shape, undermining the stated goal of reportable failures.
  • Moderate: Several tests assert only inside catch blocks and may pass when nothing throws; one candidate/baseline unreadable-path test mutates the wrong side.
  • Nits: Minor comment typos and inconsistent ESM import extensions in a few tests.

This pull request requires changes.

File Notes
test/​report-generator.test.ts Updated report tests to build ComparisonResult fixtures via updated helper/signature.
test/​png-file-pair.test.ts Reworked to instantiate PngFilePair with decoded PNGs and explicit output paths.
test/​png-file-pair-factory.test.ts Adds factory-focused tests; review flagged catch-only assertions and a candidate/baseline mutation mistake.
test/​markdown-report-generator.test.ts Updated markdown report tests; note fixture naming nit (same.png used where differences exist).
test/​image-comparer.test.ts Updated to reflect compareImages throwing ImageComparisonError; review flagged catch-only assertions and naming nits.
test/​helpers/​test-utils.ts Adds/updates helpers for scanned-file fixtures and comparison creation aligned with new output-path flow.
package.json Formatting-only change.
lib/​visual-differ.ts Switched to using PngFilePairFactory and mapping typed errors into reportable ComparisonResult output.
lib/​utils.ts Adds getOutputFilePaths() helper for consistent baseline/candidate/diff output paths.
lib/​png-file-pair.ts Simplified into a container for decoded PNG data + paths (removes prior “state flag” error modeling).
lib/​png-file-pair-factory.ts New factory centralizing read/decode/validate/normalize; review flagged missing wrapping of IO/decode failures into typed errors.
lib/​image-comparer.ts Changes error behavior to throw ImageComparisonError with file-path context.
lib/​file-scanner.ts Adds ScannedFilePair interface used by the factory/tests.
lib/​errors.ts Introduces error hierarchy; review flagged non-exported public types breaking declaration emit and a minor comment typo.

Review effort: Balanced

Comment thread lib/errors.ts Resolved
Comment thread lib/errors.ts Resolved
Comment thread lib/errors.ts Resolved
Comment on lines +28 to +31
const { baselinePath, candidatePath, diffPath } = getOutputFilePaths(outputDir, name);
// Decode the files
const baselinePng = PngFilePairFactory.#decode(readFileSync(baselineSourcePath));
const candidatePng = PngFilePairFactory.#decode(readFileSync(candidateSourcePath));

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea yea.

@stopfstedt stopfstedt Nov 6, 2026 •

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, ideally we'd catch any and all errors that could be emitted. and we'll get there. what's missing right now is a way to display these in the report.

this is one hundred percent a follow up item.

Comment thread test/image-comparer.test.ts Resolved
Comment thread lib/png-file-pair-factory.ts Resolved
Comment thread test/image-comparer.test.ts Resolved
Comment thread test/markdown-report-generator.test.ts Resolved
Comment thread test/png-file-pair-factory.test.ts Resolved
Comment thread test/png-file-pair.test.ts Resolved
at the moment, this is just a thin wrapper around object instantiation.

in future steps, we'll move logic from the generated classes into the
factory, such as pixelmatch compat checks and color-channel adjustment.
this allows for better error handling. rather than lugging any file
error state around in the created png file pair object, we now throw
errors on creation.

the rest of changes are to support this new approach at file pair
creation.
this plugs a hole in this test where the expectations aren't run in
case no error is thrown.

we can't use the usual `expect(() => ...).toThrow(...)` syntax here,
since the error message does not provide details on the actual
dimensions mismatch, so we have to do it this awkward way.
yes, we want to muck up the candidate file in this test, not the
baseline.
@stopfstedt
stopfstedt force-pushed the better-error-handling branch from 571ab41 to 2ddea30 Compare November 6, 2026 22:42
@jrjohnson
jrjohnson merged commit bcbf832 into jrjohnson:main Nov 6, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants