Skip to content

Support jsonpath-ng 1.9 and fix skip paths with special characters - #27

Merged
bentsku merged 2 commits into
mainfrom
aws-819-unpin-jsonpath-ngand-upgrade-to-18-by-making-localstack
Oct 13, 2026
Merged

bentsku merged 2 commits into
mainfrom
aws-819-unpin-jsonpath-ngand-upgrade-to-18-by-making-localstack

Conversation

@bentsku

@bentsku bentsku commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

Part of AWS-819. LocalStack pinned jsonpath-ng==1.7.0 in localstack/localstack#13875. This PR makes localstack-snapshot work correctly with jsonpath-ng 1.9+.

Skip verification paths were applied by rebuilding the path of each match from str(datum.path), and only the last node had its quotes stripped. Since 1.8, jsonpath-ng quotes many more field names when serializing (spaces, digits-only, %, unicode, ...). Intermediate nodes with such keys no longer resolved, so the skip was silently not applied and tests failed with spurious snapshot diffs. Keys containing . or : (e.g. aws:SourceArn) were already broken before 1.8.

Changes

  • Delete matched values from the parent container referenced by the match context (m.context.value), using the structured Fields.fields / Index.indices attributes instead of string parsing.
  • Require jsonpath-ng>=1.9. 1.9 drops Python 3.10, so requires-python is now >=3.11. The 3.10 classifier, CI matrix entry and ruff target are updated to match.
  • Add regression tests for skip paths through keys with special characters, and for filter expressions. Each assertion uses a new session, as skipping mutates the state.
  • Rewrite the list skip-path tests to use a new session for every assertion. Previously, assertions running after a successful skip on the same session passed regardless of their skip paths. This hid that the dot-notation cases ($..aaa.1, $..aaa..0) never matched anything on any jsonpath-ng version: .1 isn't valid JSONPath (RFC 9535 requires [1] for indices), and jsonpath-ng parses it as a dict key. No skip path in localstack uses this notation, so those cases are removed rather than supported (see Support dot notation for list indices in skip verification paths #28).

The next release should be 0.4.0, as it drops Python 3.10.

🤖 Generated with Claude Code

Skip verification paths were applied by rebuilding the matched path from
`str(datum.path)`. jsonpath-ng 1.8 quotes many more field names when
serializing (spaces, digits-only, unicode, `%`, ...), so intermediate nodes
no longer resolved and the skip was silently not applied. Keys with `.` or
`:` were already broken before.

Instead, mutate the parent container referenced by the match context
directly, using the structured `Fields.fields` / `Index.indices` attributes.

Require jsonpath-ng>=1.9, which drops Python 3.10 support.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bentsku
bentsku marked this pull request as draft October 7, 2026 23:09
@bentsku
bentsku added this pull request to stack #29 October 7, 2026 23:09
@bentsku
bentsku removed this pull request from stack #29 October 7, 2026 23:15
Skipping mutates the recorded and observed state, so assertions running
after a successful skip on the same session passed regardless of their skip
paths. This hid that dot-notation list indices like `$..aaa.1` never
matched: they are not valid JSONPath and are parsed by jsonpath-ng as dict
keys. Remove those cases instead of supporting a non-standard notation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bentsku
bentsku marked this pull request as ready for review October 8, 2026 15:01
@bentsku bentsku mentioned this pull request Oct 8, 2026
@bentsku
bentsku added this pull request to stack #31 October 8, 2026 15:05

@dominikschubert dominikschubert left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Good catch on the (non-failing) tests 👍

Code was only skimmed, but no concerns from my side otherwise. An expansion of the tested python versions for more modern versions could be a follow-up at some point.

@bentsku

bentsku commented Oct 12, 2026

Copy link
Copy Markdown
Contributor Author

An expansion of the tested python versions for more modern versions could be a follow-up at some point.

Ah, good catch, missed we were not testing 3.13, but might be worth even testing ahead! Will open one today 👍

@bentsku
bentsku merged commit ea4df10 into main Oct 13, 2026
2 checks passed
@bentsku
bentsku deleted the aws-819-unpin-jsonpath-ngand-upgrade-to-18-by-making-localstack branch October 13, 2026 15:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants