[khvn26]

Part of a stack of small local evaluation performance PRs, addressing flagsmith-python-client#198. Each PR is based on the previous one; perf/all holds the whole stack for benchmarking.

The existing benchmarks evaluate a handful of features, too few to catch regressions that only show at scale, as in #198. This adds two CodSpeed benchmarks:

• test_engine_benchmark__large_context: an identity context shaped like a large real-world environment, as built by the Python SDK in local evaluation mode. It has 262 features (a tenth of them multivariate) and 20 segments with nested trait rules overriding 5 features each, plus a $.environment.name segment and an identity overrides segment.
• test_engine_benchmark__single_feature_context: the same context trimmed to one feature and the segments overriding it, as the Python SDK evaluates flags lazily.

No engine changes.

Stack

1. test: Benchmark large and single-feature contexts #353 test: Benchmark large and single-feature contexts
2. perf: Resolve the identity key once per evaluation #354 perf: Resolve the identity key once per evaluation
3. perf: Hash object IDs with less overhead #355 perf: Hash object IDs with less overhead
4. perf: Look up casting functions by exact type #356 perf: Look up casting functions by exact type
5. perf: Skip number matching for strings not starting like one #357 perf: Skip number matching for strings not starting like one
6. perf: Get context values by name without running JSONPath queries #358 perf: Get context values by name without running JSONPath queries
7. perf: Match IN conditions against lists without building sets #359 perf: Match IN conditions against lists without building sets
8. perf: Catch typed condition errors without a context manager #360 perf: Catch typed condition errors without a context manager

perf/all = the whole stack (compare).

Stack results

p50 µs per get_evaluation_result, median of 3 interleaved rounds, Python 3.11. Contexts are built by flagsmith-python-client: f262_s20 = 262 features / 20 segments etc., with #262 applied; released_sdk_io1000 is as released SDKs build it, with 1,000 identity overrides segments. Run-to-run noise is up to ~8%, so read single-digit per-PR deltas as indicative.

scenario	main	01-identity-key-once	02-hashing	03-casting	04-string-coercion	05-jsonpath-names	06-in-list	07-typed-conditions-try	total
f262_s20/environment	86.2	76.8 (-11%)	75.6 (-1%)	80.4 (+6%)	86.6 (+8%)	75.5 (-13%)	75.4 (-0%)	76.9 (+2%)	-11%
f262_s20/identity	237.2	218.4 (-8%)	222.5 (+2%)	213.5 (-4%)	214.8 (+1%)	202.4 (-6%)	203.4 (+0%)	196.3 (-3%)	-17%
f262_s20/identity_one_feature	12.0	12.8 (+6%)	12.9 (+1%)	11.6 (-10%)	11.2 (-3%)	11.0 (-2%)	11.4 (+3%)	10.7 (-6%)	-11%
f262_s200/environment	86.6	78.7 (-9%)	76.0 (-3%)	81.3 (+7%)	87.9 (+8%)	75.5 (-14%)	81.3 (+8%)	77.2 (-5%)	-11%
f262_s200/identity	995.9	982.4 (-1%)	982.4 (+0%)	931.6 (-5%)	894.3 (-4%)	887.1 (-1%)	893.9 (+1%)	792.0 (-11%)	-20%
f262_s200/identity_one_feature	24.5	24.6 (+0%)	24.8 (+1%)	23.6 (-5%)	22.4 (-5%)	22.3 (-0%)	22.5 (+1%)	21.1 (-7%)	-14%
f1000_s20/environment	339.6	310.8 (-8%)	299.4 (-4%)	319.5 (+7%)	317.4 (-1%)	310.5 (-2%)	297.2 (-4%)	303.9 (+2%)	-11%
f1000_s20/identity	654.5	593.9 (-9%)	566.3 (-5%)	578.8 (+2%)	575.4 (-1%)	555.7 (-3%)	553.8 (-0%)	549.7 (-1%)	-16%
f1000_s20/identity_one_feature	7.9	7.9 (-0%)	7.9 (+0%)	7.7 (-4%)	7.3 (-4%)	7.3 (-1%)	7.2 (-1%)	6.9 (-4%)	-13%
released_sdk_io1000/identity	5,051.2	5,139.8 (+2%)	5,091.7 (-1%)	4,967.8 (-2%)	4,750.5 (-4%)	2,366.0 (-50%)	2,230.5 (-6%)	2,233.5 (+0%)	-56%
released_sdk_io1000/identity_one_feature	104.4	105.7 (+1%)	104.2 (-1%)	106.1 (+2%)	98.9 (-7%)	48.3 (-51%)	45.4 (-6%)	45.3 (-0%)	-57%

[github-actions]

File	Coverage		Missing
All files	100%	✅

Minimum allowed coverage is 100%

Generated by 🐒 cobertura-action against 98d0377

[codspeed]

Merging this PR will not alter performance

✅ 1 untouched benchmark
🆕 2 new benchmarks

Performance Changes

	Benchmark	BASE	HEAD	Efficiency
🆕	test_engine_benchmark__large_context	N/A	2.8 ms	N/A
🆕	test_engine_benchmark__single_feature_context	N/A	217.3 µs	N/A

---

_{Comparing perf/00-large-context-benchmark (98d0377) with main (c0204a0)}

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}