Repository navigation
Skip impossible analysis affix matches using edge segments - #574
Open
johnml1135 wants to merge 3 commits into
Open
johnml1135 wants to merge 3 commits into
johnml1135 wants to merge 3 commits into
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #574 +/- ##
==========================================
+ Coverage 74.57% 74.61% +0.04%
==========================================
Files 453 453
Lines 38342 38376 +34
Branches 5284 5297 +13
==========================================
+ Hits 28593 28635 +42
+ Misses 8583 8573 -10
- Partials 1166 1168 +2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This comment has been minimized.
This comment has been minimized.
johnml1135
added a commit
that referenced
this pull request
Oct 28, 2026
Adds the round-doc section "Combined stack and R2's cost": the full stack against master (work, unification, matcher per grammar), additivity (near-additive on Mbugwe, overlapping on Sena, super-additive on Aweti), R2's root cause (its own probe, low prune rate) and the witness-reuse fix folded into #572 @ 01bdf5a, the 1-4 point residual, and the landing decision: #572 is a draft and lands only on top of #577 and #574. Stress grammars are PR #578 (opt-in, [Explicit], not run in CI). Ledger: row 40 (R2) Outcome records the fold and the landing decision; new row 47 records the combined stack measurement. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 30, 2026
Open
…y and in parallel Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…eads Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Contributor
Public API, target framework, package, machine.py parity: None verified. The new members are F1 addressed, F2 addressed, F3 accepted, F4 withdrawn, F5 addressed. Reviewed at 0af3bc0 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Start here:
src/SIL.Machine.Morphology.HermitCrab/MorphologicalRules/AnalysisAffixProcessRule.cs--EdgePassesand the check before the matcher call are the whole behaviour change.Status: ready for review, based on master (
c2454760), no conflicts. Landing waits for a remeasure once calibrated work-steps land on PanGloss main.Analysis affix patterns are anchored at both ends. When the first or last pattern node is a mandatory segment constraint and the word's edge segment does not unify with it, the allomorph cannot match, so it is now rejected before the matcher runs. An optional edge always passes, and the failed-unapplication trace event is still emitted.
Morpher.EdgePrefilterEnabled(internal, default on) switches it off;EdgePrefilterRejectedis an internal count hook.Measured on Sena (first 300 words of the reference list, 594 parses, one thread, same binary, flag on vs off, 4 alternating runs each): median user CPU 38.5 s -> 35.3 s (-8%), managed allocation 10.2 GB -> 8.8 GB (-14%), peak working set unchanged (153-169 MB vs 156-161 MB). The PC was loaded (load average 20-35 on 20 CPUs), so the CPU figure is noisy: per-pair change ranged from -1% to -20%. Hardware instruction counters were unavailable under WSL. 468,101 allomorph attempts skipped the matcher over those 300 words. Parse output was byte-identical in every run. Analysis steps are unchanged; the saving is cheaper attempts, not fewer. Matcher work (FST traversals plus arc checks, each prefilter unification charged one arc check; not a step count) fell by 19.55% on Sena in the earlier census.
Where to look
AdvanceinTraversalMethodBase). Pinned byOptionalEdgeNode_PrefilterMatchesFullMatcherand, through a real deletion rule,ParseWord_OptionalEdgeFromUnappliedDeletion_PrefilterMatchesFullMatcher.AlphaVariableEdgePassesWithUnboundVariable).Constraintas first or last child is checked; a leadingGroupgives null (GroupFirstWithOptionalLeadingConstraintDoesNotReject).ParseWord_StressFixture_PrefilterMatchesFullMatchercompares 60 words with the flag on and off, sequentially and in parallel.Not here:
AnalysisRealizationalAffixProcessRuleis unchanged and still runs the full matcher; the spec computes its two edge references but never reads them.Verification: HermitCrab test project 172 passed, 0 failed;
dotnet csharpier check .clean;comment-hygiene.ps1clean../local_check.shwas not run (SentencePiece needscmake).Next: review
AnalysisAffixProcessRule.cs, then approve or name an edge case to add.Reading this a year from now -- start here
This is the edge-segment prefilter port tracked in #564. The census data and write-ups (
result-5-edge-prefilter.md,edge-prefilter-evidence/census-net-summary.txt) are not in the tree; the figures in this description are the record. The shipped fixtureearly-lexicon-agglutinativeundertests/SIL.Machine.Morphology.HermitCrab.Tests/Research/StressFixtures/(generator, grammar, word list) is loaded by the full-parse test.Evidence -- measurements and census
Same-binary A/B, flag toggled by reflection,
MaxDegreeOfParallelism1, 10 warm-up words excluded, Sena grammar and word list from the reference inputs.Stress fixture (60 words, 27 analyses, 3 runs): allocation 264 MB -> 123.5 MB (-53%); user CPU 2.4-2.9 s vs 2.0-3.1 s, within noise.
Earlier census, unmerged, not re-run for this edit: net matcher work fell by Indonesian 6.51%, Aweti 47.47%, Amharic 12.76%, Mbugwe 19.62%, Sena 19.55%, stress fixture 77.65%. Census trace-event counts were unchanged. Invalid or budget-aborted words were excluded from parity.
The retained size of the per-rule metadata is estimated at 108.4 KiB for the largest reference grammar (an estimate, not measured).
Surprising findings
Conformance truncate-morphotactic fixtures give different parses on repeated runs of master itself, so they cannot serve as a parity check for this change.
Closes #564
This change is
🤖 Generated with Claude Code