Repository navigation
HermitCrab: dead ends and measured-and-rejected speedups (record, from PR #490) #601
Description
Claude-drafted.
Twenty-five HermitCrab speedup candidates were measured in the #490 round and rejected, parked or closed. None is ready to build. PR #490 is closed as a record, so the rejected candidates live here. Units are work steps; wall time or other units are labelled.
What we know (entries and sources below):
- Memo and cache variants: no gain or a loss. Corpus-scope memo: Sena 28 to 32% slower (wall time), 29.3 GB added peak memory.
- Ceilings too low: gates and prunes that missed their bar, such as T2 tandem intersection (23.5% dead steps against a 30% gate) and contexted constraints (6.0%, 0.0%, 0.4%).
- Unsound or net losses: order-insensitive synthesis dedupe loses parses; R4 subsequence fallback costs Indonesian +182.9% work and has 334,508 violations on Mbugwe.
- Built, measured, then closed: R2 root prune (PR Prune analysis rule outputs that contain no root when no later rule can restore stem material #572: Sena 15.3% fewer events but 7.7% more total work), E2 trail replay (PR Skip synthesis of candidates whose MPR features or stem names fail #570: Aweti +0.07%, loses on two fixtures), E3 template replay (dropped on cost).
- Required-environment pre-check (E5): dead end. Re-measured on PanGloss#90 (2026-10-30): it would save at most about 4% of a restricted enclitic's added cost.
Conflict: E2 is closed in C# but the port shipped its own version (sillsdev/PanGloss#21, PR #70, merged).
Not known: whether the port shows the same cost for the other entries.
What would reopen an entry: a real grammar whose measured ceiling clears the entry's bar (conditions in the accordion). Until then, check this list before proposing a speedup.
Related:
- PanGloss: Port check: rejected HermitCrab speedups, do not re-propose (machine#601) PanGloss#103 (pointer); Port machine #570: skip synthesis of lexical candidates whose MPR features or stem names fail PanGloss#21 (PR Remove unnecessary Mongo attributes #70 merged); Restricted affixes and clitics are tried on every word: environment and inflection-class restrictions don't reduce parse work PanGloss#90 (open).
- Skip synthesis of candidates whose MPR features or stem names fail #570, Prune analysis rule outputs that contain no root when no later rule can restore stem material #572 (closed PRs); HermitCrab analysis: subsequence fallback and insertion classes for the root prune (v2) #568 (open); HermitCrab optimization round record: what was tried, and a faster package that is not mergeable #490 (closed PR).
Next: anyone proposing a HermitCrab speedup checks this list first.
Paths not taken -- every entry, with the deciding number
Ledger rows are in docs/hermitcrab-optimization-ledger.md at 2eba2f5. "Wall" means wall time; no step count exists for it.
Memo and cache variants (the no-memoization rule; the one exception is the per-word root-search cache in #600)
| Entry | Result | Source |
|---|---|---|
Narrow AnalysisStateKey (row 4) |
Realised state collapse Sena 1.12x against a 1.3 gate (a state count, not wall time). Reopens on a grammar with genuinely high collapse | row 4 |
| Fold-step memo (row 5) | 0 hits; 0.96x wall (4.5% slower) | row 5 |
| Corpus-scope memo (row 18) | Indonesian 1.4% less wall, +68 MB; Sena first 300 words 28 to 32% slower wall, ~29.3 GB added peak memory | row 18 |
| Memo-key caching (row 21) | Key construction on Amharic: 3 ms of 81,707 ms wall, 0.0% | row 21 |
| Rule-level memo per battery call (row 29) | Sena repeats 33.7M work units (rule-level) against 32.9M (slot-level); no bigger than row 26 | row 29 |
Ceilings too low
| Entry | Result | Source |
|---|---|---|
| Analysis length ceiling, Gate B (row 9) | Byte-identical output; no measurable speedup (4 to 7% slower wall, within noise) | row 9 |
| Lexical reachability gate (row 10) | No-op on both corpora | row 10 |
| Tandem lexical intersection, T2 (row 11) | Pooled 23.5% dead steps against a 30% gate | row 11 |
| Pool small collections (row 12) | 15 to 17% fewer bytes but +8.6% wall; reverted | row 12 |
| Surface-length pruning, P2 (row 13) | Synthesis is 14 ms (wall), not the 160 ms assumed; closed before build | row 13 |
| Constraints in the analysis key (row 15) | A Sena analysis state costs ~0.73 ms against ~0.12 ms for a synthesis input (wall); closed before build | row 15 |
| Contexted constraints (row 19) | Shape-free work 6.0% and 0.0% (two Sena probes), 0.4% (28 Amharic words) | row 19 |
Row 5's family on GenerateWords (row 20) |
0% on 19 of 23 lookup-reaching fixtures; pooled hit rate 2.67%; best case 7.8% wall inside a 22.2% spread | row 20 |
| Prune dead branches (row 27) | Dead-internal bucket 0 to 5% on 6 of 8 lists; 12.2% on v1 and 39.8% on Aweti. Reopens if a shadow summary on those two decides a meaningful share (see #605) | row 27 |
| Idea B, commutation prune (comment on #490) | 0.00 to 0.05% of rule-unapplication attempts. The widened L0 control, which is unsound, reached 6.37% on Sena | PR #490 comment |
Unsound or net losses
| Entry | Result | Source |
|---|---|---|
| Order-insensitive synthesis dedupe (row 6) | Rules are not order-invariant: 2 violations on Sena, 12 on Indonesian. The sound version is 15 to 40% | row 6 |
| Synthesis length bound, Gate A (row 8) | Rejected valid parses (unit suite 64 to 34 passing); reverted | row 8 |
| R4 subsequence fallback (#568, row 45) | Mbugwe, Aweti, Amharic 0; Indonesian 182.9% more work; 334,508 unsound violations (Mbugwe) | row 45, R4 |
Built, measured, then closed
| Entry | Result | Source |
|---|---|---|
| R2 root-substring prune (PR #572, row 40) | Ledger: Sena 15.3% less (events net of probes). Closing comment: 7.7% more total work on Sena when every unit counts equally; wins only on a synthetic early-lexicon grammar. Issue #562 is still open | row 40, R2 |
| E2 MPR and stem-name trail replay (PR #570, row 33) | Ledger: Aweti synthesis steps -56.9%, total work -0.58%. Closing comment: Aweti +0.07% (unit not stated), loses on two fixtures, 785 lines. Port version merged as PanGloss#21 | row 33, E2 |
| E3 template replay (rows 34, 38) | Dropped on cost: replay states exceed steps saved on every corpus (Mbugwe 133,606 against 15,814); code kept on a branch | row 34, row 38, E3 |
Other
| Entry | Result | Source |
|---|---|---|
| Trail-position indexing (row 7) | 0.2% of wall (0.33 s of 143 s) | row 7 |
| Nogood subsumption, P4 (row 14) | 0 subsumable (0.0%) on two Sena probes and 33 conformance fixtures | row 14 |
| Idea A, deterministic traversal dedupe (comment on #490) | Deterministic traversal is 0.02 to 3.4% of CheckInputMatch calls; dedup removes at most 0.8% of that |
PR #490 comment |
| E5 required-environment prefilter (row 36) | Sound scan disables it on 5 of 5 real grammars and 145 of 150 conformance runs (0%). Forced on with the scan bypassed (unsound): Sena 5.7% less work net, Aweti 0.4% more, two of three stress fixtures net losses. Revival needs digraph-safe segmentation and a cheaper prefilter | row 36, E5 |
Surprising findings -- the environment pre-check, re-measured
PanGloss#90, Cebuano held-out set, enclitic g (a one-letter clitic, environment [V] _): 83% of its added ~1.08M steps fall on words ending vowel + g, where the environment holds and a host search runs. Consonant + g words are 4% and words not ending in g are 13%. The PanGloss#90 analysis concluded a pre-check could save at most about 4%, and recorded it as not pursued. This agrees with E5.
Sources -- pinned, from the closed PR #490
- Ledger: rows 4 to 15, 18 to 21, 27, 29, 33, 34, 36, 38, 40, 45.
- Rounds: synthesis-to-analysis, E2 to E5; analysis, R2 and R4.
- PR Skip synthesis of candidates whose MPR features or stem names fail #570 and Prune analysis rule outputs that contain no root when no later rule can restore stem material #572 closing comments; the two ideas are a comment on PR HermitCrab optimization round record: what was tried, and a faster package that is not mergeable #490.
- Port: Analysis prefilters: edge-segment prefilter (#24) and lookup replay (#21) PanGloss#70, Restricted affixes and clitics are tried on every word: environment and inflection-class restrictions don't reduce parse work PanGloss#90.
Activity