Repository navigation
Skip fuzzy search for exact normalized matches - #1391
Open
harshitgupta31415 wants to merge 1 commit into
Open
harshitgupta31415 wants to merge 1 commit into
harshitgupta31415 wants to merge 1 commit into
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1390
When a message only changes capitalization or surrounding whitespace, its fuzzy key can already be in the candidate dictionary. This checks that dictionary first instead of comparing the text with every candidate again.
The existing search gives identical normalized strings a score of 1, so the shortcut picks the same key. The merge code is unchanged: translations, fuzzy flags, comments, previous IDs and obsolete messages still go through the same path. Actual wording changes still use get_close_matches(). No extra cache is added.
For Q exact-normalized lookups among C candidates, this avoids Q × C candidate comparisons and uses Q dictionary lookups instead, after the existing candidate preparation. String normalization still takes time, and nonexact fuzzy matching has the same complexity as before. This does not make every catalog update linear.
This leaves the current normalized-key collision behavior alone; #1330 handles that separately.
Checks:
Benchmark scripts, raw samples and reproduction steps.
All 40 benchmark rows, including controls and slower results
Measurements use Python 3.14.5 on Windows. Each row uses one fixed pair of physical cores; pairs rotate between rows. These Python functions are single-threaded, not parallel workers. Results from different pairs are shown separately because this laptop has different core types.
Each row has five paired timing samples per version (short calls batched), plus five separate tracemalloc measurements. Times and peaks are medians. GC stays enabled. Peaks cover newly traced Python allocations during the call, not total process RAM or a copy of the installed library.
The full workflow parses existing PO bytes, updates from a prepared template, and writes the entire PO. Reading files and preparing the template are outside the measurement. The catalogs are real Django/Wagtail translations; the edits are controlled capitalization, surrounding-whitespace, or punctuation changes, not production traffic.
All output bytes matched. The unchanged and punctuation-change controls do not use the shortcut, so their timing differences are not claimed as improvements. Every measured row is included, including slower results.
Base commit: e2a1791. Source, runner and fixture hashes and every raw sample are in the linked JSON.
Fixtures (downloaded from these pinned sources; not copied into the benchmark gist):