Skip to content

fix(search): rank JSON object keys below executable definitions - #4724

Closed
ScriptedAlchemy wants to merge 2 commits into
masterfrom
fix/json-key-search-ranking
Closed

ScriptedAlchemy wants to merge 2 commits into
masterfrom
fix/json-key-search-ranking

Conversation

@ScriptedAlchemy

Copy link
Copy Markdown
Owner

Root cause

Refs #4451 (JSON-key result flooding, a partial item).

The JSON extractor intentionally emits top-level configuration object members as Const nodes: the TypeScript/package configuration resolver depends on their sealed signatures. The search source-role classifier nevertheless promoted a same-name JSON key to ProductionDefinition, the same priority tier as executable functions or symbols. In a capped lexical/exact candidate page, numerous JSON keys can displace actual code definitions irrespective of BM25 score.

Focused change

  • Keep JSON object-key occurrences, signatures, normal postings, and scored evidence intact. Do not filter or change extraction; package.json, tsconfig.json, and JSON-scoped searches still need their keys.
  • Classify matches originating in .json as ProductionOther, even when the key itself matches. This is the existing source-role authority that both lexical and exact winner selection consume, so ranking and emitted evidence stay consistent.
  • Preserve test-file classification precedence and all existing literal admission, path scopes, privacy, and ranked-candidate limits.
  • No new configuration option, special-case scanner, or source-index format revision.

Regression coverage

  • Unit checks distinguish JSON keys, executable definitions and test-file references.
  • An added mixed real JSON/Rust/test fixture goes through parser extraction, sealed artifact building, bounded lexical winner selection and emitted role evidence. It requires the code declaration first with both an unbounded and capped (K=1) result; JSON still appears in the uncapped result and in an explicitly JSON-directory-scoped search.

Validation and limits

Source-level review completed for the current master and the complete two-file commit diff. Blob identity, no trailing whitespace, and final-newline checks completed. The Rust/Bazel tests, rustfmt and full suites have NOT executed: this execution environment has no Bazel or Rust toolchain or repository checkout. Hosted CI must validate the isolated PR branch before merge, especially the source-tier unit test and //crates/tracedecay-query:search_quality_suite.

This patch addresses JSON object-key priority, not the full multiword fetchWidgetAutoTagItemsForwardTotal acceptance or ranking measurements on the reported ~7k-file repository. #4710 separately repairs multiword subtoken candidate admission, #4702 handles CLI refusal wording, and the stale MCP description/privacy items have landed separately. Keep #4451 open until the full user journey is verified.

The existing PR-repair automation owns CI repair and authorized merges. This issue sweep does not modify or race with those PR branches.

JSON top-level object members are extracted as Const symbols to support
configuration resolution, but they are data keys rather than executable
definitions. Their name matches previously received the same privileged
source-role tier as code declarations, crowding capped symbol pages.

Keep JSON keys searchable, with their original lexical score, while
reserving ProductionDefinition for actual non-JSON code definitions.
Leave test-file precedence and exact/lexical authorization unchanged.

Add role classification checks and a mixed JSON/Rust/test corpus
regression through the real sealed lexical artifact. The bounded winner
heap must select the code definition, and a configuration-scoped query
must still retrieve the JSON key.

Refs #4451: the larger multiword ranking and corpus acceptance is
separate from this JSON-key source-role fix.
@changeset-bot

changeset-bot Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 610b82c

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Apply the exact rustfmt change from the repository gate failure while
retaining the test assertion and candidate-ranking behavior.
@ScriptedAlchemy

Copy link
Copy Markdown
Owner Author

Consolidated into #4742 (integrate/open-pull-requests) at the captured head shown in its description, as requested. Continue review and further fixes on #4742. This closure does not assert CI readiness; the replacement remains draft. Source branch retained.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant