[kylebernhardy]

⊙ Problem

Document native full-text search for Harper 5.3 while distinguishing the final v5.4 paging behavior. The older text incorrectly presented the 100-record prefix window as a current total-result cutoff and included unsupported filterFields schema examples.

Companion to Complete native full-text indexing and search, with executable examples in Test documented full-text examples. Document full-text paging and configurable search limits owns the detailed v5.4 request limits and restart guidance.

💡 Solution

Keep the native declaration/query guide here and the work-budget reference in the paging companion. Before v5.4, prefix and fuzzy-prefix queries restrict offset plus limit to 100. In v5.4, bounded queries fetch more ranked batches using one reader per request, count eligible records after source/version checks and filters, and report budget exhaustion as an error. Separate offset requests have no retained search or snapshot.

⚖️ Alternatives

This reconciliation updates the existing guides rather than adding another paging page or copying numeric defaults. The configuration references remain the owners of defaults and restart requirements. It changes documentation only; no runtime API or native implementation changes. Unsupported filterFields examples were removed after checking both released core tags and the final paging parser, rather than inventing a version badge for an unimplemented option.

❓ Your call: Framing-Verdict: better-alternative-exists (b1934a2d4c64; c9884425285f) — resolved by adopting the existing one-way dependency marker, together with source-backed schema/default checks on the combined tree. A new global anchor policy and new configuration-generation infrastructure were not added. The current branch rules require TypeScript, lint and formatting, but do not require companion-check; its pending status records the dependency rather than guaranteeing an administrator cannot merge out of order. No clean clearing planning verdict is claimed.

🔧 Changes

Paging compatibility

Version	Prefix paging	Between pages
Before v5.4	Offset plus limit cannot exceed 100	Each page is a separate query
v5.4	Bounded pages can pass native windows within request budgets	Concurrent changes can cause repeated or omitted records

The overview, querying reference, REST reference, query-optimization introduction and product-catalog tutorial now agree on that boundary. The query-planning introduction moved away from the companion's v5.4 work-budget section so both branches merge cleanly. Deep-offset cost guidance is explicit, and the generic condition table links to the existing freshness-control owner. The branch was refreshed from main, preserving its backup preflight and interrupted-restore guidance alongside the full-text rebuild note. Duplicate 5.3 release-note sections were folded into one, and the new paging section distinguishes 400 budget limits, 503 execution limits and terminal streaming errors.

• Learn and navigation: product catalog guide, reference landing page, Learn sidebar, and Reference sidebar make the feature discoverable and provide a complete working example.
• Full-text reference: overview, configuration, and querying define the storage boundary, schema options, match modes, BM25 ordering, highlights, field permissions, freshness controls, error contracts, and limitations.
• Existing API references: database overview, schema directives, JavaScript environment, GraphQL behavior, Operations API, query optimization, Resource API, REST headers, and REST querying connect full-text behavior to Harper’s existing surfaces instead of creating parallel documentation.
• Operations and recovery: backup behavior and transaction-log cleanup explain that native files are local derived state and that rebuilds depend on retained authoritative history.
• Release and authoring support: 5.3 release notes, authoring conventions keep version and storage-engine annotations consistent.

✅ Verification

The final guide head and paging companion merge cleanly and pass a combined 414-document production build, typecheck and formatting. Fresh unchanged-core verification passes 119 focused schema/runtime/query tests (two LMDB-only skips in RocksDB mode) and all 21 deployed full-text/example integration tests.

❓ Your call: Final-artifact delta after the completed full review of 06b8ff0: seven already-reviewed files, 22 insertions and 20 deletions, not re-reviewed. These are narrow corrections requested by that review plus restoration of unrelated formatting to main; there is no runtime, API or schema change. Individual and exact combined-tree builds, typecheck and formatting pass. The canonical result remains non-converged: audit defaults, budget references and v5.4 release notes are supplied by the explicit paging dependency. Count and header guidance were narrowed, and the streaming, range-version and dictionary-link findings were corrected. Earlier framing alternatives remain resolved by the recorded adoption; no clean clearing planning verdict is claimed.

• Production build: 414 documents; typecheck and repository-wide formatting pass.
• Paging companion build: 410 documents; repository-wide formatting passes.
• The exact combined branches merge without conflicts and pass production build (414 documents), typecheck and formatting.
• Historical prefix-window checks were traced to the core base; current paging semantics are covered by the core companion's passing CI, including all Node 22/24/26 resource and LMDB stages and native full-text tests on Linux, macOS and Windows.
• All seven existing documented-example integration tests were rerun and pass on the final paging core commit, covering declarations and Table.search, REST and Operations API query shapes. This is not a new whole-feature qualification run.
• A static compatibility check found two unsupported schema declarations before the fix and zero afterward against the exact released and final-core option contracts.
• A narrow combined-tree check evaluated the actual core resolver and verified all nine documented defaults, the paging owner anchor, 400/503/413 guidance, the 5.4 paging release note, and one 5.3 full-text heading.
• Kyle authorized outside review of both documentation diffs. Claude and Cursor reviewed the reconciliation; their unsupported-schema, duplicate-heading, error-status and fuzzy-prefix wording findings were corrected. Final outside review and domain adjudication completed (14 rounds / eight full cumulatively), but the canonical result remains non-converged. The retained major is the isolated-branch configuration gap supplied by the paging companion; the marker below and exact combined-tree verification record that dependency. The final narrow corrections make budget errors explicit, add a REST range example, fix date-rule placement, cross-link the error summary, and restore badge styling exactly to main. The latest closeout also corrects the REST version attribution, links dictionary fields directly, explains terminal streaming failures, separates table counts from full-text match totals, scopes coverage guidance to full-text, and removes unrelated list/badge restyling. Those corrections address reviewed findings inside reviewed files and were verified locally, without another full round.
• All 12 human inline threads now have factual replies and are resolved: the freshness loop no longer buffers records; describe_table has a generic introduction and separate full-text subsection; obsolete prerelease guidance is removed; highlighting, Tantivy, audit defaults and POST versus GET are explained; unsupported filter metadata and duplicate release notes are removed. The prior changes-requested review remains until a human reviewer updates it. No new approval, reviewer request or merge is claimed.

Comment generated by kAIle (GPT-6)

Release target: main — native full-text guide with version-qualified v5.4 paging.

Related PRs: #739 overlaps (Document full-text paging and configurable search limits, coordinated paging references), 11 others independent

Publication dependency: Document full-text paging and configurable search limits supplies configuration and 5.4 release-note owners before this native guide. The combined tree is the verified publication result. The one-way companion status waits for that PR; neither PR is a standalone replacement for the other. No merge or auto-merge is authorized.

Depends-on: #739

Complexity: medium

_{Review-Coverage: authored=codex; ran=claude,cursor-composer,cursor-grok; adjudicated=domain; blocked=gemini(auth); declined=cursor-kimi,cursor-muse; rounds=14; full=8 @ 51713cc}

_{Review-Attention: study ~9m (decisions: do-less-alternative, authorization-opt-in, paging-model, budget-failure-contract, audit-activation-rule; raised: open major) @ 51713cc}

[gemini-code-assist]

Code Review

This pull request introduces comprehensive documentation for the new native BM25-ranked Full-Text Search feature in Harper v5.3.0, including a new developer guide, configuration references, querying guides, and updates to existing database, REST, GraphQL, and Operations API documentation. The review feedback highlights several style and formatting improvements: separating a critical security warning into distinct sentences for better visibility, replacing the <VersionBadge> component with plain text when used mid-sentence in prose, and replacing em dashes with colons as field separators in Markdown lists.

[cb1kenobi]

Native full-text documentation is consistent on authoritative versus derived state, REST and JavaScript permission boundaries, rebuild and lag error codes, and prefix-window limits. Existing inline comments already cover VersionBadge placement, em dashes, and the learn-guide security-warning wording. Nothing remaining on the changed lines is a confirmed blocking defect.

—
Reviewed 45c1c5d

[cb1kenobi]

The field-only FullText schema, REST versus trusted in-process permission boundary, rebuild and lag error codes, and backup or log-retention recovery guidance are consistent across the new pages and updated API references. Existing discussion already covers VersionBadge placement, em dashes, and the older learn-guide warning wording. Nothing remaining on the changed lines is a confirmed blocking defect.

—
Reviewed 6855bed

[kriszyp]

The Resource code example basically ignores all of our best practices. It should:

• Not use loadAsInstace
• Use static methods
• Don't accumulate search results in an array (memory hazard, slower latency).

And I don't know why you would need to set audit: true, that's the default.

_{🤖 Reviewed with Claude}

[cb1kenobi]

No additional blocking findings were confirmed. Existing discussion already covers the product-guide issues found at this commit.

—
Reviewed 6855bed

[kylebernhardy]

Thanks, Kris. All four points are addressed:

• Highlight behavior now matches core: every configured source can match and rank, while only fields with highlight: true appear in $highlights. The redundant highlight switch was removed from the example.
• The custom resource now uses the static Resource pattern, checks the authenticated caller in post, passes context, and opts into table and source-field authorization with checkPermission: true. The Resource API now documents this opt-in and makes clear that passing context alone is not enough.
• The freshness example consumes the async iterator, which is when coverage is checked.
• The engine badge now sits with the version badge at the top of the page.

The accuracy pass also tightened the audit migration path, all five readiness states, write backpressure, rebuild and replica behavior, Blob limits, and the handling of derived Tantivy files.

Validation is green: formatting, typecheck, workflow tests, production docs build (411 documents), 11 focused Harper integration tests, and 96 focused Harper unit tests.

Comment generated by kAIle (GPT-6 Astra)

[github-actions]

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-691

This preview will update automatically when you push new commits.

[kylebernhardy]

Executable coverage for the published schema, JavaScript, REST, and Operations API examples is now in Harper PR #2915, "Test documented full-text examples". The test fixture also pins permissions, index coverage, reduced-index metadata, highlight limits, synonyms, multiple indexes, and Blob sources.\n\nComment generated by kAIle (GPT-6 Astra)

[cb1kenobi]

The filter-metadata docs stay consistent with the query-only FullText field, permission boundary, and rebuild or restore contracts. Open threads already cover the merged duplicate 5.3 section, audit-default wording, freshness example buffering, describe_table layout, filter staleness, and the oversized filter-string gap. No new blocking defect was confirmed on the changed lines.

—
Reviewed a035305

[Ethan-Arrowood]

Quick nudge; this PR has a lot of unresolved review comments already. @kylebernhardy can you take another pass at it and request fresh reviews from Kris and Chris when its ready for review again? Thank you!

[github-actions]

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-691

This preview will update automatically when you push new commits.

[kylebernhardy]

Thanks for the nudge, Ethan. The remaining inline concerns have now been checked against the current core implementation, addressed, and resolved with individual replies. The guide fixes cover audit-default wording, freshness iteration without buffering, generic describe_table layout, unsupported filter metadata, duplicate release notes, POST versus GET, highlighting, Tantivy and obsolete prerelease guidance. The final cleanup also clarifies streaming errors and count behavior and removes unrelated formatting churn.

The individual and combined documentation builds pass (414 documents), as do typecheck and formatting. Fresh core verification passes 119 focused tests (two LMDB-only skips in RocksDB mode) and 21 deployed full-text/example tests without skips. Claude and Cursor completed another outside review; its paired-reference findings remain covered by Document full-text paging and configurable search limits, which this guide explicitly depends on. The older changes-requested review still needs a fresh human assessment of the updated head.

Comment generated by kAIle (GPT-6)

[github-actions]

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-691

This preview will update automatically when you push new commits.