Skip to content

Document full-text paging and configurable search limits - #739

Open
kylebernhardy wants to merge 48 commits into
mainfrom
kyle/docs-fulltext-continuation
Open

kylebernhardy wants to merge 48 commits into
mainfrom
kyle/docs-fulltext-continuation

Conversation

@kylebernhardy

@kylebernhardy kylebernhardy commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

⊙ Problem

The earlier draft described a separate full-text continuation API. Fill full-text and autocomplete pages using Harper's existing paging now fills ordinary limit/offset pages and removes that API.

❓ Your call: Keep paging guidance in the existing REST and query references, as requested, with no separate continuation API or new guide.

💡 Solution

Explain paging, limits and recovery in the existing references. There are no continuation functions, tokens, close operations or retained searches, and no separate paging page or sidebar entry. Each page runs a new query and can see later changes.

⚖️ Alternatives

❓ Your call: Framing-Verdict: option-set-too-narrow (0364589df96e) — resolved by adopting the concrete smaller structure: one restart-reference owner with links, existing storage/default owners, and source-grounded error/audit guidance. Earlier verdict 9d0c7d605cb1's existing-owner alternative was also adopted. A vendored core-default snapshot checker would create another artifact to synchronize, so it is deferred; product-wide error-code changes belong to core qualification rather than prose.

Document native full-text search owns the broader declaration/query guide. The reconciliation qualifies its historical 100-record prefix limit as pre-v5.4 and gives both guides the same independent-page and work-budget semantics. The two branches merge cleanly and their combined documentation build passes; numeric defaults and restart requirements remain in the existing configuration references. The broader guide also removes unsupported filterFields declarations and consolidates its duplicate 5.3 release-note sections. Publish this reference-owner change before the broader guide. Only the broader guide carries a Depends-on marker, pointing here; no reciprocal marker is added.

❓ Your call: Framing-Verdict: better-alternative-exists (b1934a2d4c64; c9884425285f) — resolved by adopting one existing Depends-on marker in the broader guide, pointing here. No reciprocal marker or repository-wide anchor policy is added. The combined tree is checked against exact core contracts; no clean clearing planning verdict is claimed.

🔧 Changes

  • reference/configuration/options.md — Owns all three request defaults and six native process budgets; corrects auditLog to the source default true.
  • reference/configuration/operations.md — Owns the request-worker versus native-process restart contract, states that full-text updates use individual parameter names rather than a whole storage_fullText object, and uses the actual configuration response text.
  • reference/configuration/overview.md — Links file-edit restart guidance to its reference owner.
  • reference/database/storage-tuning.md — Adds a versioned full-text tuning pointer before Related, without repeating numeric defaults.
  • reference/database/transaction.md — Includes full-text indexing among transaction-log consumers and warns that per-table audit opt-out prevents full-text index registration.
  • reference/operations-api/operations.md — Makes full-text comparators discoverable in search_by_conditions and links configuration restart guidance.
  • reference/resources/query-optimization.md — Explains independent offset requests, an offset-20/limit-10 example counting eligible records, filtering, raw/result/time budgets, native capture, freshness condition fields, common error statuses and recovery with describe_table readiness inspection, audit retention and concurrency costs; also covers response-envelope overflow from multi-hit ordinary batches.
  • reference/resources/resource-api.md — Lists full-text comparators and removes a promise of a missing Table.search example.
  • reference/rest/querying.md — Documents comparator syntax, filtered page filling, concurrent-write caveats and the difference between exhausted matches and budget errors, plus count behavior including the absence of an exact total for full-text and autocomplete queries.
  • release-notes/v5-lincoln/5.4.md — Describes full-text paging, the three request controls and native process-capacity budgets without introducing a continuation API.

✅ Verification

❓ Your call: Reconciliation verification: version-qualified prefix paging, shared error contracts, configuration ownership and coordinated publication. Both individual builds and the combined build pass. All nine defaults match the actual resolver from the final core commit, and all seven documented-example integration tests pass on that commit. Kyle authorized new outside review. The first review completed, and the narrow response-overflow and audit-reference findings were corrected; the planning alternative to use one existing dependency marker was adopted in the broader guide. Final outside review and domain adjudication completed (12 rounds / nine full cumulatively), without canonical convergence. The protocol-specific worker-code, readiness-inspection and runtime-release-note findings were then corrected within reviewed files and rebuilt. Declaration/examples/freshness-reference findings are supplied by the broader guide in the verified combined tree. Numeric defaults and native limits keep their existing owners; unrelated logging.console and archived-v4 findings remain outside this change.

Production build passes and processes 410 documents; repository-wide formatting passes after the paging and configuration clarification. The clarification and nine defaults were checked against the final paging core commit, including the accepted-record window, replay work and explicit budget-error contract. Default values and request/runtime names were checked against core's defaultConfig, validation, policy resolver and native binding. Error statuses were checked against query error mapping; the table is explicitly common failures, with 413 / other 500 behavior described separately. Local anchors and the public v0.6.0 native-limits link were checked; the build alone is not claimed to validate every anchor.

Earlier core verification covered 101 focused tests and all 14 deployed HTTP tests on both Node 24.16 and CI Bun 1.4.3. Its runtime fixture separately verifies unchanged captured budgets across replacement workers after configuration changes. The HTTP tests alone are not claimed to prove that negative restart boundary.

Earlier coverage comprised 10 rounds, eight full, with Claude, Cursor Composer/Muse and completed domain adjudication. Native Gemini CLI supplemented that earlier state and raised no documentation defects. The current cumulative canonical coverage is 12 rounds, nine full, as recorded above and in the generated footer. The last planning review did not clear automatically: its concrete placement, error, audit and reservation findings were adopted as described above; a generic build-source checker and product-wide error redesign were deferred rather than claimed complete. Native Gemini is supplemental coverage, not a canonical Gemini receipt.

Verify maxHeaderSize defaults across Harper HTTP configuration references remains an unrelated accuracy question. No version4 docs or ambiguous console default were changed. The core companion is ready for review and its current head passes all active CI checks. This documentation head also passes CI; all three bot review threads are resolved, and no new actionable feedback was found. The paging overlap with the broader full-text guide has been reconciled locally; that PR still has older unresolved human review threads requiring its own closeout. Kyle authorized moving this PR out of Draft after readiness checks clear. Neither PR may be merged by this task.

Comment generated by kAIle (GPT-6)

Release target: main — reference documentation for v5.4 full-text paging and limits.

Related PRs: #691 overlaps (Document native full-text search, complementary full-text guide), 4 others independent

Complexity: medium

Review-Coverage: authored=codex; ran=cursor-composer,cursor-muse,claude; adjudicated=domain; declined=gemini,cursor-grok,cursor-kimi; rounds=12; full=9 @ df5264e

Review-Attention: skim ~2m (decisions: coordinated-publication, defaults-ownership, native-limits-ownership) @ df5264e

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces comprehensive documentation for the new Full-Text Paging feature (continuation paging) in version 5.4.0, including a new dedicated guide, configuration options, REST header details, and sidebar integration. The review feedback suggests improving the formatting and placement of the <VersionBadge> component across several files to avoid awkward usage in running prose, recommending plain text or standalone placement under headings instead.

Comment thread reference/configuration/options.md Outdated
Comment thread reference/rest/headers.md Outdated
Comment thread reference/resources/full-text-paging.md Outdated
@github-actions

Copy link
Copy Markdown

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-739

This preview will update automatically when you push new commits.

@github-actions
github-actions Bot temporarily deployed to pr-739 October 20, 2026 16:10 Inactive
@github-actions

Copy link
Copy Markdown

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-739

This preview will update automatically when you push new commits.

@kylebernhardy kylebernhardy changed the title Document full-text and autocomplete snapshot paging Document full-text paging and configurable search limits Oct 21, 2026
@github-actions
github-actions Bot temporarily deployed to pr-739 October 21, 2026 06:14 Inactive
@github-actions

Copy link
Copy Markdown

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-739

This preview will update automatically when you push new commits.

@github-actions
github-actions Bot temporarily deployed to pr-739 October 21, 2026 13:27 Inactive
@github-actions

Copy link
Copy Markdown

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-739

This preview will update automatically when you push new commits.

@github-actions

Copy link
Copy Markdown

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-739

This preview will update automatically when you push new commits.

@github-actions
github-actions Bot temporarily deployed to pr-739 October 21, 2026 16:38 Inactive
@kylebernhardy
kylebernhardy marked this pull request as ready for review October 21, 2026 22:39
@kylebernhardy
kylebernhardy requested a review from a team as a code owner October 21, 2026 22:39
@github-actions

Copy link
Copy Markdown

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-739

This preview will update automatically when you push new commits.

This branch was successfully deployed

1 active deployment
pr-739 — df5264e6 Deployed Oct 21, 2026 by github-actions[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants