Skip to content

ADFA-6740: docdb-studio and the db writers read and write schema 3.0.0 - #57

Merged
hal-eisen-adfa merged 1 commit into
mainfrom
fix/ADFA-6740
Oct 23, 2026
Merged

hal-eisen-adfa merged 1 commit into
mainfrom
fix/ADFA-6740

Conversation

@hal-eisen-adfa

@hal-eisen-adfa hal-eisen-adfa commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Jira: ADFA-6740

The master documentation.db is moving to schema 3.0.0 (ADFA-3305). This teaches the tools that edit it to read and write that shape instead of refusing it. v2 support is kept: the 2.x master and the 2.x-only tools (check-tools, sync_kdoc, TooltipManager) still exist.

docdb-studio

  • Opens major 2 and 3; refuses >3. Version is read as the app reads it (ORDER BY changeTime DESC, rowid DESC; was rowid only).
  • Reads: tooltip SQL names {tooltips} / {buttons}; _tooltip_sql fills in the tables (v2) or 2.x-shaped subqueries over Tooltips+TooltipTexts and TooltipButtons+TooltipButtonDescriptions+TooltipButtonUris (v3). Browse, search, view, edit, export, Validate URIs and Browse content all go through it.
  • Writes: a new tooltip inserts TooltipTexts first and takes its id. Text updates go to TooltipTexts. Button strings are deduped through the lookup tables. Deletes drop the text row and any button strings nothing uses, before the existing VACUUM.
  • Templates page (new toolbar button): list, filter, edit, save. In v3 it decodes against CompressionDictionary and encodes with brotli -D -w 24, rejects large-window output (brotli4j refuses it) and checks the round trip before writing. In v2 it reads and writes plain.
  • Removed add_tooltip_button / update_tooltip_button (no callers).

Writers: populate_db.py, sync_javadoc_json_to_db.py, load_android_json_db.py

  • From 3.0.0 templates are written compressed (-w 24, decoded back before storing) instead of refused. Below 3.0.0 they're written plain, as before.
  • A 3.x database keeps its version. populate_db no longer rewrites it to (2, 0, 0), and the Android loader no longer appends 3.1.0.
  • A major above 3 (an unknown format) is refused before any backup or copy.
  • verify_android_json_db.py decodes compressed templates.
  • DictionaryCompressor.compress now passes -w 24 explicitly. That's the CLI default for stdin, so the bytes are unchanged.

Tests

  • The major ≥ 3 refusal tests are replaced with round trips on v3 fixtures (ticket scope).
  • docdb-studio/tests/test_schema_v3.py builds its fixture by running the real migrate_tooltip_schema.py on a 2.x database. It covers reads, insert (id from TooltipTexts, app queries find it), rename, delete with no orphans, CSV import, template round trip, and version ordering.
  • Two populate_db version tests encoded "replace any higher version with 2.0.0". They now assert that 3.x is kept and that an unknown major is refused.
  • Local runs: CI's writer suite 668 passed; docdb-studio 204 passed.

Checked against the real 3.0.0 database (copy of the 2026-10-23 file)

  • docdb-studio: 46,136 tooltips browse and search; all 1,650 templates decode; insert/edit/delete round-trips and the file stays 3.0.0.
  • After editing one tooltip, adding one and editing one template through docdb-studio, verify_remint_dictionary.py original edited reports 22,725/22,725 items and 1,649/1,650 templates byte-identical. The one mismatch is the edited template.

Not done here

  • The docdb-studio UI was not exercised by hand. The new Templates page and the tooltip flows on a v3 file need a manual pass.
  • The app-side acceptance criterion (CoGo with #2163/#2164 renders the edited tooltip and template) was not run.
  • No full populate_db / javadoc / Android run against the real 3.0.0 master. The writers are covered by fixture tests only.

…and the db writers

docdb-studio
- Opens majors 2 and 3 (refuses >3); version read newest changeTime, then rowid, as the app does.
- Tooltip reads name {tooltips}/{buttons}; on schema 3 they become 2.x-shaped subqueries over
  Tooltips+TooltipTexts and TooltipButtons+Descriptions+Uris.
- Writes: insert TooltipTexts first and key Tooltips on its id; text updates go to TooltipTexts;
  button strings deduped through the lookup tables; deletes remove the text row and any button
  strings nothing uses, before the existing VACUUM.
- New Templates page: list/filter/edit/save. Schema 3 decodes with the dictionary and encodes
  with brotli -D -w 24, rejecting large-window output and checking the round trip.
- Removed add_tooltip_button / update_tooltip_button (no callers).

populate_db.py, sync_javadoc_json_to_db.py, load_android_json_db.py (+ verify_android_json_db.py)
- From 3.0.0 templates are written compressed against CompressionDictionary (-w 24, decoded back
  before storing) instead of refused; below it, plain as before.
- A 3.x database keeps its version: populate_db no longer rewrites it to 2.0.0 and the Android
  loader no longer appends 3.1.0.
- A major above 3 (unknown format) is refused before backup or copy.
- The verifier decodes compressed templates.

v2 support is kept: the master and the other 2.x-only tools still exist.
Tests: refusal tests replaced with round trips on v3 fixtures; docdb-studio's v3 fixture is a 2.x
database converted by the real migrate_tooltip_schema.py.
@hal-eisen-adfa
hal-eisen-adfa merged commit 81186ab into main Oct 23, 2026
1 check passed
@hal-eisen-adfa
hal-eisen-adfa deleted the fix/ADFA-6740 branch October 23, 2026 20:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant