Repository navigation
Docs: what to do after a graph batch was interrupted before close() - #79
Merged
Merged
Conversation
Adds "If a graph batch is interrupted" to the bulk-loads page: what a kill -9 leaves committed, why the committed edges are not traversable until close() links them, and the recovery that works on the native build (CHECK DATABASE FIX DELETE ORPHANS, then load the edges again over the existing vertices). CHECK DATABASE FIX relinks nothing, and re-running without the delete leaves duplicate edge records. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…h point as what close() does Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds an "If a graph batch is interrupted" section to
docs/modules/ROOT/pages/how-to/bulk-loads.adoc(the page from #75). Python and embedded only;CHECK DATABASEis linked to the main ArcadeDB docs, not repeated.What it says, and where the numbers come from
A
graph_batch()load that is killed beforeclose()returns (kill -9, defaultcommit_every=1000, with and withoutuse_wal=True, 40 kills in the probe) loses no committed chunk and leaves no edge that points at a missing vertex. But the engine links the edge records to their vertices (head pointers and incoming lists) inclose(), so until then the committed edges exist (count(*)on the edge type counts them) andoutE(),inE(),out()andMATCHreach none of them.CHECK DATABASEreports them asORPHAN RECORD.Measured on one kill (30,000 vertices, 70,000 edge records) on the native build
26.11.1.dev0, each step on the reopened database:CHECK DATABASEORPHAN RECORDCHECK DATABASECHECK DATABASE FIXCHECK DATABASE FIX DELETE ORPHANSnew_edgesover the existing vertices +close()Re-running the same load with the same keys:
create_verticesfails withDuplicatedKeyExceptionon a unique key and changes nothing; re-running only the edges without the delete gives 160,000 edge records for 90,000 distinct edges (70,000 old orphans stay and keep being reported). The section therefore recommendsFIX DELETE ORPHANS, then an edge-only reload.One sentence says that splitting a long edge load over several closed batches leaves only the open batch unlinked after a kill. The probe did not measure that, and the page says so.
Check
Docs only. The new section has no hard-wrapped prose and the link target was fetched before it went in.
🤖 Generated with Claude Code