Repository navigation
blackbox: reclaim space by evicting the oldest log when the SD card is full - #12150
sensei-hacker wants to merge 5 commits into
Conversation
…s full Builds on the freefile-refill-at-boot fix (iNavFlight#12056): when the card is still completely full after that - no free clusters left anywhere - delete the oldest log and reclaim its space so logging can continue indefinitely on a card dedicated to blackbox, instead of stopping. Off by default (new blackbox_sdcard_wraparound setting), since it destroys old logs without asking. Also closes the current log and starts a new one when the current log can't grow any further but the card still has room to reclaim, rather than just stopping. Also fixes three bugs found while stress-testing the above against real SITL-driven SD card images: - Guard the shared current-directory cursor state against two concurrent file-creation operations interleaving, which could make one operation wrongly conclude the directory ends at an earlier cluster than it really does and re-extend it there, corrupting the existing chain. - Skip FAT chain erasure when deleting a file that never had a cluster allocated to it (e.g. a flight too short to log a single frame). This previously wrote real chain links into the FAT's two reserved entries, corrupting the whole filesystem rather than just the one file. - Exclude never-allocated (cluster 0) directory entries from the oldest-log scan's candidate selection, since one would otherwise always look like the oldest log and permanently stop wraparound from reclaiming space even though reclaimable logs exist elsewhere.
|
ⓘ Qodo reviews are paused because the subscription is no longer active. Ask your workspace admin to reactivate the subscription to resume reviews. Manage billing |
|
RAM / Flash usage vs. base commit
See RAM/flash optimization guide for techniques to reduce usage. |
|
Test firmware build ready — commit Download firmware for PR #12150 251 targets built. Find your board's
|
Drop comments that restate the code or narrate how it was developed; keep the non-obvious why (cluster 0 handling, flush ordering, cursor ownership). Also hoist the repeated entryFirstCluster != 0 checks into one guard.
…around Removes the setting and the parameter group version bump to keep the feature simple; a full SD card now always reclaims the oldest log.
|
why did you remove settings? sdcardWraparound |
The setting would have the pilot choose between these two options:
It occurred to me that between those two, there's really only one reasonable option. If there's a log from eight months ago you want to keep, of course you'd copy it to your computer, because any time you fly you know you may never see that SD card again. So the setting was adding complexity to allow for something that wouldn't make sense anyway. I'd rather pilots be able to look at the available INAV settings and set what they want, without wading through a bunch of pointless settings. |
|
First round of tests: I used 4GB SD card, MATEK H743, flashed this PR. I copied to SD card aaprox 3.8GB data. creating log if there is enought space is OK,, indexes 1 2 3 4 5, then space left and should be created log index 6, but on the sd card is index 7. Index 1 is missing, it's correct because log1 was used as the oldest one. 
Patch for fix: Then I noticed that this aproach make fragmentation, LOG1 size 5MB, if is needed LOG1 is deleted and to free space is saved new LOG[big number], but if new LOG[big number] ends early than LOG1, for example size of LOG[big number] will be 2MB then there will be 3MB gap, because next log LOG[big number + 1], starts on position LOG2. Solution could be expand FREESPACE for size of deleted log, but it's complicated opeartion and probably I would not recomend it during fligt. So propably mention in manual that there can be fragmentation. Add some information to header? Then there can be situation when a single flight is in multiple logs. But how I know which loggs belongs to single flight? And probably would be nice to adjust inav blackbox viewer to support multiple file for a single flight. |
When the evicted log was the last one on the card, the reclaim size came from its fileSize. A short flight's fileSize is below one supercluster, so wraparound gave up on every arm and logging never resumed. Logs grow a whole supercluster at a time, so round fileSize up to whole superclusters. If a chosen log is still too small to reclaim, clear its directory entry (including first cluster and size) and rescan instead of giving up, so the same log is not picked on every arm. Each pass removes one candidate, and wraparound still gives up once no logs remain.
…empty When the freefile cannot hold one supercluster, run the oldest-log reclaim before creating the log, instead of creating a log that is split before its first frame and discarded, which skipped a log number. Fix and diagnosis by error414, from testing this PR on a MATEK H743. Document how the SD card behaves when it fills: the oldest log by position is deleted, a flight can span several files, and unused space can be left behind.
|
@error414 thank you for the real-card testing. It found problems that SITL didn't. Here is what changed (169f05a):
I also ran it on a Hummingbird H743 with a cheap 31 GB card. Three splits during an armed run each cost about 65-70 ms of log frames, with no other gaps. I cut power once between deleting the oldest log and moving the free-space file. The card stayed consistent and logging resumed, but about 4 MB was leaked. That was one trial on one card. |
|
great, tested and it works |
Summary
Builds on the freefile-refill-at-boot fix (#12056): when the card is still completely full after that - no free clusters left anywhere - delete the oldest log and reclaim its space so logging can continue indefinitely on a card dedicated to blackbox, instead of stopping. This is always active when logging to an SD card: a full card silently loses its oldest log rather than stopping, so download anything you want to keep before reusing a card for long or continuous logging.
Also closes the current log and starts a new one when the current log can't grow any further but the card still has room to reclaim, rather than just stopping.
Changes
Testing
Tested (with wraparound active) against real SITL-built firmware (not a standalone model) driving actual SD card images, verified independently with
fsck.vfat/mtoolsand direct raw FAT reads (not just the project's own test scripts):mtoolssuccessfully mounts and lists the entire filesystem afterward.