Repository navigation
feat(persistent_data): pure-Python counterpart (espp.persistent_data) - #864
Merged
Merged
Conversation
Reads and writes the firmware's versioned settings files from Python with no extension module: a 20-byte header (magic, schema version, layout fingerprint, length, CRC32) in front of an alpaca payload encoded with fixed_length_encoding | strict, which is fixed-width little-endian integers and 32-bit length prefixes, so struct + zlib cover it. Schema versions are dataclasses, newest first, mirroring the C++ type list: a VERSION class variable, migrate(older) on every version but the last, width aliases (u8 ... u64, i8 ... i64, f32, f64, char) for numeric fields (a bare int / float is an error, never a guess), array(T, N) for std::array, unordered(dict/set) for the unordered containers, Optional[T], tuple[...], enums with an Annotated underlying width, and nested dataclasses. The layout fingerprint is derived from the annotations by the grammar in the component README, so a file written in Python verifies on the device and vice versa; a mismatch is layout_mismatch, never wrong values. PersistentData(versions, path): load / load_or_create / save / update / erase / encode / decode, atomic temp + rename writes, the same error taxonomy as PersistentDataErrc (PersistentDataError.errc), thread-safe. python/persistent_data_test.py checks describe() bytes, fingerprints and whole encoded files byte-for-byte against vectors generated by the C++ codec (python/persistent_data_vectors.txt), the migration chain, every error path, the other field types and the file operations (76 checks); it runs from the source tree without a built wheel and is added to the wheel test command. Exported as espp.persistent_data; documented in the component README and python/README.md. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DFjjnq3XCTRAXENSxsCxJU
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
|
✅Static analysis result - no issues found! ✅ |
- Code-quality bot: the two bare `except: pass` (erase of a missing file, temp-file cleanup after a failed write) are explicit now, and the unused ClassVar import is gone. - load(): a failed post-migration rewrite is logged (logging, module logger) instead of silently swallowed, matching the C++ warning. - A VERSION declared as a dataclass field instead of a ClassVar is rejected with a message saying so (it used to surface as a confusing "bare int" error from the payload spec). - A dataclass constructor that raises during decode (a __post_init__ validation) is reported as deserialize_failed instead of escaping as a raw exception. - parse() is exported; the test imports Annotated from typing rather than through the module; the module docstring's type-mapping table is a valid RST table (column widths). - Two new test checks for the VERSION-as-field and constructor-failure cases (78 total). pyflakes clean. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DFjjnq3XCTRAXENSxsCxJU
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
espp.persistent_data: the Python counterpart of thepersistent_datacomponent (#857), so a PC tool can read, edit or generate a device's versioned settings file, and migration chains can be exercised on the host.fixed_length_encoding | strict: fixed-width little-endian integers, 32-bit length prefixes, nothing else.struct+zlibcover it (alpaca's CRC32 is the zlib one; verified on the vectors). pyalpaca stays an upstream tool; it would have added a native build to every wheel for the variable-length encoding this format never uses.VERSIONclass variable,migrate(older)on every version but the last, width aliases (u8 … u64,i8 … i64,f32,f64,char) for numeric fields — a bareint/floatis aTypeError, never a silent width guess — plusarray(T, N),unordered(dict | set),Optional[T],tuple[...], enums with anAnnotatedunderlying width, nested dataclasses.layout_mismatch.PersistentData(versions, path):load,load_or_create,save,update,erase,encode,decode; atomic temp-and-rename writes;PersistentDataError.errcmirrorsPersistentDataErrc; thread-safe.python/persistent_data_test.py(78 checks):describe()bytes, fingerprints and whole encoded files byte-for-byte againstpython/persistent_data_vectors.txtgenerated by the C++ codec (enum, fixed array, nested struct, every integer width, float/double, string, vector), the v1 → v2 → v3 migration chain, dropped / newer / reused versions, every corruption path, the remaining field types (map, unordered_map, set, optional, tuple, char) and the file operations. Runs from the source tree without a wheel; added to the cibuildwheel test command.from . import persistent_data), documented in the component README ("Python" section) andpython/README.md.Test plan
python/persistent_data_test.py: 78/78 on Python 3.15 and 3.12 from the source treeespppackage: host library built withESPP_BUILD_PYTHON=ONfrom this branch,persistent_data.pypresent in the installed packageVERSIONdeclared as a field,deserialize_failedfor a dataclass constructor that raises,parseexported; pyflakes clean; 78/78 on 3.15 / 3.12 from source and through the installed package🤖 Generated with Claude Code
https://claude.ai/code/session_01DFjjnq3XCTRAXENSxsCxJU