Add dataclass encoding by zjkmxy · Pull Request #111 · named-data/python-ndn

zjkmxy · 2026-10-08T06:57:27Z

Ref: #97 #108

New wire format is longer but with advantage:

Built-in dataclass support, such as =, repr and asdict. Also, models can be easily encoded into JSON and other formats for human/subagents to read.
Built-in typing annotation. Type checker now can correctly understand field types.
Faster than v1. Using type hints cache, v2 is about 80% of the time in v1 for encoding and decoding.

Not so good points:

v2 typing is longer than v1.
Signature representation is still ugly.
Annotation handling could change with interpreters.
bytes field could actually hold memoryview. Need further effort to improve.

Co-authored-by: Cursor <cursoragent@cursor.com>

Copilot

🔵 Needs a closer look

Module shadowing leaves eight migrated examples with invalid attribute lookups that prevent startup or Interest expression.

0 open findings

What changed in this PR

This breaking migration adopts dataclass-based TLV models across python-ndn and makes ndn.app the canonical application API.

Changes:

Migrate protocol models, validators, tools, and examples.
Remove legacy application and schema APIs.
Add wire-format regression tests and update documentation.

File	Description
tests/security/signer_test.py	Assert the new validator contract.
tests/misc/svs_tlv_test.py	Test SVS dataclass wire formats.
tests/misc/security_v2_test.py	Test certificates and SafeBag encoding.
tests/misc/nfd_mgmt_test.py	Test management command and response formats.
tests/misc/light_versec_test.py	Test binary model round trips.
tests/integration/app_v2_test.py	Remove the separate v2 integration suite.
tests/integration/app_test.py	Migrate application tests and cover registration.
tests/encoding/ndnlp_v2_test.py	Extend NDNLP encoding and parsing coverage.
tests/encoding/ndn_format_0_3_test.py	Add packet and signature regression tests.
tests/encoding/ndn_format_0_3_2017_test.py	Test legacy packet-format compatibility.
src/ndn/types.py	Remove legacy callback aliases.
src/ndn/security/validator/known_key_validator.py	Return `ValidResult` from known-key validators.
src/ndn/security/validator/digest_validator.py	Update digest validation; remove legacy composition.
src/ndn/security/validator/cascade_validator.py	Remove the cascade validator.
src/ndn/security/validator/__init__.py	Update validator exports.
src/ndn/schema/util.py	Remove schema pattern utilities.
src/ndn/schema/simple_trust.py	Remove schema trust policies.
src/ndn/schema/simple_node.py	Remove schema resource nodes.
src/ndn/schema/simple_cache.py	Remove schema caching.
src/ndn/schema/schema_tree.py	Remove the namespace schema framework.
src/ndn/schema/policy.py	Remove schema policy interfaces.
src/ndn/schema/__init__.py	Remove the schema package initializer.
src/ndn/name_tree.py	Remove legacy application tree helpers.
src/ndn/encoding/tlv_model_v2.py	Retire the separate v2 codec module.
src/ndn/encoding/ndnlp_v2.py	Convert NDNLP models to dataclasses.
src/ndn/bin/tools/cmd_serve_rdrcontent.py	Switch to canonical application imports.
src/ndn/bin/tools/cmd_serve_data.py	Switch to canonical application imports.
src/ndn/bin/tools/cmd_fetch_rdrcontent.py	Switch to canonical application imports.
src/ndn/bin/tools/cmd_fetch_data.py	Switch to canonical application imports.
src/ndn/bin/nfdc/utils.py	Rename the command-expression helper.
src/ndn/bin/nfdc/cmd_set_strategy.py	Update application and command-helper references.
src/ndn/bin/nfdc/cmd_remove_strategy.py	Update application and command-helper references.
src/ndn/bin/nfdc/cmd_remove_route.py	Update application and command-helper references.
src/ndn/bin/nfdc/cmd_remove_face.py	Use dataclass encoding for face management.
src/ndn/bin/nfdc/cmd_new_route.py	Update command expression.
src/ndn/bin/nfdc/cmd_new_face.py	Pass URI strings to dataclass models.
src/ndn/bin/nfdc/cmd_get_strategy.py	Parse strategy dataclasses.
src/ndn/bin/nfdc/cmd_get_status.py	Parse status dataclasses.
src/ndn/bin/nfdc/cmd_get_route.py	Parse routing dataclasses.
src/ndn/bin/nfdc/cmd_get_face.py	Encode queries and parse face dataclasses.
src/ndn/appv2.py	Remove the separate v2 application module.
src/ndn/app_support/svs/tlv.py	Convert SVS models to dataclasses.
src/ndn/app_support/svs/sync.py	Adopt canonical application and codec APIs.
src/ndn/app_support/segment_fetcher.py	Remove the legacy segment fetcher.
src/ndn/app_support/security_v2.py	Migrate certificate models and signing.
src/ndn/app_support/nfd_mgmt.py	Migrate management models and helpers.
src/ndn/app_support/light_versec/validator.py	Remove legacy LVS validation integration.
src/ndn/app_support/light_versec/binary.py	Convert LVS binary models to dataclasses.
src/ndn/app_support/light_versec/__init__.py	Remove legacy validator imports.
src/ndn/app_support/keychain_register.py	Use canonical application imports.
src/ndn/app_support/dispatcher.py	Remove the legacy dispatcher.
examples/rpc_producer.py	Remove the legacy RPC producer.
examples/rpc_consumer.py	Remove the legacy RPC consumer.
examples/rdrnode.py	Remove the schema-based RDR example.
examples/putchunks.py	Remove the legacy chunk producer.
examples/producer.py	Remove the legacy producer example.
examples/nfd_status.py	Remove the legacy status example.
examples/lvs/producer.py	Remove the legacy LVS producer.
examples/lvs/consumer.py	Remove the legacy LVS consumer.
examples/dpdk_experimental/udp_producer.py	Migrate DPDK producer application references.
examples/dpdk_experimental/udp_consumer.py	Migrate DPDK consumer application references.
examples/consumer.py	Remove the legacy consumer example.
examples/catchunks.py	Remove the legacy chunk consumer.
examples/app/svs/sync_example.py	Migrate SVS application references.
examples/app/keychain_cert/keychain_register.py	Use the canonical application constructor.
examples/app/keychain_cert/fetch_certificate.py	Migrate certificate-fetching application references.
examples/app/forwarding_hint/producer.py	Migrate producer application references.
examples/app/forwarding_hint/consumer.py	Migrate consumer application references.
examples/app/basic_packets/producer.py	Migrate basic producer application references.
examples/app/basic_packets/consumer.py	Migrate basic consumer application references.
docs/src/security/security.rst	Document the new validator contract.
docs/src/schema/utils.rst	Remove schema utility documentation.
docs/src/schema/schema.rst	Remove the schema overview.
docs/src/schema/schema_tree.rst	Remove schema tree documentation.
docs/src/schema/policies.rst	Remove schema policy documentation.
docs/src/schema/ex1.rst	Remove the schema file-sharing tutorial.
docs/src/schema/custom_node.rst	Remove custom-node documentation.
docs/src/lvs/package.rst	Update the type-alias documentation directive.
docs/src/examples/tlv_model.rst	Replace descriptor examples with dataclasses.
docs/src/examples/basic_app.rst	Document canonical consumer and producer APIs.
docs/src/encoding/tlv_model.rst	Document the dataclass codec API.
docs/src/encoding/encoding.rst	Update the encoding overview.
docs/src/appv2.rst	Remove the separate v2 application reference.
docs/src/app.rst	Document the canonical application API.
docs/index.rst	Remove retired documentation entries.
CHANGELOG.rst	Record the breaking migration and removals.

🧠 Review effort: Balanced

Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

Co-authored-by: Cursor <cursoragent@cursor.com>

zjkmxy · 2026-10-12T15:53:34Z

@yoursunny Do you mind giving a review of this new version? Thanks!

Copilot

🟡 Changes recommended

Certificate-cache expiration and empty KeyLocator handling leave unresolved validation defects.

2 open issues

_{Review effort: Balanced}

zjkmxy · 2026-10-19T00:43:11Z

+                self.storage.save_with_expiration(
+                    cert_name,
+                    key_bits,
+                    expires_at,


This is a design question. Let's discuss what behavior should we do.

A cache, by definition, should have the same behavior when the cached entry is present or when the cache is cleared.
If the validator would be rejecting the certificate when the cache is cleared but is accepting the certificate when the cache entry is present, this is incorrect behavior.

However, if the base commit has the same behavior, you can keep it as is, as it’s out of scope of this change.
In that case, please create/find an issue, and then insert a comment in the code linking to that issue.

It's Copilot's error to claim it's a cache. In design, it is considered a simple storage, not a cache. The user could inject trust anchor into the storage manually.

The question is: if a certificate C1 expires at T1, and it signs a certificate C2 expiring at T2 (T2 > T1), should we consider C2 invalid at T1?

The question is: if a certificate C1 expires at T1, and it signs a certificate C2 expiring at T2 (T2 > T1), should we consider C2 invalid at T1?

Open a separate issue for this debate, but add a comment in the code so that Copilot doesn't flag this again.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

yoursunny · 2026-10-19T00:54:27Z

+                name,
+                validator=app.pass_all,
+                app_param=app_param,
+                signer=DigestSha256Signer(for_interest=True),


Previous example was parametrized Interest but this example is signed Interest.
If python-ndn no longer supports sending parametrized Interest, this limitation should be mentioned in the changelog.

Sorry I'm not sure what's the difference.

In NDNApp v1: a default signer is automatically used if app_param presents, unless the user explicitly specifies signer=None, which is not the case in the example:

python-ndn/examples/rpc_consumer.py

Lines 43 to 44 in 00727d8

data_name, meta_info, content = await app.express_interest(

name, app_param.encode(), must_be_fresh=True, can_be_prefix=False, lifetime=6000)

python-ndn/src/ndn/app.py

Lines 211 to 216 in 00727d8

if 'signer' in kwargs:

signer = kwargs['signer']

elif app_param is not None:

signer = self.keychain.get_signer(kwargs)

else:

signer = None

In NDNApp v2: an Interest with app_param but without a signer is rejected. (Though the user can use express_raw_interest to bypass this restriction)

python-ndn/src/ndn/appv2.py

Line 617 in 00727d8

:param signer: Signer for Interest signing. This is required if `app_param` is specified.

python-ndn/src/ndn/appv2.py

Lines 637 to 638 in 00727d8

if app_param is not None and signer is None:

raise ValueError('An Interest with AppParam is required to be signed.')

This PR retains NDNApp v2's behavior, and the function signature clearly states that signer is mandatory if app_param presents.

zjkmxy and others added 7 commits October 7, 2026 23:25

encoding: add dataclass TLV codec to canonical model

dc3c99e

Co-authored-by: Cursor <cursoragent@cursor.com>

encoding: migrate packet and LP models to dataclasses

a9cfd47

Co-authored-by: Cursor <cursoragent@cursor.com>

app-support: migrate protocol models to dataclasses

32f76fd

Co-authored-by: Cursor <cursoragent@cursor.com>

app: promote v2 implementation to ndn.app

5ea1c18

Co-authored-by: Cursor <cursoragent@cursor.com>

app: remove legacy application surfaces

ac29339

Co-authored-by: Cursor <cursoragent@cursor.com>

encoding: remove legacy TLV descriptor API

9144999

Co-authored-by: Cursor <cursoragent@cursor.com>

docs: document breaking dataclass and app APIs

12e89f2

Co-authored-by: Cursor <cursoragent@cursor.com>

zjkmxy requested a balanced review from Copilot October 8, 2026 06:57

Copilot started reviewing on behalf of zjkmxy October 8, 2026 06:57

View session

zjkmxy self-assigned this Oct 8, 2026

Copilot AI reviewed Oct 8, 2026

View reviewed changes

style: use Python 3.11 typing syntax

39afa3f

Co-authored-by: Cursor <cursoragent@cursor.com>

yoursunny suggested changes Oct 13, 2026

View reviewed changes

yoursunny reviewed Oct 13, 2026

View reviewed changes

Comment thread docs/src/encoding/tlv_model.rst

Migrate some obsoleted examples

f20ea04

yoursunny reviewed Oct 14, 2026

View reviewed changes

Comment thread CHANGELOG.rst Outdated

zjkmxy added 2 commits October 18, 2026 11:11

fix: CHANGELOG and cmd_fetch_data bug

4ca2cb3

encoding: Make tlv_field helper class

a69153a

zjkmxy force-pushed the dev3 branch from f8eb5c1 to a69153a Compare October 18, 2026 18:30

docs: Fix document for tlv_model

1e89a7e

zjkmxy requested a balanced review from Copilot October 18, 2026 23:04

Copilot started reviewing on behalf of zjkmxy October 18, 2026 23:05

View session

Copilot AI reviewed Oct 18, 2026

View reviewed changes

lvs: Fix key_locator name check condition

583dbfe

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

yoursunny reviewed Oct 19, 2026

View reviewed changes

	data_name, meta_info, content = await app.express_interest(
	name, app_param.encode(), must_be_fresh=True, can_be_prefix=False, lifetime=6000)

	if 'signer' in kwargs:
	signer = kwargs['signer']
	elif app_param is not None:
	signer = self.keychain.get_signer(kwargs)
	else:
	signer = None

	if app_param is not None and signer is None:
	raise ValueError('An Interest with AppParam is required to be signed.')

Conversation

zjkmxy commented Oct 8, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

zjkmxy commented Oct 12, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

zjkmxy Oct 19, 2026

Choose a reason for hiding this comment

Uh oh!

yoursunny Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zjkmxy Oct 19, 2026

Choose a reason for hiding this comment

Uh oh!

yoursunny Oct 19, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

yoursunny Oct 19, 2026

Choose a reason for hiding this comment

Uh oh!

zjkmxy Oct 19, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

yoursunny Oct 19, 2026 •

edited

Loading