Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [2.5.0] - 2026-10-25

### Added

- **`VRRP` and `VRRPv6`: the Virtual Router Redundancy Protocol, VRRPv2 (RFC 3768) and VRRPv3 (RFC 5798) (#181).** IP protocol 112 (`IPProtocol.VRRP`, "VRRP") now dispatches to `netprotocols.VRRP` from IPv4 and to its subclass `VRRPv6` from IPv6 and its extension headers. One class decodes both versions and branches on the version nibble: the fixed fields `version`, `type`, `vrid`, `priority`, `count_ip_addrs` and `checksum`, the 16 bits between Count and Checksum kept raw as `adver_word` and read through per-version accessors that return `None` where the version has no such field (`auth_type`/`auth_type_name` and `adver_int` in seconds for VRRPv2; `reserved` and `max_adver_int` in centiseconds for VRRPv3; `advertisement_interval` in seconds for either), the virtual `addresses`, and VRRPv2's 8 bytes of `auth_data` (the null-padded password under Auth Type 1, keepalived's `auth_type PASS`). The address width depends on the IP family, not on anything in the header, which is why the class comes as a pair: `VRRP` reads 4-byte addresses and `VRRPv6` 16-byte ones, and `VRRPv6` accepts VRRPv3 only, since RFC 3768 defines VRRPv2 for IPv4 alone. VRRP has no length field, but the version and count give its exact length, which is `header_len`, so Ethernet padding and an FCS after the message are left undecoded and `bytes(VRRP.decode(x)) == x[:header_len]`. A version other than 2 or 3 raises `InvalidFieldError` (`field="version"`, `offset=0`), because the version decides the layout; an unknown type, a priority of 0 (a master resigning), a count of 0 and the reserved bits are carried verbatim. A count, or VRRPv2's authentication data, past the buffer raises `TruncatedHeaderError` (`field="count_ip_addrs"` or `"auth_data"`). Construction also checks `count_ip_addrs == len(addresses)`, the authentication data's length against the version (8 bytes or none), and that `type` fits its nibble. The walk ends at VRRP. Registering over protocol 112 in `ip.proto` reaches IPv4 only, as for AH's 51; `_defaults.FAMILY_SPECIFIC_NUMBERS` now names both.
Expand All @@ -20,6 +22,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Development

- **Every comparative number the README quotes was re-measured for this release** (Rule 4 of `docs/CLAIMS.md`), on the same machine class as 2.4.0's figures but on the 265-frame corpus Tier 2 grew from 140, against `dpkt 1.9.8` and `scapy 2.8.0`. **The README's "within 15% of dpkt on decode" no longer holds and is replaced**: dpkt decodes the corpus 1.30× faster (1.27-1.30× over three runs), while netprotocols stays 3.4× ahead of scapy. A file-by-file A/B against the 2.4.0 tree puts the gap in the five new routing captures, on which dpkt takes about two thirds of netprotocols' time, while the 140 frames 2.4.0 was measured on still read within 10% of dpkt on both trees, so it is the workload rather than a regression; why dpkt is faster on those protocols was not profiled. Encode is 4.62× ahead of dpkt, import 70.0 ms (from 55.0 ms, the five new modules) against `scapy.all`'s 396.9 ms, and the wheel is 176.5 KiB, 15.3× smaller than scapy 2.8.0's 2.63 MiB. Decode depth against dpkt is 79 of 265 frames deeper, the 40 new ones being the BGP and RIP frames, which dpkt leaves as raw bytes. `docs/CLAIMS.md` records each run, the per-file table and the A/B, and renames claim 1.2 for the figure it now holds.
- **The decode benchmark baseline is re-recorded on the 265-frame corpus.** `benchmarks/baseline.json` was recorded at 2.2.1 on the 97-frame corpus (6.5677 normalized); it now holds 2.5.0 on 265 frames, 5.4344 normalized (97,453 f/s, 30 repetitions, best of 9, CPython 3.12.3), via `scripts/benchmark.py --update-baseline`. The corpus `--check` walks grew with Tier 1 and Tier 2 captures that cost more per frame than the old ones, so normalized throughput fell with the workload: the same library code read -14.1% on master (`03387af`) and -17.1% (-16.5% on the re-measure) on the release PR, either side of the 15% gate on runner noise alone, and the file-by-file A/B of 2.4.0 against 2.5.0 in `docs/CLAIMS.md` shows no regression beyond that harness's noise on the 140 frames 2.4.0 had. Re-anchoring accepts the drift since 2.2.1: the gate now guards against regressions relative to 2.5.0 on this corpus.
- **A real BGP capture joins the corpus: `bgp.pcap`, 19 frames (265 across 27 scenarios), and its whole session goes into `tests/fixtures/bgp/`.** `scripts/capture_fixtures_bgp.sh` runs three BIRD 2.14 speakers in netns on one bridge, an eBGP and an iBGP session, with offload disabled, through OPENs, KEEPALIVEs, UPDATEs (five prefixes each with its own MED, 600 /24s that BIRD packs into a few UPDATEs, some longer than a segment), a withdrawal and a Cease NOTIFICATION with an RFC 9003 shutdown communication. It reassembles each direction to assert what was announced and withdrawn, prefix by prefix, and that the capture holds a segment of several whole messages, one of whole messages then a cut one, and one starting inside a message; it writes every frame to `bgp/bgp_session.pcap` (94 frames, outside the corpus globs) and the frames whose payload is whole messages to `bgp.pcap`. `scripts/check_fixtures.py` needed no change: it verifies the TCP checksums that cover the BGP payload. The corpus's deepest chain is now 10 layers (a segment of 7 UPDATEs); `walk.py` and `docs/CLAIMS.md` §5.7 say so. `tests/test_fuzz.py` gains BGP accessor-safety and built-UPDATE properties.
- **A real PIM-SM capture joins the corpus: `pim.pcap`, 13 frames (246 across 26 scenarios).** `scripts/capture_fixtures_pim.sh` runs two FRR 8.4 routers (zebra + pimd) between a multicast source and an IGMPv3 receiver, four netns over three veth pairs, with static RPs on opposite routers for two groups so that a (*,G) tree and a Register exchange both cross the captured link, then has the receiver leave and stops one pimd. It keeps the first frame of each of 13 kinds — Hellos with Holdtime, LAN Prune Delay, DR Priority, Generation ID and Address List options, a Holdtime-0 goodbye, a data Register and a Null-Register, a Register-Stop, and Join/Prune messages with joined and pruned (*,G), (S,G) and (S,G,rpt) entries — and fails unless it saw all of them. FRR sums its Registers' checksums over the first 8 bytes. `scripts/check_fixtures.py` verifies every PIM checksum (IPv4, and IPv6 with its pseudo-header), accepts either Register form, and checks the Register's encapsulated packet in turn, skipping only a Null-Register's zero dummy IPv4 header checksum, which RFC 7761 says a receiver must not check; an IPv6 Null-Register's dummy PIM header (version 0, after the dummy IPv6 header) is checked by its own checksum over the dummy header's source and group, as §4.9.3 asks, rather than as a PIM message; `tests/test_pim.py` runs it on the frames and on tampered copies. The capture host has no IPv6, so PIM over IPv6 is covered by constructed frames whose pseudo-header checksums the test computes itself; Assert, Graft and the raw types are covered by constructed messages, and real frames are tested with zero padding and a 4-byte trailer appended. PIM joins the fuzz suite (`ALL_PROTOCOLS`, a round-trip strategy, an accessor-safety property and trailer properties), the truncation and damaged-payload contract tests, the corpus checksum cases and the corpus coverage set.
- **`_tlv.walk_kind_length_value` takes a `width`** (default 1), the size of the kind and length fields, so PIM Hello options (2-byte type and length) walk through it. The floor and overrun message are now derived once per call, which takes the walker from radon C (13) to B (10); a seeded 3000-case comparison against a verbatim copy of the previous walker pins every byte-wide caller (TCP, IPv4, IPv6 options) to identical results and errors. `checksum.compute()`'s PIM arm is one call to a `_pim_checksum` helper, beside the `_pseudo_header_checksum` helper the ICMPv6, TCP and UDP arms already use, so `compute()` stays at radon B. The scoped mutation runs on `netprotocols._tlv.*` and `netprotocols.checksum.*` leave no survivor beyond the one already allowlisted.
Expand Down
44 changes: 26 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,31 +51,33 @@ Requires Python 3.12+. Fully typed (`py.typed`, mypy strict).

## Why NETProtocols

Measured against `dpkt 1.9.8` and `scapy 2.8.0` on the 140 frames of
real captured traffic the corpus held at 2.4.0 (CPython 3.12.3, x86-64 Linux; re-measured for
2.4.0 on 2026-10-25 — see [docs/CLAIMS.md](docs/CLAIMS.md) for every
Measured against `dpkt 1.9.8` and `scapy 2.8.0` on the 265-frame corpus
of real captured traffic (CPython 3.12.3, x86-64 Linux; re-measured for
2.5.0 on 2026-10-25 — see [docs/CLAIMS.md](docs/CLAIMS.md) for every
number below, its reproduction command, and its caveats):

- **Within 15% of dpkt on decode, 4.1× faster than scapy** —
- **3.4× faster than scapy on decode; dpkt is 1.3× faster** —
`uv run --group bench python scripts/benchmark.py --compare`. dpkt is
the faster of the two comparators, and on this corpus it and
netprotocols are at parity, the sign of the gap flipping between
runs. The gap moves both ways as this library and dpkt each change, and this
file's job is to say so honestly rather than only when it's
the faster of the two comparators. On the 140 frames the corpus held
at 2.4.0 the two are still within 10% of each other; the
routing captures 2.5.0 added (OSPF, BGP, PIM, RIP, VRRP) are where
dpkt pulls ahead, taking about two thirds of netprotocols' time on
them. The gap moves both ways as this library and dpkt each change,
and this file's job is to say so honestly rather than only when it's
flattering.
- **Decodes further into the stack than dpkt on 39 of those 140
frames** — DNS, DHCP and VXLAN-tunnelled payloads that dpkt leaves as
raw bytes, and 802.1Q tags it folds into the Ethernet header,
netprotocols decodes as layers of their own. A throughput number
means little without knowing how much work it bought.
- **4.2× faster than dpkt at re-encoding** (`bytes(header)`,
- **Decodes further into the stack than dpkt on 79 of those 265
frames** — BGP, RIP, DNS, DHCP and VXLAN-tunnelled payloads that dpkt
leaves as raw bytes, and 802.1Q tags it folds into the Ethernet
header, netprotocols decodes as layers of their own. A throughput
number means little without knowing how much work it bought.
- **4.6× faster than dpkt at re-encoding** (`bytes(header)`,
`scripts/benchmark_encode.py`) — the other half of "codec" that
nobody else benchmarks.
- **Imports in 55 ms; `scapy.all` takes 428+ ms** (`scripts/benchmark_import.py`),
- **Imports in 70 ms; `scapy.all` takes 397+ ms** (`scripts/benchmark_import.py`),
and never touches the host doing it — importing scapy populates live
interface and routing tables as a side effect of the import
statement.
- **A 124.4 KiB wheel against scapy's 2.63 MiB** — 21.7× smaller.
- **A 176.5 KiB wheel against scapy's 2.63 MiB** — 15.3× smaller.
- **Regression-gated in CI, which — as far as we could establish by
auditing ten comparable Python packet libraries' CI configurations
(dpkt, scapy, pypacker, construct, pcapkit, dnspython, pyshark,
Expand Down Expand Up @@ -479,7 +481,13 @@ and SCTP
[#231](https://github.com/EONRaider/NETProtocols/pull/231),
[#232](https://github.com/EONRaider/NETProtocols/pull/232),
[#233](https://github.com/EONRaider/NETProtocols/pull/233),
[#234](https://github.com/EONRaider/NETProtocols/pull/234)) — as 2.4.0
[#234](https://github.com/EONRaider/NETProtocols/pull/234)) — as 2.4.0,
and the second — VRRP, RIP, OSPF, PIM and BGP
([#243](https://github.com/EONRaider/NETProtocols/pull/243),
[#245](https://github.com/EONRaider/NETProtocols/pull/245),
[#246](https://github.com/EONRaider/NETProtocols/pull/246),
[#247](https://github.com/EONRaider/NETProtocols/pull/247),
[#248](https://github.com/EONRaider/NETProtocols/pull/248)) — as 2.5.0
(see [CHANGELOG.md](CHANGELOG.md)).

The roadmap before it, the post-1.3.0
Expand All @@ -506,7 +514,7 @@ released as
2.1.0's and 1.3.1/1.4.0's work shipped folded into 2.0.0 and 2.2.0
rather than as their own versions, but the roadmap's original "only at
a major version bump" release policy wasn't actually held to: 2.0.0,
2.2.0, 2.2.1, 2.3.0 and now 2.4.0 are all on PyPI, and the comparative claims above
2.2.0, 2.2.1, 2.3.0, 2.4.0 and now 2.5.0 are all on PyPI, and the comparative claims above
draw on the finished tree. The wave before that one — TCP
and IPv4 option parsing, ICMP message bodies, NDP, IPv6
extension-header TLV options, the GRE checksum arm, a DNS-over-TCP
Expand Down
10 changes: 5 additions & 5 deletions benchmarks/baseline.json
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
{
"frames_per_sec": 98007.5,
"calibration_per_sec": 14922.6,
"normalized": 6.5677,
"frames": 97,
"frames_per_sec": 97453.2,
"calibration_per_sec": 17932.6,
"normalized": 5.4344,
"frames": 265,
"repetitions": 30,
"trials": 9,
"recorded": {
"on": "CPython 3.12.3 / x86_64 / Linux",
"python": "3.12.3",
"netprotocols": "2.2.1"
"netprotocols": "2.5.0"
}
}
Loading
Loading