From 0200dbb9725729f4d5dfb02529fc9024cf7c96ee Mon Sep 17 00:00:00 2001 From: Ray Walker Date: Sat, 24 Oct 2026 04:42:12 +0000 Subject: [PATCH 1/3] docs(matrix): rate Python stampede prevention by the 0.23.0 single-flight The Python cell of the Cache stampede prevention row still described the pre-0.23.0 behaviour. cachekit 0.23.0 ships in-process single-flight (cachekit-py#547) for the async path in every mode and for the sync path without a backend. The sync path with a backend still does not coalesce, so the cell moves to a partial rating and states that gap. --- sdk-feature-matrix.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sdk-feature-matrix.md b/sdk-feature-matrix.md index 5d33628..4568439 100644 --- a/sdk-feature-matrix.md +++ b/sdk-feature-matrix.md @@ -180,7 +180,7 @@ The contract a storage backend must satisfy per SDK (bytes in / bytes out; seria | Backpressure | ✅ | ✅ Semaphore + bounded queue, decision recorded (LAB-729)¹³ | ⚠️ Refresh cap only — deliberate, decision recorded (LAB-519) | ❌ | | Distributed locking | ✅ Redis + SaaS backends | ✅ Redis + SaaS + Workers (`LockableBackend`; LAB-426) | ✅ Redis + SaaS backends; wired into `wrap()` cold miss (opt-in `stampede.distributedLock`, LAB-519) | ❌ | | L1/L2 dual-layer cache | ✅ | ✅ moka (native) / `l1` feature | ✅ | ❌ | -| Cache stampede prevention | ✅ Async decorators via backend lock; sync path none by design | ✅ Cold-miss single-flight wired into every `#[cachekit]` expansion — per-key in-process gate plus a distributed fill lock on lock-capable backends (the `single_flight` call in `expand`'s generated body, `cachekit-macros/src/lib.rs`; `SingleFlight::acquire` in `flight.rs`; LAB-518)¹³ | ✅ Cold-miss single-flight + SWR version tokens (LAB-519) | ❌ | +| Cache stampede prevention | ⚠️ In-process single-flight from `cachekit` 0.23.0+ ([cachekit-py#547](https://github.com/cachekit-io/cachekit-py/pull/547)): concurrent cold misses on one key in one process run the function once on the async path with a backend, `@cache(backend=None)` or `@cache.local()`, and on the sync path with `@cache(backend=None)` or `@cache.local()`; an async herd with a backend makes one call's backend requests, so on a lockable backend it takes the distributed lock once. **Not the sync path with a backend:** it takes no lock and does not coalesce yet, so every concurrent miss runs the function. Verified 2026-10-24 on the published 0.23.0 and 0.24.1 wheels: each tag's `tests/unit/test_single_flight.py` passes against its own wheel, and a cold herd of 20 calls ran the function once in each covered mode and 20 times on the sync path with a backend (0.22.0: 20 times in every mode) | ✅ Cold-miss single-flight wired into every `#[cachekit]` expansion — per-key in-process gate plus a distributed fill lock on lock-capable backends (the `single_flight` call in `expand`'s generated body, `cachekit-macros/src/lib.rs`; `SingleFlight::acquire` in `flight.rs`; LAB-518)¹³ | ✅ Cold-miss single-flight + SWR version tokens (LAB-519) | ❌ | | Cross-instance L1 invalidation (pub/sub) | ✅ 0.22.0+ — opt-in listener (`CACHEKIT_INVALIDATION_LISTENER_ENABLED`) over a Redis pub/sub channel, publish automatic on the tenant-scoped Redis backend ([cachekit-py#470](https://github.com/cachekit-io/cachekit-py/pull/470), [#471](https://github.com/cachekit-io/cachekit-py/pull/471), [#472](https://github.com/cachekit-io/cachekit-py/pull/472); wheel inspected 2026-10-05)⁹ | ❌ | ✅ Opt-in `invalidation` config over a Redis pub/sub channel (`invalidation/redis-channel.ts`) | ❌ | | TTL management | ✅ Redis + SaaS + File; Memcached refresh-only (see note) | ✅ Redis + SaaS + File + Workers (`TtlInspectable`); Memcached refresh-only (LAB-429/426) | ✅ Redis + SaaS + File (`TTLBackend`); Memcached refresh-only (LAB-430) | ❌ | | Stale-while-revalidate (client L1) | ⚠️ L1-only mode (`backend=None`) only, and only with a TTL: the `production` and `dev` presets supply one by default from `cachekit` 0.20.0, a bare `@cache(backend=None)` still needs `ttl=`¹⁰ | ✅ Refresh-ahead within the L1 TTL: an entry past `swr_threshold_ratio` of its TTL is still served while one single-flight background refresh runs, and is never served after the TTL (LAB-728)¹⁰ ¹³ | ✅ `getWithSwr` — refresh-ahead within the L1 TTL: version tokens + background refresh, `maxConcurrentRefreshes` cap, never served after the TTL¹⁰; on Workers requires a bound `ExecutionContext` (see [Cache Backends](#cache-backends) note ¹) | ❌ | From cce76b6799447634cb83bf7293f6ea55dc31c844 Mon Sep 17 00:00:00 2001 From: Ray Walker Date: Sat, 24 Oct 2026 04:52:10 +0000 Subject: [PATCH 2/3] docs(matrix): scope async single-flight to one event loop A caller on another thread's event loop runs its own call, so the async path coalesces per event loop, not per process. --- sdk-feature-matrix.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sdk-feature-matrix.md b/sdk-feature-matrix.md index 4568439..7b27e55 100644 --- a/sdk-feature-matrix.md +++ b/sdk-feature-matrix.md @@ -180,7 +180,7 @@ The contract a storage backend must satisfy per SDK (bytes in / bytes out; seria | Backpressure | ✅ | ✅ Semaphore + bounded queue, decision recorded (LAB-729)¹³ | ⚠️ Refresh cap only — deliberate, decision recorded (LAB-519) | ❌ | | Distributed locking | ✅ Redis + SaaS backends | ✅ Redis + SaaS + Workers (`LockableBackend`; LAB-426) | ✅ Redis + SaaS backends; wired into `wrap()` cold miss (opt-in `stampede.distributedLock`, LAB-519) | ❌ | | L1/L2 dual-layer cache | ✅ | ✅ moka (native) / `l1` feature | ✅ | ❌ | -| Cache stampede prevention | ⚠️ In-process single-flight from `cachekit` 0.23.0+ ([cachekit-py#547](https://github.com/cachekit-io/cachekit-py/pull/547)): concurrent cold misses on one key in one process run the function once on the async path with a backend, `@cache(backend=None)` or `@cache.local()`, and on the sync path with `@cache(backend=None)` or `@cache.local()`; an async herd with a backend makes one call's backend requests, so on a lockable backend it takes the distributed lock once. **Not the sync path with a backend:** it takes no lock and does not coalesce yet, so every concurrent miss runs the function. Verified 2026-10-24 on the published 0.23.0 and 0.24.1 wheels: each tag's `tests/unit/test_single_flight.py` passes against its own wheel, and a cold herd of 20 calls ran the function once in each covered mode and 20 times on the sync path with a backend (0.22.0: 20 times in every mode) | ✅ Cold-miss single-flight wired into every `#[cachekit]` expansion — per-key in-process gate plus a distributed fill lock on lock-capable backends (the `single_flight` call in `expand`'s generated body, `cachekit-macros/src/lib.rs`; `SingleFlight::acquire` in `flight.rs`; LAB-518)¹³ | ✅ Cold-miss single-flight + SWR version tokens (LAB-519) | ❌ | +| Cache stampede prevention | ⚠️ In-process single-flight from `cachekit` 0.23.0+ ([cachekit-py#547](https://github.com/cachekit-io/cachekit-py/pull/547)): concurrent cold misses on one key run the function once, per event loop on the async path (with a backend, `@cache(backend=None)` or `@cache.local()`) and per process on the sync path (`@cache(backend=None)` or `@cache.local()`); an async herd with a backend makes one call's backend requests, so on a lockable backend it takes the distributed lock once. **Not the sync path with a backend:** it takes no lock and does not coalesce yet, so every concurrent miss runs the function. Verified 2026-10-24 on the published 0.23.0 and 0.24.1 wheels: each tag's `tests/unit/test_single_flight.py` passes against its own wheel, and a cold herd of 20 calls on one key ran the function once in each covered mode and 20 times on the sync path with a backend (0.22.0: 20 times in every mode) | ✅ Cold-miss single-flight wired into every `#[cachekit]` expansion — per-key in-process gate plus a distributed fill lock on lock-capable backends (the `single_flight` call in `expand`'s generated body, `cachekit-macros/src/lib.rs`; `SingleFlight::acquire` in `flight.rs`; LAB-518)¹³ | ✅ Cold-miss single-flight + SWR version tokens (LAB-519) | ❌ | | Cross-instance L1 invalidation (pub/sub) | ✅ 0.22.0+ — opt-in listener (`CACHEKIT_INVALIDATION_LISTENER_ENABLED`) over a Redis pub/sub channel, publish automatic on the tenant-scoped Redis backend ([cachekit-py#470](https://github.com/cachekit-io/cachekit-py/pull/470), [#471](https://github.com/cachekit-io/cachekit-py/pull/471), [#472](https://github.com/cachekit-io/cachekit-py/pull/472); wheel inspected 2026-10-05)⁹ | ❌ | ✅ Opt-in `invalidation` config over a Redis pub/sub channel (`invalidation/redis-channel.ts`) | ❌ | | TTL management | ✅ Redis + SaaS + File; Memcached refresh-only (see note) | ✅ Redis + SaaS + File + Workers (`TtlInspectable`); Memcached refresh-only (LAB-429/426) | ✅ Redis + SaaS + File (`TTLBackend`); Memcached refresh-only (LAB-430) | ❌ | | Stale-while-revalidate (client L1) | ⚠️ L1-only mode (`backend=None`) only, and only with a TTL: the `production` and `dev` presets supply one by default from `cachekit` 0.20.0, a bare `@cache(backend=None)` still needs `ttl=`¹⁰ | ✅ Refresh-ahead within the L1 TTL: an entry past `swr_threshold_ratio` of its TTL is still served while one single-flight background refresh runs, and is never served after the TTL (LAB-728)¹⁰ ¹³ | ✅ `getWithSwr` — refresh-ahead within the L1 TTL: version tokens + background refresh, `maxConcurrentRefreshes` cap, never served after the TTL¹⁰; on Workers requires a bound `ExecutionContext` (see [Cache Backends](#cache-backends) note ¹) | ❌ | From 3df9312d5144f353d66767b85b2f25e00c8ca941 Mon Sep 17 00:00:00 2001 From: Ray Walker Date: Sat, 24 Oct 2026 05:05:40 +0000 Subject: [PATCH 3/3] docs(matrix): qualify async backed single-flight to a serializable result A joined caller that gets no decodable envelope takes its own trip, so a herd whose result the serializer rejects runs once per caller, and a caller whose tenant cannot be extracted shares nothing. --- sdk-feature-matrix.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sdk-feature-matrix.md b/sdk-feature-matrix.md index 7b27e55..407005d 100644 --- a/sdk-feature-matrix.md +++ b/sdk-feature-matrix.md @@ -180,7 +180,7 @@ The contract a storage backend must satisfy per SDK (bytes in / bytes out; seria | Backpressure | ✅ | ✅ Semaphore + bounded queue, decision recorded (LAB-729)¹³ | ⚠️ Refresh cap only — deliberate, decision recorded (LAB-519) | ❌ | | Distributed locking | ✅ Redis + SaaS backends | ✅ Redis + SaaS + Workers (`LockableBackend`; LAB-426) | ✅ Redis + SaaS backends; wired into `wrap()` cold miss (opt-in `stampede.distributedLock`, LAB-519) | ❌ | | L1/L2 dual-layer cache | ✅ | ✅ moka (native) / `l1` feature | ✅ | ❌ | -| Cache stampede prevention | ⚠️ In-process single-flight from `cachekit` 0.23.0+ ([cachekit-py#547](https://github.com/cachekit-io/cachekit-py/pull/547)): concurrent cold misses on one key run the function once, per event loop on the async path (with a backend, `@cache(backend=None)` or `@cache.local()`) and per process on the sync path (`@cache(backend=None)` or `@cache.local()`); an async herd with a backend makes one call's backend requests, so on a lockable backend it takes the distributed lock once. **Not the sync path with a backend:** it takes no lock and does not coalesce yet, so every concurrent miss runs the function. Verified 2026-10-24 on the published 0.23.0 and 0.24.1 wheels: each tag's `tests/unit/test_single_flight.py` passes against its own wheel, and a cold herd of 20 calls on one key ran the function once in each covered mode and 20 times on the sync path with a backend (0.22.0: 20 times in every mode) | ✅ Cold-miss single-flight wired into every `#[cachekit]` expansion — per-key in-process gate plus a distributed fill lock on lock-capable backends (the `single_flight` call in `expand`'s generated body, `cachekit-macros/src/lib.rs`; `SingleFlight::acquire` in `flight.rs`; LAB-518)¹³ | ✅ Cold-miss single-flight + SWR version tokens (LAB-519) | ❌ | +| Cache stampede prevention | ⚠️ In-process single-flight from `cachekit` 0.23.0+ ([cachekit-py#547](https://github.com/cachekit-io/cachekit-py/pull/547)): concurrent cold misses on one key run the function once, per event loop on the async path (with a backend, `@cache(backend=None)` or `@cache.local()`) and per process on the sync path (`@cache(backend=None)` or `@cache.local()`); an async herd with a backend makes one call's backend requests, so on a lockable backend it takes the distributed lock once. That backed sharing holds only for a result that serializes: a joined caller that gets no envelope it can decode takes its own trip, so a herd whose result the serializer rejects runs the function once per caller, and in multi-tenant encryption a caller whose tenant cannot be extracted shares nothing. **Not the sync path with a backend:** it takes no lock and does not coalesce yet, so every concurrent miss runs the function. Verified 2026-10-24 on the published 0.23.0 and 0.24.1 wheels: each tag's `tests/unit/test_single_flight.py` passes against its own wheel, and a cold herd of 20 calls on one key ran the function once in each covered mode, 20 times (40 GET, 20 lock) on the async path with a backend when the result does not serialize, and 20 times on the sync path with a backend (0.22.0: 20 times in every mode) | ✅ Cold-miss single-flight wired into every `#[cachekit]` expansion — per-key in-process gate plus a distributed fill lock on lock-capable backends (the `single_flight` call in `expand`'s generated body, `cachekit-macros/src/lib.rs`; `SingleFlight::acquire` in `flight.rs`; LAB-518)¹³ | ✅ Cold-miss single-flight + SWR version tokens (LAB-519) | ❌ | | Cross-instance L1 invalidation (pub/sub) | ✅ 0.22.0+ — opt-in listener (`CACHEKIT_INVALIDATION_LISTENER_ENABLED`) over a Redis pub/sub channel, publish automatic on the tenant-scoped Redis backend ([cachekit-py#470](https://github.com/cachekit-io/cachekit-py/pull/470), [#471](https://github.com/cachekit-io/cachekit-py/pull/471), [#472](https://github.com/cachekit-io/cachekit-py/pull/472); wheel inspected 2026-10-05)⁹ | ❌ | ✅ Opt-in `invalidation` config over a Redis pub/sub channel (`invalidation/redis-channel.ts`) | ❌ | | TTL management | ✅ Redis + SaaS + File; Memcached refresh-only (see note) | ✅ Redis + SaaS + File + Workers (`TtlInspectable`); Memcached refresh-only (LAB-429/426) | ✅ Redis + SaaS + File (`TTLBackend`); Memcached refresh-only (LAB-430) | ❌ | | Stale-while-revalidate (client L1) | ⚠️ L1-only mode (`backend=None`) only, and only with a TTL: the `production` and `dev` presets supply one by default from `cachekit` 0.20.0, a bare `@cache(backend=None)` still needs `ttl=`¹⁰ | ✅ Refresh-ahead within the L1 TTL: an entry past `swr_threshold_ratio` of its TTL is still served while one single-flight background refresh runs, and is never served after the TTL (LAB-728)¹⁰ ¹³ | ✅ `getWithSwr` — refresh-ahead within the L1 TTL: version tokens + background refresh, `maxConcurrentRefreshes` cap, never served after the TTL¹⁰; on Workers requires a bound `ExecutionContext` (see [Cache Backends](#cache-backends) note ¹) | ❌ |