Skip to content

feat(metrics): Add thread-local histograms - #311

Merged
TheJokr merged 2 commits into
cloudflare:mainfrom
nox:split-histogram
Nov 20, 2026
Merged

TheJokr merged 2 commits into
cloudflare:mainfrom
nox:split-histogram

Conversation

@nox

@nox nox commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

ThreadLocalHistogram keeps independent per-thread fixed-bucket state and aggregates it into one standard classic histogram during collection. The thread_local crate provides runtime-owned slots that are reused after threads exit while preserving accumulated observations.

The implementation uses boxed per-thread atomic state to avoid false sharing, and it reuses HistogramSnapshot and the existing histogram encoding path for text and protobuf rendering. HistogramBuilder constructs the metric, so metrics macros and labeled families can use it directly.

The type is exported from foundations-metrics and the backend-gated foundations facade.

Exemplar support remains deferred because sharing exemplar storage across thread-local values requires additional ownership and conflict semantics.

@gregor-cf

Copy link
Copy Markdown
Contributor

Why not use the thread_local crate? It already has the ability to aggregate/query the values from each thread. ThreadLocalCounter already uses that f8d656b
With the thread_local create you can also create a new histogram at runtime and it can work with the metrics macros.

@nox

nox commented Nov 17, 2026

Copy link
Copy Markdown
Contributor Author

@gregor-cf I wasn't aware of ThreadLocalCounter, but also I don't think we need the thread_local crate, do we? This performs an additional lookup, poll hooks are hot, and my current SplitHistogram can also work with the metrics macros (though I forgot to implement MetricConstructor<SplitHistogram> for HistogramBuilder) and can be created at runtime, you can even manually store the shards with the thread_local crate too if you want.

@nox

nox commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor Author

Tbh I don't think TLS-backed counters should use the thread_local crate either. We could either align my split histogram with ThreadLocalCounter, or we could also introduce SplitCounter, and reimplement ThreadLocalCounter for compat using SplitCounter internally.

@TheJokr

TheJokr commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

I don't think TLS-backed counters should use the thread_local crate either.

It adds an extra 2 (or 3, if you count the boxing we do for ThreadLocalCounter) pointers to chase. All of those are in cache lines that only get written if a new thread's value is initialized, so should rarely be invalidated. And the rest of your code can execute OOO while the CPU is waiting for the cache. I don't think the overhead is noticeable.

That being said, I'm not against explicitly-sharded metric types, but they are very different from ThreadLocalCounter. The latter is a 1:1 API replacement for a regular Counter, whereas explicit sharding requires the user to figure out what to do with the shards. (For example, stuffing them into static thread locals as in your example.)

@nox nox changed the title feat(metrics): Add explicitly sharded histograms feat(metrics): Add thread-local histograms Nov 18, 2026
`ClassicHistogramBounds` now owns fixed-bucket normalization, bucket
lookup, and snapshot construction. `Histogram` retains its existing
mutex-protected sum, count, and bucket counters while using the shared
immutable bounds representation.

This separates classic bucket semantics from their synchronization model
so other histogram storage strategies can reuse them without changing
rendering or public behavior.
@nox

nox commented Nov 18, 2026

Copy link
Copy Markdown
Contributor Author

@TheJokr You convinced me with the NIH free list needed with my way, so let's just use thread_local.

`ThreadLocalHistogram` keeps independent per-thread fixed-bucket state
and aggregates it into a standard classic histogram during collection.
Thread slots are reused after threads exit, while accumulated values
remain visible.

The `thread_local` crate provides runtime-owned storage, and boxed shard
state avoids false sharing between threads. `HistogramBuilder` constructs
the metric, making it compatible with metric macros and families.

The implementation reuses `ClassicHistogramBounds`, `HistogramSnapshot`,
and the existing histogram encoder. The type is available from
`foundations-metrics` and its backend-gated `foundations` facade.

Exemplar support remains deferred because sharing exemplar storage across
thread-local values requires additional ownership and conflict semantics.
@TheJokr
TheJokr merged commit c93d6bf into cloudflare:main Nov 20, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants