Skip to content

searcher: accept storageSize in M, Mi, G and Gi for cache sizing - #948

Open
marcleblanc2 wants to merge 1 commit into
mainfrom
searcher-storagesize-g-units
Open

marcleblanc2 wants to merge 1 commit into
mainfrom
searcher-storagesize-g-units

Conversation

@marcleblanc2

@marcleblanc2 marcleblanc2 commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

This PR fixes the issue for impacted customers without requiring them to make a change to their deployment, beyond upgrading.

Problem

  • The calculation of SEARCHER_CACHE_SIZE_MB and SYMBOLS_CACHE_SIZE_MB assumes the Gi suffix on searcher.storageSize
  • Any other supported unit (e.g. 150G) casts to 0, so both env vars render as "0"
  • searcher treats 0 as "never evict"
  • Both caches grow unbounded until the PVC fills and the pod fails to start:
failed to setup TMPDIR: mkdir /mnt/cache/searcher-0/.searcher.tmp/tmpfriend-...: no space left on device
  • Seen on a customer instance with searcher.storageSize: 150G after upgrading 6.5.0 → 7.2.0
  • The template has behaved this way since v6.4 (Reapply "Remove symbols statefulset" (#676) #677), so the disk had been filling for months, only noticed when checking the logs after upgrade

Fix

  • New sourcegraph.searcher.cacheSizeMB helper in templates/_helpers/_searcherCacheSize.tpl accepts a whole number of M, Mi, G or Gi and returns 45% of it in MB (treating binary units as decimal, as the old Gi * 450 math did, so existing Gi values render identically)
  • Any other value fails the render, with a pointer to searcher.autoCacheSize=true, instead of silently disabling eviction
  • Both env blocks in searcher.StatefulSet.yaml call the helper; the default 26Gi still renders 11700
  • values.yaml comment, README (helm-docs), CHANGELOG, and a new searcherCacheSize_test.yaml unittest suite

Verification

./scripts/helm-docs.sh
helm unittest charts/sourcegraph   # 32 suites, 193 tests passed
helm lint charts/sourcegraph
searcher.storageSize before after
(default 26Gi) 11700 11700
100Gi 45000 45000
150G 0 (no eviction) 67500
500000M 0 (no eviction) 225000
25Mi 0 (no eviction) 11
1Ti, 1.5G 0 (no eviction) render fails
1Ti + autoCacheSize: true env vars omitted env vars omitted

SEARCHER_CACHE_SIZE_MB and SYMBOLS_CACHE_SIZE_MB were derived with
trimSuffix "Gi" | mul 450, so a storageSize like 150G left a non-numeric
string that sprig cast to 0. searcher treats 0 as "never evict", so both
caches grew unbounded until the PVC filled and the pod failed to start with
"failed to setup TMPDIR: ... no space left on device".

Move the math into a sourcegraph.searcher.cacheSizeMB helper in its own
file under templates/_helpers/. It accepts a whole number of M, Mi, G or Gi
and fails the render on anything else, unless searcher.autoCacheSize is
enabled (the env vars are omitted in that case).

Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-01a146da-9505-715c-949c-1146cc4e7e33
@marcleblanc2
marcleblanc2 force-pushed the searcher-storagesize-g-units branch from c3ca70d to e203311 Compare October 16, 2026 22:39
@marcleblanc2 marcleblanc2 changed the title searcher: accept storageSize in G as well as Gi for cache sizing searcher: accept storageSize in M, Mi, G and Gi for cache sizing Oct 16, 2026
@github-actions

Copy link
Copy Markdown

@marcleblanc2
marcleblanc2 requested a review from a team October 16, 2026 23:11
@DaedalusG

Copy link
Copy Markdown
Contributor

I don't mind this, but it seems like it might be a little auto magic for the style we typically take with our manifests. I'd like to get a second opinion from someone else on the team, wdyt @filiphaftek ?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants