Skip to content

test: run Redis-backed tests on a fake clock - #1162

Merged
alexluong merged 12 commits into
mainfrom
test/fake-clock-redis
Oct 29, 2026
Merged

alexluong merged 12 commits into
mainfrom
test/fake-clock-redis

Conversation

@alexluong

@alexluong alexluong commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Run the scheduler, idempotence and infra tests on a fake clock, with an in-memory Redis, so they stop waiting on real timers.

These tests waited out real visibility timeouts, lock delays and TTLs and asserted on wall-clock windows, which made them slow and sensitive to CPU load. testing/synctest runs a test in a bubble with a fake clock: time.Sleep advances the fake clock instead of waiting, and synctest.Wait() returns once every goroutine in the bubble is blocked. Redis kept these tests out of a bubble: miniredis needed a TCP listener, and a bubble can't do network I/O. miniredis v2.40.0 adds Dial() over an in-memory pipe and ClockTTL() (TTLs follow the fake clock).

  1. miniredis v2.38.0 → v2.40.0.
  2. RedisConfig.Dialer in internal/redis: optional. When set, the regular client dials through it and maintenance notifications are off (they look up the address themselves, bypassing the dialer). Nil by default; yaml/env go through config.RedisConfig, which doesn't carry it. The cluster client ignores it.
  3. testutil.NewSynctestRedis(t): a miniredis, a config whose Dialer reaches it, and a client from redis.New, usable inside a bubble.
  4. scheduler, idempotence and infra tests converted.
  5. contributing/test.md: a short "Timing in tests" section, with a guideline to keep each package under 2 s in go test -short -p 1 ./....

Each of these packages now runs in under 1 s. Integration tests (TESTINFRA) stay on real time.

Notes:

  • Assertions are kept; several are now exact (fake-clock instants instead of ranges).
  • TestScheduler_VisibilityTimeout: context 4 s → 3.5 s. The 4th receive lands at exactly 4 s, so the old deadline raced it, in real time too.
  • TestInfra_LockExpiry now also asserts the lock key expired and Declare didn't hit its 5 s retry; on a fake clock the retry alone would let it pass.
  • Scheduler tests call synctest.Wait() before cancel(): Monitor returns before in-flight handlers finish, and a handler still running when the test ends fails the bubble.

Each converted package was stress-tested with repeated runs, plain and with -race.

Not in this PR: Monitor waiting for in-flight handlers on shutdown.

🤖 Generated with Claude Code

@leggetter leggetter left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Claude review notes (from Phil's Claude Code session, non-blocking)

These are agent-generated notes, not Phil's own review.

  • Dialer (internal/redis/config.go, internal/redis/redis.go): when it's set, go-redis uses it in place of its default dialer, and go-redis only applies TLSConfig inside that default dialer (v9.22.0 options.go:494 and :617-626). So with a custom Dialer, TLSEnabled is silently ignored. Only tests set it today, so there's no user impact. Worth adding "bypasses TLS" to the field comment so nobody wires it up in production later?
  • Monitor shutdown (internal/scheduler/scheduler.go on main, not changed here): Monitor returns as soon as ctx is cancelled, without waiting for the handler goroutines it started, and those handlers run on the same cancelled ctx. The new startMonitor helper works around this in tests by letting handlers finish before cancelling. Is that intended for production shutdown, or worth a follow-up? I haven't traced what the service does after Monitor returns, so the impact is unverified.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alexluong

Copy link
Copy Markdown
Collaborator Author

Thanks! Updated the Dialer comment to say TLSEnabled doesn't apply when it's set.

On Monitor shutdown: a cut-off handler leaves its message in rsmq and it's received again after the visibility timeout, so nothing is lost, but the task can run twice. I'll open a separate PR so Monitor waits for running handlers on shutdown, like internal/consumer does.

@alexluong
alexluong merged commit 8a775ec into main Oct 29, 2026
1 check passed
@alexluong
alexluong deleted the test/fake-clock-redis branch October 29, 2026 12:21
@alexluong

Copy link
Copy Markdown
Collaborator Author

Opened #1185 for the Monitor drain.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants