You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Bug]: A new service with replicas: 0..N starts a replica on creation, without any request #4419
A replica is provisioned straight away. In our case, a 0..1 RTX 5090 service submitted at 10:12:11 UTC had a job submitted and an instance created 14 seconds later, without any request. The replica then ran until scale_down_delay passed and was scaled down (SCALED_DOWN). After that the service behaved as expected: it stayed at zero until a request arrived.
An in-place update of an existing service does not do this. Only a new run does.
Expected behaviour
A new service with replicas: 0..N stays at zero replicas until the first request, as described in #3003.
dstack version
0.22.5 (server and gateway)
Server logs
No response
Additional information
The cause appears to be the placeholder replica count combined with the pre-0.20.7 compatibility branch:
submit_run (server/services/runs/__init__.py) creates every run with desired_replica_count=1 # a relevant value will be set in RunPipeline, and leaves desired_replica_counts empty.
When the pending run is first processed, _process_pending_service (background/pipeline_tasks/runs/pending.py) calls compute_desired_replica_counts (background/pipeline_tasks/runs/common.py).
There, desired_replica_counts is empty and the service has the single default replica group. So the special case for services created by a pre-0.20.7 server applies: prev_counts = {DEFAULT_REPLICA_GROUP_NAME: run_model.desired_replica_count}, which is the placeholder 1.
The gateway has no stats for a service that was just registered. So RPSAutoscaler.get_desired_count returns current_desired_count as is (if not stats: return current_desired_count), which is 1.
total == 1, so the run is submitted with one replica.
Two possible fixes, either of which should be enough:
Apply the compatibility branch only to runs that already have jobs, for example prev_counts == {} and run_model.jobs and .... A service created by a pre-0.20.7 server had jobs by the time a 0.20.7+ server processed it, while a new run has none.
Or, for services, create the run with desired_replica_count set to the replica minimum (0 for 0..N) instead of the placeholder 1.
As hardening, the no-stats return in RPSAutoscaler.get_desired_count could also be clipped to [min_replicas, max_replicas], like the other branches.
Impact: every new scale-to-zero service costs one unrequested cold start. For a GPU service that means the full image pull, weight download and warm-up (about 20 minutes for us), plus the scale_down_delay.
I'm happy to send a PR if one of these fixes looks right to you.
Hi @vitostrokov, thanks for reporting. I believe #4433 fixes the bug. The new dstack release with the fix will most likely be available later this week, please give it a try when you have a chance
Activity
peterschmidt85 commentedon Oct 27, 2026
cc @jvstme
jvstme commentedon Oct 29, 2026
Hi @vitostrokov, thanks for reporting. I believe #4433 fixes the bug. The new
dstackrelease with the fix will most likely be available later this week, please give it a try when you have a chance