Skip to content

Use Nebius Capacity Advisor for GPU offers - #4430

Open
peterschmidt85 wants to merge 4 commits into
masterfrom
pr_nebius_capacity
Open

peterschmidt85 wants to merge 4 commits into
masterfrom
pr_nebius_capacity

Conversation

@peterschmidt85

@peterschmidt85 peterschmidt85 commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Fetch GPU capacity and quota advice together from Nebius Capacity Advisor for catalog offers. Match advice by region, platform, preset, spot mode, and fabric. Missing, stale, or failed advice stays unknown.

Use per-fabric advice when creating or reusing InfiniBand clusters, preserving catalog compatibility and the configured fabric allowlist. Placement checks use cached advice without additional API calls.

Notes

  • Preserve Nebius's default AUTO reservation use. Reserved capacity keeps otherwise blocked on-demand offers eligible as unknown, since tenant-wide advice does not check project reservation limits. This uses the same API response; spot offers use only preemptible advice.
  • Fetching fresh Capacity Advisor data adds API latency (~0.7–2.7 s in local runs). Cache its capacity/quota response for 10 minutes, matching AWS's quota-cache TTL.
  • Accept an optional shared advice cache so Factory can reuse it across project backends, following its existing AWS pattern. Entries are scoped by tenant and credential IDs. Existing callers retain a private cache; Factory wiring: https://github.com/dstackai/dstack-factory/pull/22.
  • Pricing stays offline. The existing 3-minute offer cache stays unchanged. The two cache layers can retain a fetched snapshot for up to ~13 minutes.
  • Local offer-collection medians (169 offers, warm catalog): 2.3 ms before → 3.51 ms with cached advice, or 2.59 s when fetching fresh advice. Offer-cache hits remain ~0.001 ms. Initial requests also perform existing tenant/project discovery.
  • Capacity requests use the existing 10-second timeout per page. Tenant discovery is shared between project lookup and capacity collection; project selection and existing tenant retry behavior are preserved.

@peterschmidt85
peterschmidt85 marked this pull request as ready for review October 29, 2026 08:16
@peterschmidt85
peterschmidt85 requested a review from jvstme October 29, 2026 08:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant