diff --git a/docs/build/api/gpu_cloud.md b/docs/build/api/gpu_cloud.md index 9cc9193..c9e84f4 100644 --- a/docs/build/api/gpu_cloud.md +++ b/docs/build/api/gpu_cloud.md @@ -1,156 +1,329 @@ --- sidebar_position: 4 -description: "GPU Cloud API: list GPU plans, deploy and manage GPU containers, VMs, and bare metal servers, with permissions, rate limits, and errors." +description: "GPU Cloud API v3: find GPU offers and prices, rent GPU containers, VMs, and bare metal on demand or as spot, and manage instances, with permissions, retries, and errors." --- # GPU Cloud API -The GPU Cloud API lets you list GPU plans and deploy GPU containers, VMs, and bare metal servers. The same API manages them after deployment. For workload types, statuses, and billing, see [GPU Cloud concepts](../gpu_cloud/overview.md). +The GPU Cloud API finds GPU offers, prices them, and rents GPU containers, VMs, and bare metal servers. The same API manages the instances after you rent them. For workload types, spot instances, statuses, and billing, see [GPU Cloud concepts](../gpu_cloud/overview.md). -- Base URL: `https://api.fluence.dev/gpu` -- OpenAPI document: [api.fluence.dev/gpu/api-docs/openapi.json](https://api.fluence.dev/gpu/api-docs/openapi.json) -- Interactive reference: [api.fluence.dev/gpu/api-docs](https://api.fluence.dev/gpu/api-docs) +- Base URL: `https://api.fluence.dev`. Every path on this page starts with `/v3/gpu`. +- OpenAPI document: [api.fluence.dev/v3/gpu/openapi.json](https://api.fluence.dev/v3/gpu/openapi.json) -The GPU API is a separate service. It has no v3 version, and its requests, responses, and errors differ from the rest of the [Fluence API](./overview.md). +The GPU endpoints have their own OpenAPI document. Their error body has `code`, `error`, and `details`, which differs from the rest of the [Fluence API](./overview.md). See [Errors](#errors). ## Authentication and permissions -Send an API key in the `X-API-KEY` header, or an access token in `Authorization: Bearer`. The default images endpoint needs no credentials. +Send an API key in the `X-API-KEY` header, or an access token in `Authorization: Bearer`. See [Authentication](./overview.md#authentication). -| Workload | Scope | +| To | Scope | |---|---| -| Containers | `akash:write` | -| VMs and bare metal servers | `spheron:write` | +| List images, read offers, get quotes, and read your instances | `gpu:read` | +| Create, change, and delete instances, and everything that `gpu:read` allows | `gpu:write` | -The GPU API does not accept read-only scopes. Every GPU endpoint except the default images list needs `akash:write` or `spheron:write`, including the endpoints that list and read instances. The scopes `akash:read` and `spheron:read` exist, but the GPU API does not use them. A key with the **Read only** preset of the Fluence Console cannot use the GPU API. +When you create a key in the Fluence Console, give it **View & manage** access to **GPU containers** or **GPU VMs & bare metal**. See [Settings](../settings.md). These options give `akash:write` and `spheron:write`, the scopes of the [legacy GPU API](#legacy-gpu-api). A key with either scope also works with `/v3/gpu`, for reading and for changes. + +`GET /v3/gpu/offers` and `GET /v3/gpu/offers/{id}` also work without credentials. Without credentials, the list shows only offers that you can rent now. The OpenAPI document needs no credentials either. Every other endpoint returns `401` with `unauthenticated` without credentials. ## Endpoints -### Plans +### Offers | Method | Path | What it does | |---|---|---| -| `GET` | `/gpu/plans/` | Lists container plans | -| `GET` | `/gpu/plans/vms` | Lists VM plans | -| `GET` | `/gpu/plans/baremetal` | Lists bare metal plans | - -Keep the trailing slash in `/gpu/plans/`, `/gpu/instances/`, and `/gpu/default-images/`. Without it, the API answers with a redirect to an `http://` URL. +| `GET` | `/v3/gpu/offers` | Lists offers: one placement each, with hardware, price, and availability | +| `GET` | `/v3/gpu/offers/{id}` | Returns one offer | +| `GET` | `/v3/gpu/offer-groups` | Groups offers by GPU configuration, with the lowest price of each group | +| `GET` | `/v3/gpu/offer-facets` | Lists the values you can filter by, such as GPU models, countries, and providers, with counts | +| `GET` | `/v3/gpu/offer-picks` | Returns the cheapest offers with at least the GPU memory you need | +| `POST` | `/v3/gpu/instance-quotes` | Shows which offer a request would get and at what price, without renting | -### Containers +### Instances | Method | Path | What it does | |---|---|---| -| `POST` | `/gpu/instances/` | Deploys a container | -| `GET` | `/gpu/instances/` | Lists containers | -| `GET` | `/gpu/instances/{instance_id}` | Returns a container | -| `PUT` | `/gpu/instances/{instance_id}` | Updates a container | -| `DELETE` | `/gpu/instances/{instance_id}` | Terminates a container | -| `GET` | `/gpu/instances/{instance_id}/logs` | Returns the container output | -| `GET` | `/gpu/instances/{instance_id}/events` | Returns the cluster events of the container | - -### VMs and bare metal servers +| `POST` | `/v3/gpu/instances` | Creates an instance | +| `GET` | `/v3/gpu/instances` | Lists your instances | +| `GET` | `/v3/gpu/instances/{id}` | Returns an instance | +| `GET` | `/v3/gpu/instances/by-idempotency-key` | Finds an instance by the `Idempotency-Key` it was created with | +| `PATCH` | `/v3/gpu/instances/{id}` | Renames an instance or changes the container of a running one | +| `DELETE` | `/v3/gpu/instances/{id}` | Ends an instance | +| `GET` | `/v3/gpu/instances/{id}/logs` | Returns the last lines of the instance output | +| `GET` | `/v3/gpu/instances/{id}/events` | Returns recent provider events of the instance | -VMs and bare metal servers use the same requests and responses. For bare metal, replace `vms` with `baremetal` in the path. +### Images and the API document | Method | Path | What it does | |---|---|---| -| `POST` | `/gpu/instances/vms` | Deploys a VM | -| `GET` | `/gpu/instances/vms` | Lists VMs | -| `GET` | `/gpu/instances/vms/{instance_id}` | Returns a VM | -| `PUT` | `/gpu/instances/vms/{instance_id}` | Renames a VM | -| `DELETE` | `/gpu/instances/vms/{instance_id}` | Terminates a VM | +| `GET` | `/v3/gpu/default-images` | Lists predefined container images with their ports, environment variables, and SSH user | +| `GET` | `/v3/gpu/openapi.json` | Returns the OpenAPI document of `/v3/gpu` | -### Images +The OpenAPI document also has volume endpoints. Volumes are not available yet. See [Volumes](#volumes). -| Method | Path | What it does | -|---|---|---| -| `GET` | `/gpu/default-images/` | Lists the predefined container images | +## Find an offer + +An offer is one placement that you can rent. It names a provider, a region, a GPU model and count, the rest of the machine, a rental type, and a price. Each offer has a `workload`: `container`, `vm`, or `baremetal`. + +Choose the call that matches what you know: + +- You know how much GPU memory you need: call `GET /v3/gpu/offer-picks` with `workload` and `minVramGib`, the memory of each GPU in GiB. It returns up to 10 of the cheapest offers, 3 by default. Each item has `requirements` that you pass to the quote and to the create request unchanged. If no offer fits, `items` is empty, `rejected` says why the other offers do not fit, and `largestOnSale` gives the largest GPU memory and GPU count on sale. +- You want to compare GPU models: call `GET /v3/gpu/offer-groups`. Each group has its lowest price and `requirements` for the quote. To create from a group, also set `offerId` to the group's `cheapestOfferId`, or set `maxPricePerHour`. +- You want to see every placement: call `GET /v3/gpu/offers`. + +`GET /v3/gpu/offers`, `GET /v3/gpu/offer-groups`, and `GET /v3/gpu/offer-facets` take the same filters: + +| Filter | What it matches | +|---|---| +| `workload` | `container`, `vm`, or `baremetal` | +| `rentalType` | `on_demand` or `spot`. Without it, only on-demand offers are listed. | +| `gpuVendor`, `gpuModel`, `gpuInterface` | The GPU as offers name it, for example `nvidia`, `h100`, `sxm`. Case does not matter. | +| `gpuVramGib`, `minGpuVramGib` | Memory of each GPU in GiB, exact or at least | +| `gpuCount`, `minGpuCount` | GPUs per machine, exact or at least | +| `minVcpu`, `minMemoryGib`, `minStorageGib` | Machine size, at least | +| `country`, `regionLabel`, `provider` | Location. `country` is a two-letter ISO code. `provider` takes several names separated by commas. | +| `hostTier` | `dedicated`, `shared`, or `unknown` | +| `cpuArchitecture` | `x86_64`, `arm64`, or `unknown` | +| `maxPricePerHour` | The highest price per instance-hour in USD, as a decimal string, for example `0.50` | +| `available` | `false` also lists offers without capacity left. Without credentials, only offers on sale are listed. | +| `q` | Text in the GPU vendor, GPU model, provider, or region label | + +To sort, set `sort` to `price`, `vram`, `vcpu`, `memory`, `storage`, or `availability`, and `order` to `asc` or `desc`. Without `sort`, the cheapest offers come first. + +Name GPU models exactly as offers name them, for example `rtx4090` or `h100`. `GET /v3/gpu/offer-facets` lists the models on sale. A filter with an unknown model returns an empty list. In a quote or a create request, an unknown model in `requirements.gpu.model` returns `invalid_input`. + +```sh +curl "https://api.fluence.dev/v3/gpu/offers?workload=vm&rentalType=spot&minGpuVramGib=80" +``` + +### Read an offer + +Besides the hardware and the price, an offer has these fields: + +- `offerKind`: `provider_sku` is an item of a provider catalog, and `host_offer` is one host. `market_profile` is an Akash offer that fixes the hardware profile and the highest bid, but not the host. Akash picks the host from bids when you create the instance. +- `hostTier`: `dedicated`, `shared`, or `unknown`. +- `region`, `dataCenter`, and `origin`: where the machine runs. +- `topology`: GPU links, such as NVLink. +- `provisioning`: how long the instance usually takes to be ready, in `typicalSeconds`, `upperSeconds`, or both. It can be `null`. +- `availability`: `status` is `available`, `limited`, `sold_out`, or `unknown`. `lastOutOfStockAt` is the last time the provider refused this offer for lack of capacity. Such an offer stays on sale, and the next launch can succeed. +- `capabilities`: what an instance of the offer supports. `logs`, `events`, and `update` say whether you can read logs and events and change the container. `portProtocols`, `portRequired`, `environment`, and `privateRegistry` limit the container settings. See [Create the instance](#create-the-instance). +- `osImages`: the OS images that a VM or bare metal offer accepts. An empty list means the provider takes any name, and a wrong name fails only at the provider. +- `preemption`: only for spot offers. `noticeSeconds` is the warning the provider gives before it takes the machine back, and `0` means no warning. `dataSurvives` says whether the disk survives. `signal` is `explicit` when the provider reports the take-back, and `inferred` when Fluence detects it from the machine stopping. +- `groupKey`: the offer group of the offer, as in `GET /v3/gpu/offer-groups`. + +### Offer prices + +Every offer has a `price`. Here is the price of an 8 × RTX 4090 VM offer from `GET /v3/gpu/offers`: + +```json +{ + "currency": "USD", + "unit": "instance", + "perHour": "4.2240", + "gpuHour": "0.5280", + "stepSeconds": 60, + "perStep": "0.0704", + "minimumPeriodSeconds": 1200, + "minimumCharge": "1.4080", + "components": [{ "kind": "gpu", "perHour": "4.2240" }] +} +``` + +- `perHour` is the full price of the instance per hour in USD. `gpuHour` is the same price per GPU, so you can compare offers with different GPU counts. +- `stepSeconds` is the billing step, and `perStep` is the price of one step. +- `minimumCharge` is the least you pay for an instance that starts running. It covers `minimumPeriodSeconds` of running time. + +Amounts are decimal strings. See [Billing model](../gpu_cloud/overview.md#billing-model). + +## Rent an instance + +### Get a quote + +Send `POST /v3/gpu/instance-quotes` with the same `requirements` and workload settings that you plan to create. The quote returns: + +- `match`: the offer that the create request would select, with its price. `null` if no offer fits. +- `alternatives`: up to 3 other offers that fit. +- `rejected`: why other offers do not fit, for example `price_ceiling`, `out_of_stock`, or `port_required`, with the number of offers for each reason. + +The quote also checks the image, the ports, and the environment variables. A quote holds nothing, and prices can change before you create the instance. + +### Create the instance + +Send `POST /v3/gpu/instances` with an `Idempotency-Key` header. The body has: + +- `name`: 1 to 25 characters, lowercase letters, digits, and hyphens. +- `requirements`: + - `workload`: `container`, `vm`, or `baremetal`. + - `gpu`: `vendor`, `model`, and `count`, as offers name them. + - `offerId` to rent one exact offer, or `maxPricePerHour` to let the API choose the cheapest matching offer at or below this price. One of them is required. With `offerId`, the other requirements must still fit the offer. + - `rentalType`: `on_demand` or `spot`. On-demand is the default. + - Optional limits, such as `minVcpu`, `minMemoryGib`, `minStorageGib`, `region`, `providers`, `hostTier`, and `cpuArchitecture`. +- Exactly one of `container`, `vm`, or `baremetal`, matching the workload. + +For a container: -## Choose a plan +- `image` is required: an image reference, as for `docker pull`. `GET /v3/gpu/default-images` lists predefined images with the ports and environment variables they expect. +- `expose` lists the ports to open, each with `port` and `protocol`: `tcp`, `udp`, or `http`. The offer's `capabilities.portProtocols` lists the protocols it accepts. Where `capabilities.portRequired` is `true`, add at least one port. You cannot change the ports after creation. +- `environment` holds up to 64 variables, where the offer's `capabilities.environment` is `true`. Other offers refuse every variable except `SSH_PUBKEY`. A name starts with an uppercase letter and has uppercase letters, digits, and underscores. Set `secret: true` to hide a value. The API never returns secret values. +- `startupCommand` replaces the entry point of the image. +- `registryCredentials` with `host`, `username`, and `password` pulls an image from a private registry, where the offer's `capabilities.privateRegistry` is `true`. +- `sshKey` passes a public SSH key to the container in the `SSH_PUBKEY` variable. The predefined images read it. If you also define `SSH_PUBKEY` in `environment`, the API refuses the request. -- Container plans come as a flat list. Each plan groups offers that can run in different locations. To deploy, pass the plan ID and, optionally, a location. -- VM and bare metal plans are grouped by GPU model and then by GPU count. Each group has options, each in one location. To deploy, pass the option ID as `plan_id`. Each option lists `os_options`. Pass one of them as `os_image`. +For a VM or a bare metal server, `osImage` and `sshKey` are required. If the offer lists `osImages`, `osImage` must be one of them. -## Deploy an instance +```json +{ + "name": "train-1", + "requirements": { + "workload": "vm", + "rentalType": "on_demand", + "gpu": { "vendor": "nvidia", "model": "rtx4090", "count": 8 }, + "offerId": "" + }, + "vm": { + "osImage": "", + "sshKey": "ssh-ed25519 AAAA... user@host" + } +} +``` + +Your balance must cover the minimum charge of the new instance on top of the amount that your running resources already hold. The quote shows the minimum charge in `match.price.minimumCharge`. If the balance is too low, the API returns `insufficient_balance` with the required and the available amounts. -Your balance must cover at least 1 hour of the instance's price. Otherwise the API refuses the request. +The API answers `201` with the new instance. A refused request returns an error, for example `insufficient_balance`, `no_offer_matches`, or `fleet_too_large`. See [Errors](#errors). -### Deploy a container +### Retry a create request -Send `POST /gpu/instances/` with at least a plan ID, an instance name, and container settings. The settings need an image and at least one exposed port. You can also pass environment variables, a start command, private registry credentials, a location, and an SSH key. +The `Idempotency-Key` is 1 to 128 characters: letters, digits, `.`, `_`, `:`, and `-`. -- Only port 80 can use the `http` protocol. Other ports use `tcp` or `udp`. -- A container can have up to 10 exposed ports and 64 environment variables. -- `SSH_PUBKEY` is reserved for the SSH key. If you pass an SSH key and also define `SSH_PUBKEY`, the deployment fails. -- Private registries can be `docker.io` or `ghcr.io`. -- `GET /gpu/default-images/` lists the predefined images. +- The same key with the same body returns the same instance with `200`. It does not create a second one. +- The same key with a different body returns `idempotency_key_reused`. +- While the first request is still running, a repeat returns `request_in_progress` with a `Retry-After` header. +- A refused create stays refused for its key. To try again, for example after a top-up, use a new key. -### Deploy a VM or a bare metal server +If the response is lost, send `GET /v3/gpu/instances/by-idempotency-key` with the same `Idempotency-Key` header. It returns the instance if the request created one, and `404` with `not_found` if it did not. Do not put the key in the URL. -Send `POST /gpu/instances/vms` or `POST /gpu/instances/baremetal` with the option ID as `plan_id`, an instance name, an SSH key, and an `os_image`. If `os_image` is not in the `os_options` of the option, the error lists the valid values. +### Wait for the instance -### After the request +Read `GET /v3/gpu/instances/{id}` every few seconds until `status` is `running`, or `isTerminal` is `true`. -Provisioning usually takes 2 to 3 minutes. Read the instance until its status is `ACTIVE`. Then its connection details are in the response: `ssh_connection` for a VM or a bare metal server, and `network.domain` with `network.forwarded_ports` for a container. +- When the instance is `running`, `connection` tells you how to reach it. For a VM or a bare metal server, `connection.sshCommand`, when present, is the ready SSH command. Otherwise use `connection.host`, `connection.port`, and `connection.user`. For a container, `connection.ports` maps each exposed port to an `externalPort` on `connection.host`. +- When the instance is `failed` or `ended`, `statusDetail`, `failure.code`, and `failure.reason` say why. -If the response to a deploy request is lost, list your instances before you send it again. A repeated request can create a second instance. +If the instance provisions longer than the offer's `provisioning.upperSeconds`, read its events. + +### Instance fields + +An instance has the fields of its request, and these fields: + +- `status`: see [Instance statuses](../gpu_cloud/overview.md#instance-statuses). +- `statusDetail`: why the instance is in its status. Some values: + - `awaiting_bids`: Akash collects bids for the instance. + - `provider_rate_limited`: the provider limits how often instances are created. The launch continues by itself, and nothing is billed before the instance runs. + - `out_of_stock`: the provider had no capacity for the offer. Nothing is billed, and another offer can succeed. + - `user_requested`: you deleted the instance. + - `preempted`: the provider took a spot instance back. `failure.code` is then `provider_terminated`, and `failure.reason` is `preempted`. + - `provider_terminated`: the provider ended the instance. +- `isBilling`: `true` while the instance is billed. +- `isTerminal`: `true` for `ended` and `failed`. +- `placement`: the offer that the instance runs on, with its provider, region, hardware, and capabilities. +- `price`: the price fixed when the instance was created. +- `charged`: the amount charged so far in `total`, as of `asOf`. +- `actions`: what you can do with the instance now: `rename`, `delete`, `update`, `logs`, and `events`. +- `createdVia`: `v3`, or `legacy` for instances created through the [legacy GPU API](#legacy-gpu-api). +- `createdAt`, `activatedAt`, and `endedAt`: when the instance was created, started running, and ended. ## Manage instances -### Update a container +### List instances -`PUT /gpu/instances/{instance_id}` replaces the whole container configuration. Send every field, not only the ones you change. You cannot change the exposed ports. The ports from the deployment stay. The API refuses an update while a previous update is still being applied. +`GET /v3/gpu/instances` lists every instance you created, including ended and failed ones. Instances that you created through the [legacy GPU API](#legacy-gpu-api) are in the list too, with `createdVia: "legacy"`. These filters narrow the list: -### Rename a VM or a bare metal server +- `live=true` lists only instances that are not ended or failed. `live=false` lists only ended and failed ones. +- `status` takes one status or several, separated by commas. +- `workload`, `rentalType`, `provider`, `country`, and `createdVia` match those fields. +- `q` matches text in the instance ID, name, or GPU model. -`PUT` on a VM or bare metal instance changes only its name. To change anything else, terminate the instance and deploy a new one. +To sort, set `sort` to `createdAt`, `name`, `status`, `pricePerHour`, or `activatedAt`. The newest instances come first by default. -### Debug a container +### Rename an instance or change a container -- `GET /gpu/instances/{instance_id}/logs` returns the container output. Add `?last=N` to get the last `N` lines. -- `GET /gpu/instances/{instance_id}/events` returns cluster events, such as scheduling, image pulls, and container starts. Read them when a container doesn't start. +`PATCH /v3/gpu/instances/{id}` changes: -### Terminate an instance +- `name`, at any time until the instance starts terminating. +- For a running container, if the offer supports updates: `container.image`, `container.environment`, `container.startupCommand`, and `container.registryCredentials`. Omitted fields stay as they are. `container.environment` replaces the whole list of variables. To keep a secret value, list its name with `secret: true` and no value. -Send `DELETE` to the instance path. You can terminate only an instance with the status `ACTIVE`. Billing stops, and the unused part of the current hour is not returned. +A container change answers `200` with `status: updating`. The instance returns to `running` when the provider applies the change. The same change sent again while it is applied returns the instance. A different change returns `update_in_progress`. -## Rate limits +Nothing else changes after creation: not the offer, price, workload, or ports, and not the OS image or SSH key of a VM or bare metal server. To change them, create a new instance and delete the old one. -The API limits requests per API key, or per IP address for requests without a key. +### Read logs and events -| Requests | Per second | Per minute | -|---|---|---| -| `GET` | 60 | 100 | -| `POST`, `PUT`, `DELETE` | 30 | 50 | +`GET /v3/gpu/instances/{id}/logs` and `GET /v3/gpu/instances/{id}/events` work when the instance's `actions` list `logs` or `events`. Otherwise, the API returns `409` with `unsupported_by_provider` or `instance_state_conflict`. Some providers return logs and events while the instance is still provisioning, so you can see why a workload does not start. -Every response has the headers `X-RateLimit-Limit`, `X-RateLimit-Remaining`, and `X-RateLimit-Reset`. Over the limit, the API returns `429` with a `Retry-After` header. +Add `tail=N` to get the last `N` lines. The provider can return fewer. -## Errors +### Delete an instance -Errors have a `detail` field. +`DELETE /v3/gpu/instances/{id}` ends the instance. It answers `202` when it starts closing the instance, and `200` if the instance is already closing or ended, so you can repeat it. Billing stops when you delete the instance: `isBilling` becomes `false` and `endedAt` is set, while `status` can still read `terminating` for a short while. The local disk of the instance and its data are deleted. -Validation errors return `422`, and `detail` is a list with one object per invalid field: +After you delete an instance, read it until `isTerminal` is `true`. After an instance ends, `charged.total` can read `0` or grow for a few minutes. It is final when it is above `0` and does not change between two reads a minute apart. -```json -{ - "detail": [ - { - "type": "value_error", - "loc": ["body", "container_settings", "expose", 0, "port"], - "msg": "Value error, Port must be between 1 and 65535", - "input": "999999999", - "ctx": { "error": {} } - } - ] -} -``` +## Volumes + +:::warning +Volumes are not available yet. The volume API may change before it opens. +::: + +A volume is block storage that outlives instances. The OpenAPI document has the volume endpoints under `/v3/gpu/volume-offers`, `/v3/gpu/volume-quotes`, and `/v3/gpu/volumes`. Until volumes open, `GET /v3/gpu/volume-offers` returns an empty list and you cannot create a volume. Offers show `capabilities.volumes: 0`. + +## Errors -Other errors return `detail` as a string, for example: +Every error has a `code`, an `error` message, and a `details` list: ```json -{ - "detail": "Invalid API key" -} +{ "code": "unauthenticated", "error": "Authentication is required.", "details": [] } ``` -A `503` means that the authentication service is unavailable, or that deploying new instances is turned off for a while. Retry later. +Write your error handling against `code`. These codes are common: + +| Code | Meaning | +|---|---| +| `invalid_input` | A field or query parameter is invalid. A `field` detail names it. | +| `unauthenticated`, `api_key_expired` | The credentials are missing, invalid, or expired | +| `insufficient_scope` | The key lacks `gpu:read` or `gpu:write` | +| `instance_not_found`, `offer_not_found` | No such instance in your account, or no such offer on sale | +| `no_offer_matches` | No offer fits the requirements. `selection` details show how many offers each reason excluded. | +| `insufficient_balance` | Your balance does not cover the minimum charge of this instance. A `balance` detail has `required` and `available` amounts in USD, and the `window` in seconds that the amounts are counted for. | +| `fleet_too_large` | You have reached the limit of live instances on your account | +| `idempotency_key_reused`, `request_in_progress` | See [Retry a create request](#retry-a-create-request) | +| `update_in_progress`, `instance_state_conflict` | The instance is busy or in a status that does not allow the action | +| `unsupported_by_provider` | The provider of this instance does not support the action | +| `rate_limited` | Too many requests. Wait for the number of seconds in `Retry-After`. | +| `creation_disabled`, `provider_unavailable` | Creating instances is turned off for a while, or the provider is unavailable. Retry later. | + +The full list is in the `ErrorBody` schema of the OpenAPI document. A code that is not in the list is newer than the document: act on the HTTP status. The same applies to other enumerations, such as `status`, `statusDetail`, and `offerKind`: do not reject a value that you do not know. + +Every response has an `X-Request-Id` header. Quote it when you contact support. You can send your own `X-Request-Id`, 1 to 64 letters, digits, and hyphens, and the API returns it. + +Lists take `page` and `perPage` (up to 200, 50 by default) and return `items` and `pagination`. + +## Legacy GPU API + +:::warning +The legacy GPU API is deprecated. Use `/v3/gpu` for new integrations, and move existing integrations to it. +::: + +The earlier GPU API under `https://api.fluence.dev/gpu/...` still works for existing integrations, and its paths and responses do not change. New instances rented through the legacy API are billed the same way as `/v3/gpu` instances: by the billing step and minimum of their offer. + +The legacy API needs the scope `akash:write` for containers, or `spheron:write` for VMs and bare metal, on the endpoints below except `GET /gpu/default-images/`. The endpoints under `/gpu/billing/` need valid credentials only. The legacy API has these endpoints for instances: + +| Method | Path | What it does | +|---|---|---| +| `GET` | `/gpu/plans/`, `/gpu/plans/vms`, `/gpu/plans/baremetal` | Lists container, VM, and bare metal plans | +| `POST`, `GET` | `/gpu/instances/` | Deploys or lists containers | +| `GET`, `PUT`, `DELETE` | `/gpu/instances/{instance_id}` | Returns, updates, or terminates a container | +| `GET` | `/gpu/instances/{instance_id}/logs`, `/gpu/instances/{instance_id}/events` | Returns the output or the events of a container | +| `POST`, `GET` | `/gpu/instances/vms`, `/gpu/instances/baremetal` | Deploys or lists VMs or bare metal servers | +| `GET`, `PUT`, `DELETE` | `/gpu/instances/vms/{instance_id}`, `/gpu/instances/baremetal/{instance_id}` | Returns, renames, or terminates a VM or a bare metal server | +| `GET` | `/gpu/default-images/` | Lists the predefined container images | + +Legacy errors have a `detail` field instead of `code`. diff --git a/docs/build/api/overview.md b/docs/build/api/overview.md index 49f0522..ad319eb 100644 --- a/docs/build/api/overview.md +++ b/docs/build/api/overview.md @@ -16,8 +16,11 @@ The Fluence API lets you do from code what you do in the Fluence Console. You ca | Version | Paths | Use it for | |---|---|---| | Public v3 | `/v3/...` | New integrations: CPU virtual machines, disks, networks, public IP addresses, SSH keys, prices, account, API keys, top-ups, and promo codes | +| GPU v3 | `/v3/gpu/...` | New integrations: GPU offers, GPU containers, GPU virtual machines, and bare metal. See [GPU Cloud API](./gpu_cloud.md). | | Public v1 and v2 | `/v1/...`, `/v2/...` | Existing integrations. These paths still work. | -| GPU | `/gpu/...` | GPU containers, GPU virtual machines, and bare metal. See [GPU Cloud API](./gpu_cloud.md). | +| Legacy GPU (deprecated) | `/gpu/...` | Existing GPU integrations. These paths still work. Move them to GPU v3. See [Legacy GPU API](./gpu_cloud.md#legacy-gpu-api). | + +The GPU v3 endpoints have their own OpenAPI document: [api.fluence.dev/v3/gpu/openapi.json](https://api.fluence.dev/v3/gpu/openapi.json). The x402 top-up and wallet sign-in endpoints use `/v2/x402/...` and `/v1/auth/siwe...`. They are not in the API reference. See [Pay with x402](./x402.md). @@ -60,12 +63,13 @@ Each key has a set of permissions, called scopes, and an expiration date. The AP | API keys | `api_key:list` | `api_key:create`, `api_key:remove` | | Account | `account:read` | `account:write` | | Top-ups and payments | `billing:get_history`, `billing:get_spending_history` | `billing:top_up` | -| GPU containers | | `akash:write` | -| GPU virtual machines and bare metal | | `spheron:write` | +| GPU instances and offers (`/v3/gpu`) | `gpu:read` | `gpu:write` | +| Legacy GPU containers (`/gpu`) | | `akash:write` | +| Legacy GPU virtual machines and bare metal (`/gpu`) | | `spheron:write` | - A key can only get scopes that your account has. -- A `write` scope does not include the matching `read` scope. To create and then read a VM, a key needs both `vms:write` and `vms:read`. -- The GPU API does not accept read-only scopes. To list or read GPU instances, a key needs `akash:write` or `spheron:write`. +- A `write` scope does not include the matching `read` scope. To create and then read a VM, a key needs both `vms:write` and `vms:read`. The exception is `gpu:write`, which also allows reading `/v3/gpu`. +- A key with `akash:write` or `spheron:write` also works with `/v3/gpu`. The legacy GPU API does not accept read-only scopes: to list or read instances there, a key needs `akash:write` or `spheron:write`. - In the Fluence Console, the **Read only** preset does not include `clusters:read`, `vm:list`, or `prices:read`. A key with only this preset cannot read the catalog or prices. ## Requests @@ -87,6 +91,7 @@ Some v3 operations take an `Idempotency-Key` header. The key is 1 to 128 charact - `POST /v3/vms` and `POST /v3/vms/{vm_id}/terminate` require the header. - Creating a disk, a public IP address, a VPC, a subnet, a security group, or a draft VM's disk accepts the header. - A different body with the same key returns `409` with `idempotency_conflict`. +- `/v3/gpu` has its own retry rules and codes. See [Retry a create request](./gpu_cloud.md#retry-a-create-request). - A request that is still running returns `409` with `request_in_progress` and a `Retry-After` header. Other changes, such as provisioning, restarting, editing, and creating SSH keys, API keys, or top-ups, have no `Idempotency-Key`. If the result of such a request is unknown, read the resource before you send it again. Repeating the request can, for example, restart a VM twice or create a second API key. @@ -109,7 +114,7 @@ The API returns `2xx` on success and a JSON error body on failure: Every v3 error body has a `code` field, such as `forbidden`, `vm_not_draft`, or `insufficient_balance`. Write your error handling against `code`. Other fields of the error body differ between operations. The API reference lists them for each operation. -The GPU endpoints return errors in a different format. See [GPU Cloud API](./gpu_cloud.md). +GPU v3 error bodies also have `code`, plus `error` and `details`. The legacy GPU API returns `detail`. See [GPU Cloud API errors](./gpu_cloud.md#errors). ## Pages in this section diff --git a/docs/build/api/x402.md b/docs/build/api/x402.md index e404c51..0610f37 100644 --- a/docs/build/api/x402.md +++ b/docs/build/api/x402.md @@ -198,4 +198,4 @@ For GPU containers, VMs, and bare metal servers, see the [GPU Cloud API](./gpu_c - Terminating a VM stops billing for the VM only. Delete its disks with `DELETE /v3/storages/` and its public IP address with `DELETE /v3/public-ips/` to stop paying for them. - A wallet account has no email address, so it gets no low-balance warnings. Watch `GET /v3/account/balances` and top up before the balance runs out. - If the balance runs out and the debt exceeds 5 USD or stays unpaid for 3 days, Fluence removes your VMs. Your disks and public IP addresses stay. If the debt then reaches 20 USD or stays unpaid for 10 more days, Fluence deletes all your resources, including disks and public IP addresses. -- GPU instances are billed by the hour from the moment they start. When the balance runs out, Fluence ends them at the next hourly charge or earlier. See [GPU Cloud billing](../gpu_cloud/overview.md#billing-model). +- GPU instances are billed from the moment they start running, by the billing step of their offer, with a minimum charge. When the balance runs out, Fluence ends them within minutes. See [GPU Cloud billing](../gpu_cloud/overview.md#billing-model). diff --git a/docs/build/balance.md b/docs/build/balance.md index f609caf..5db368a 100644 --- a/docs/build/balance.md +++ b/docs/build/balance.md @@ -71,6 +71,12 @@ The **Top-up history** table on the **Billing** page lists your top-ups with the | `Canceled` | You canceled the top-up | | `Failed` | The payment did not go through | +## GPU Cloud usage + +The **GPU Cloud usage** block on the **Billing** page lists your GPU resources for the selected calendar month in UTC. For each resource, it shows the name, type, hours, and cost. + +The page of a GPU instance shows what the instance cost since launch. For an instance that runs across the start of a month, the two numbers differ. + ## Promo codes To activate a promo code, open the **Promo code** card on the **Billing** page, or the **Promo code** tab in the top-up dialog. Enter the code and click **Activate**. diff --git a/docs/build/gpu_cloud/instance_rent.md b/docs/build/gpu_cloud/instance_rent.md index 14442b8..3f184c0 100644 --- a/docs/build/gpu_cloud/instance_rent.md +++ b/docs/build/gpu_cloud/instance_rent.md @@ -1,53 +1,73 @@ --- -description: Rent a GPU container, GPU VM, or bare metal server in the GPU Marketplace of the Fluence Console. +description: Rent a GPU container, GPU VM, or bare metal server, on demand or as a spot instance, in the GPU cloud section of the Fluence Console. --- # Rent a GPU instance -This guide shows how to rent a GPU instance in the **GPU Marketplace** of the Fluence Console. For workload types, statuses, and billing, see [GPU Cloud concepts](./overview.md). To rent from code, see the [GPU Cloud API](../api/gpu_cloud.md). +This guide shows how to rent a GPU instance in the **GPU cloud** section of the Fluence Console. For workload types, spot instances, statuses, and billing, see [GPU Cloud concepts](./overview.md). To rent from code, see the [GPU Cloud API](../api/gpu_cloud.md). -Before you start, make sure your balance covers at least 2 hours of the instance's price. The console doesn't let you launch with less. See the [billing model](./overview.md#billing-model). +GPU Cloud is not open to every account. If the sidebar has no **GPU cloud** section, or the console shows **GPU Cloud is temporarily unavailable**, your account has no access to GPU Cloud yet. Contact support at support@fluence.ai. -## Choose the GPU and the configuration +Before you start, top up your balance. To launch, your balance must cover the minimum charge of the instance on top of what your running resources already hold. See the [billing model](./overview.md#billing-model). -These steps are the same for all workload types. +## Choose a GPU -1. Open **GPU Marketplace** and choose a tab: **Containers**, **VM**, or **Baremetal**. -2. In **Select GPU Model**, choose a row. The table shows each GPU model with its **vRAM**, **Interface** (PCIe or SXM), the lowest **Price** per hour, and a **Count** of GPUs. If a model comes with several vRAM sizes or interfaces, pick one in the row. Pick the number of GPUs in **Count**. -3. In the configuration table, choose an offer. Each row shows the **GPU**, its **Locations**, **vCPU**, **RAM**, **Storage**, and the **Price** per hour. If the offers are in several locations, filter them with **All locations**. To see only offers with NVLink between GPUs, turn on **NVLink supported**. The switch appears only when such offers exist. -4. Under **Configure deployment**, in **Basic information**, keep the generated name or enter your own. +1. In the sidebar, open **GPU cloud** > **Marketplace**. The table lists GPU models with their lowest **Price** per hour, the **GPUs per instance** you can get, the **Locations**, and the number of **Offers**. +2. To narrow the list, use the filters: + - The workload tabs: **All**, **Container**, **VM**, or **Bare metal**. + - GPU class, minimum VRAM, GPUs per instance, and country. + - **More filters**: **Max price**, **Min vCPU**, **Min RAM**, **Min local disk**, and **GPU interconnect**. + - To see spot prices too, turn on **Include spot**. +3. Click a GPU model. The row expands. +4. Choose the number of GPUs under **GPUs per instance**. If the model has spot offers, choose **On-demand** or **Spot** under **Rental**. +5. Choose an offer: + - To take the cheapest offer, click **Launch cheapest**. + - To choose a location, find the offer in the table and click **Launch** on its row. The table shows each offer's **Location**, **Workload**, **vCPU**, **RAM**, **Disk**, **Ready in**, and **Price**. -The next steps depend on the workload type. +An offer marked **Recently out of capacity** is still on sale. If the launch fails because the provider has no capacity, nothing is charged. + +The console opens the **Create GPU instance** page. + +## Configure the instance -## Configure a container +1. In **Instance name**, keep the generated name or enter your own: 1 to 25 lowercase letters, numbers, or hyphens. +2. In the **GPU** section, check the GPU, the number of GPUs, and the rental type. To pick another GPU, click **Change GPU**. The location field shows **Cheapest available** by default. To rent in a specific location, choose it from the list. +3. If the GPU comes in several workload types, choose **Container**, **VM**, or **Bare metal**. + +The next steps depend on the workload type. -1. Under **Container image**, choose a predefined image in **Select image**, or click **Custom image** to use your own: - - In **Image**, enter the full image reference, for example `ghcr.io/owner/image:tag`. - - For an image in a private registry, turn on **Private mode**. Choose the registry, **Docker Hub - docker.io** or **GitHub container registry - ghcr.io**, and enter your **Username** and **Password** or access token. - - Click **Set image**. -2. Optional: in **Container Start Command**, enter a command that replaces the entry point of the image. -3. In **Port**, enter the ports to open, separated by commas: - - A port number, for example `443`. It opens the port for all protocols. - - A port with a protocol, for example `7090/tcp` or `53/udp`. - - A range, for example `5000-5005`. +### Configure a container - For a predefined image, the console adds port `22` for SSH. For a custom image, add every port yourself. Port `10250` is reserved. To serve HTTP on port 80, select **Enable http on port 80**. -4. Optional: to add environment variables, click **Add variable** and enter a **Name** and a **Value** for each. -5. Optional: in **SSH key (optional)**, choose one SSH key, or click **Add key** to add one. You cannot choose a key for a custom image in private mode. +1. In **Image**, choose a predefined image, or choose **Custom image** and enter the **Image reference**, for example `registry/image:tag`. +2. Optional: in **Startup command (optional)**, enter a command that replaces the entry point of the image. +3. Optional: in **SSH key**, choose a key, or click **Add SSH key**. The console passes the key to the container in the `SSH_PUBKEY` variable. Only images that read `SSH_PUBKEY` use it. Some predefined images need a key. +4. In **Exposed ports**, set the ports to open and their **Protocol**. The list offers the protocols that the location supports. Click **Add port** to add one. You cannot change the ports after launch. +5. Optional: in **Environment variables**, click **Add variable** and enter a **Name** and a **Value**. A name has uppercase letters, digits, and underscores, and starts with an uppercase letter. To hide a value after launch, select **Secret**. You cannot read a secret value back. Some locations do not accept environment variables. To set them, pick another location. +6. Optional: to pull the image from a private registry, select **Pull from a private registry** and enter the **Registry host**, **Registry username**, and **Registry password**. This option appears only where the offer supports it. :::warning -With a predefined image, the platform uses the environment variable `SSH_PUBKEY` to pass your SSH key into the container. If you also define `SSH_PUBKEY`, the deployment fails. +The **SSH key** field sets the `SSH_PUBKEY` variable. Do not add `SSH_PUBKEY` to the environment variables yourself. ::: -## Configure a VM or a bare metal server +### Configure a VM or a bare metal server -1. In **OS image**, choose the image. The images come with GPU drivers. -2. In **SSH key**, choose the key to sign in with, or click **Add key** to add one. A VM and a bare metal server need an SSH key. +1. In **Operating system**, choose the OS image. The images come with GPU drivers. +2. In **SSH key**, choose the key to sign in with, or click **Add SSH key**. A VM and a bare metal server need an SSH key. ## Review and launch -For a container, the review panel shows the name, the location, and the image. For a VM or a bare metal server, it shows the name, the data center, and the type. It also shows the hardware: GPU, vRAM, interface, vCPU, RAM, and disk or storage. Under **Payment** it shows the **Hourly** price, the **Upfront** hours, and the **Total**: the hourly price times 2 hours. +The **Review** panel shows the instance settings: name, GPU, location, expected time to be ready, rental type, workload, image or OS, and SSH key. Below them, it shows the price: + +- **Price per hour**: the hourly price of the instance. +- **Billing**: how often you are **Billed**, for example per minute, the **Minimum** time you pay for, and the amount **Reserved at launch**, which is the minimum charge. +- **If your balance runs out**: the instance ends within minutes, and its disk is erased. + +If no offer matches your configuration, the panel says why and can suggest alternatives. Click **Use this** to switch to an alternative. + +For a spot instance, select the confirmation checkbox. It confirms that the provider can take back the instance at any time. It also confirms that any stop ends the instance and erases its disk. + +Click **Launch**. The console opens the instance page, where you can follow the status. See [Manage GPU instances](./manage_instances.md). -Click **Launch**. The console opens the **GPU instances** page, where you can follow the status. If your balance is lower than the total, the button reads **Top up balance** and opens the **Billing** page instead. +If your balance is too low, the console refuses the launch and shows a **Top up** link. Add funds, then launch again. -If the console cannot confirm that the instance was created, check the **GPU instances** page before you launch again. Otherwise you can end up with two instances. +If the console cannot confirm the launch, open **GPU cloud** > **Instances** and check your instances before you launch again. Otherwise you can end up with two instances. diff --git a/docs/build/gpu_cloud/manage_instances.md b/docs/build/gpu_cloud/manage_instances.md index c2da433..b5bdd58 100644 --- a/docs/build/gpu_cloud/manage_instances.md +++ b/docs/build/gpu_cloud/manage_instances.md @@ -1,57 +1,54 @@ --- -description: View, connect to, update, and terminate GPU containers, VMs, and bare metal servers on the GPU instances page of the Fluence Console. +description: View, connect to, rename, update, and terminate GPU containers, VMs, and bare metal servers in the GPU cloud section of the Fluence Console. --- # Manage GPU instances -You manage rented GPU instances on the **GPU instances** page of the Fluence Console. It has a tab for each type: **Container**, **VM**, and **Bare Metal**. To do the same from code, see the [GPU Cloud API](../api/gpu_cloud.md). +You manage rented GPU instances in the Fluence Console under **GPU cloud** > **Instances**. To do the same from code, see the [GPU Cloud API](../api/gpu_cloud.md). ## Instance list -The list shows each instance with its **Location**, **Configuration** (GPU, vRAM, interface, vCPU, RAM, and disk), **Domain**, and **Status**. To find an instance, filter the list by name, or by status: Running, Initialized, Provisioning, Ended, or Failed. +The list shows each instance with its **Name** and location, **GPU**, **Workload**, **Rental**, **Price** per hour, **Created at**, and **Status**. -The buttons on a row depend on the instance: +By default, the list shows active instances only. Ended and failed instances are hidden. To see them, set the status filter to **All**, or click **Show all**. You can also filter by name, status, workload, and rental type. -- The arrow opens the instance page. Every instance has it. -- The trash button terminates the instance. Only running instances have it. -- The pencil renames the instance. Only running containers have it. +The statuses are **Pending**, **Provisioning**, **Running**, **Updating**, **Terminating**, **Ended**, and **Failed**. See [Instance statuses](./overview.md#instance-statuses). Hover over a status to see more detail, for example **Reserving your balance** or **Looking for capacity**. A spot instance that the provider took back shows **Reclaimed**. -## Instance page +A running instance can show a warning that it is still billed: -The instance page refreshes every 30 seconds. It shows: +- **Powered off — still billed**: the VM was powered off from inside. It is still billed, and its disk is kept. Terminate it to stop the charges. +- **Container not running — still billed**: the container stopped, or it did not start after an update. Update the image to start it again, or terminate the instance to stop the charges. -- **Configuration**: GPU, vRAM, interface, vCPU, RAM, and disk. -- **Network**: - - For a container: the **Domain**, the **Forwarded ports**, and the **URI** if port 80 serves HTTP. The platform maps each of your ports to an external port on the domain. For example, `22:30694` means that you connect to port `30694` of the domain to reach port `22` of the container. - - For a VM or a bare metal server: the **Domain**, the **SSH key**, and an **SSH command** that you can copy. -- The image of the instance. The heading reads **Container image** for every type. -- **Renting information**: **Deployment Date**, **Next billing time**, **Hourly usage**, and **Total spendings**. See the [billing model](./overview.md#billing-model). +The list and the instance page refresh by themselves: every 5 seconds while an instance is changing status, and every 30 seconds otherwise. -## Containers +## Instance page -### Logs and events +Click an instance to open its page. The **Overview** tab has three parts: -The **Logs** tab shows the output of the container. The **Events** tab shows cluster events, such as scheduling, image pulls, and container starts. Read the events when a container doesn't start. +- Configuration: **Instance ID**, **Offer ID**, GPU, VRAM per GPU, vCPU, RAM, **Local disk**, location, workload, image or OS, command, SSH key, and environment variables. Secret values show as **Hidden**. +- Connection: the **Host**, and for each port the external address in the form `host:port`, which you can copy. For a VM, a bare metal server, or a container with SSH, the page shows a ready SSH command. Connection details appear once the instance is running. +- Billing: **Created**, **Started**, and **Ended** times, the **Price** per hour, how often the instance is **Billed**, the **Minimum charge**, and **Charged so far**. After the instance ends, the last value reads **Total charged**. -### Update a running container +While an instance provisions, the page shows how long it usually takes to be ready. -To change a running container, click **Update deployment** on its page. You can change: +When an instance ends or fails, the page explains why and what you were charged. To rent the same configuration again, click **Launch again**. The console copies the settings into a new instance, except secret values, which you enter again. -- The container image, predefined or custom. -- The container start command. -- The environment variables. -- The SSH key. +### Logs and events -The ports cannot be changed after launch. The fields work as in [Configure a container](./instance_rent.md#configure-a-container). Click **Redeploy** and confirm. The container restarts with the new settings. +The **Logs** and **Events** tabs appear while the instance is running, if its provider offers them. **Logs** shows the output of the container. **Events** shows provider events, such as scheduling, image pulls, and container starts. Click **Refresh** to load new lines. -### Rename a container +## Change an instance -Click the pencil next to the name of a running container. +Open **Manage instance** on the instance page. The menu shows only the actions that the instance supports. -## VMs and bare metal servers +- **Rename**: enter a new **Name** and click **Confirm**. +- **Update image**: for a running container, enter a new **Container image** and click **Update**. The container restarts with the new image. It may be unavailable during the restart. -You cannot change a running VM or bare metal server in the console. To use other settings, terminate the instance and rent a new one. The API can rename these instances. See the [GPU Cloud API](../api/gpu_cloud.md). +In the console, you can change only the name and the image. The [GPU Cloud API](../api/gpu_cloud.md#rename-an-instance-or-change-a-container) also changes the startup command, environment variables, and registry credentials of a container. To change anything else, terminate the instance and launch a new one. ## Terminate an instance -Click the trash button on the instance page or in the list, and confirm with **Terminate**. The status changes to Ended, and billing stops. The unused part of the current hour is not returned. +1. Open **Manage instance** and click **Terminate**. +2. Type the instance name to confirm, and click **Terminate**. + +The instance ends, and its disk is erased. Billing stops when you terminate the instance, even if its status still shows **Terminating** for a short while. diff --git a/docs/build/gpu_cloud/overview.md b/docs/build/gpu_cloud/overview.md index 124a26a..e4785f5 100644 --- a/docs/build/gpu_cloud/overview.md +++ b/docs/build/gpu_cloud/overview.md @@ -1,44 +1,75 @@ --- sidebar_position: 0 -description: "GPU Cloud concepts: containers, GPU VMs and bare metal, instance statuses, and how hourly GPU billing works." +description: "GPU Cloud concepts: offers and providers, containers, GPU VMs and bare metal, on-demand and spot instances, instance statuses, and per-minute GPU billing." --- # GPU Cloud concepts -Fluence GPU Cloud rents GPUs from providers in data centers around the world. You can run three types of GPU workloads: containers, VMs, and bare metal servers. In the Fluence Console you rent them in the **GPU Marketplace** and manage them on the **GPU instances** page. To do the same from code, use the [GPU Cloud API](../api/gpu_cloud.md). +Fluence GPU Cloud rents GPUs from several providers in data centers around the world. You can run three types of GPU workloads: containers, VMs, and bare metal servers, on demand or as spot instances. In the Fluence Console you rent them under **GPU cloud** > **Marketplace** and manage them under **GPU cloud** > **Instances**. To do the same from code, use the [GPU Cloud API](../api/gpu_cloud.md). + +## Offers + +An offer is one machine configuration that you can rent: a provider, a region, a GPU model and count, vCPU, memory, local storage, and a price. Each offer is for one workload type and one rental type. Offers also show: + +- Availability. An offer that recently ran out of capacity can stay on sale. If a launch fails because the provider has no capacity, nothing is charged. +- An estimate of how long provisioning takes. +- What the instance supports, for example logs, events, container updates, and private registries. + +Most offers are one machine type of a provider. Some Akash offers fix only the hardware and the highest price: Akash picks the host from provider bids when you launch the instance. + +Prices and availability change as providers change them. ## Workload types | | Container | VM | Bare metal | |---|---|---|---| -| What it is | A container on a shared host with full access to its GPUs | A virtual machine with GPU passthrough | A dedicated physical server | +| What it is | A container with access to the GPUs of its host | A virtual machine with GPU passthrough | A dedicated physical server | | Image | A container image | An OS image | An OS image | -| Connection | A domain with forwarded ports | SSH to a public IP address | SSH to a public IP address | -| What you can change after launch | Image, start command, environment variables, and SSH key | Name, through the API only | Name, through the API only | +| Connection | A host with forwarded ports | SSH | SSH | +| What you can change after launch | Name. If the offer supports updates: image, startup command, environment variables, and registry credentials. The console changes only the image. | Name | Name | + +A container runs an image that you choose, with an optional startup command, environment variables, and ports. The platform maps each of your ports to an external port. For example, your port `22` can become port `30694` on the host. -A container runs a Docker image that you choose, with an optional start command, environment variables, and ports. The platform gives the container a domain and maps each of your ports to an external port. For example, your port `22` can become port `30694` on the domain. +A VM and a bare metal server both run an OS image with GPU drivers, and you connect to them over SSH. A VM runs on a hypervisor. A bare metal server is a physical machine with no virtualization layer. -A VM and a bare metal server both run an OS image with GPU drivers, and you connect to them over SSH. A VM runs on a hypervisor. A bare metal server is a physical machine with no virtualization layer. The console and the API rent both the same way. +## On-demand and spot instances + +An on-demand instance runs until you terminate it, your balance runs out, or the provider ends it. The instance page in the console and `statusDetail` in the API give the reason. + +If an instance does not become ready in time after launch, Fluence stops the launch, and the instance is `failed`. A running instance that stops being ready keeps running and is billed until you delete it, unless the provider ends it. + +A spot instance usually costs less. It runs on capacity that the provider can take back at any time, without warning. A spot instance has one life: + +- When the provider takes the machine back, the instance ends. +- Any stop ends a spot instance, including a shutdown from inside the machine. +- When a spot instance ends, its disk and data are deleted. + +Use spot instances for work that can stop and start again, and keep your data outside the instance. ## Instance statuses -| API status | Console label | Meaning | -|---|---|---| -| `INITIALIZED` | Initialized or Provisioning | The instance is being provisioned. For a container, the console shows the provisioning step, from 1/4 to 4/4. | -| `ACTIVE` | Running | The instance is running. Its connection details are available. | -| `FAILED` | Failed | Provisioning failed. Nothing was charged. | -| `ENDED` | Ended | You terminated the instance, or the platform ended it | +| Status | Meaning | +|---|---| +| `pending` | Fluence is reserving funds on your balance | +| `provisioning` | The provider is creating the instance | +| `running` | The instance is running, and its connection details are available | +| `updating` | The provider is applying a container change | +| `terminating` | The provider is closing the instance | +| `ended` | The instance ended, for example because you terminated it, the provider took back a spot machine, or your balance ran out. The API gives the reason in `statusDetail`, and the console on the instance page. | +| `failed` | The instance was not created, or it did not survive creation | -Provisioning usually takes 2 to 3 minutes. +`ended` and `failed` are final. A final instance stays in your list, so you can see its history and its total charge. ## Billing model -GPU instances are billed by the hour in USD, from your account balance. +GPU instances are billed in USD from your account balance, for the time they run. + +- Billing starts when the instance becomes `running`. A launch that fails before that is not billed. +- Each offer has a billing step and a minimum period. Most offers bill per minute with a minimum of 20 minutes. Some offers bill per hour with a minimum of 1 hour. The offer and the instance show their terms. +- An instance that starts running costs at least the minimum charge: the price for the minimum period. After the minimum period, you pay per billing step. +- Billing stops when you terminate the instance, or when it ends for another reason. The instance can still show `terminating` for a short while. If the instance lived less than the minimum period, the minimum charge applies. +- After an instance ends, its total charge can read 0 or grow for a few minutes. It is final when it is above 0 and does not change between two reads a minute apart. -- Billing starts when the instance becomes Running. At that moment Fluence charges the first hour. -- After that, Fluence charges one hour at a time, every hour counted from the moment the instance started. For example, an instance that started at 11:13 is charged at 11:13, 12:13, 13:13, and so on. **Next billing time** on the instance page shows the next charge. -- Every started hour is charged in full. When an instance ends, the unused part of the hour is not returned. -- Nothing is reserved or held on the instance. A failed instance is not charged. -- To launch an instance in the console, your balance must cover the first 2 hours. The console shows this as **Upfront** on the review panel. Through the API, the balance must cover 1 hour. +To launch an instance, your balance must cover its minimum charge on top of the amount that your running resources already hold. -If your balance cannot cover the next hour, Fluence still charges the hour, and your balance goes below zero. At the following charge, Fluence ends the instance. Fluence also ends all your GPU instances as soon as your balance reaches zero. GPU instances do not get the grace period that CPU resources get. To keep instances running, keep enough balance. See [Payment and balance management](../balance.md). +When your balance runs out, Fluence ends your GPU instances within minutes. Their disks and data are deleted. Keep enough balance to keep instances running. See [Payment and balance management](../balance.md). diff --git a/docs/build/overview.md b/docs/build/overview.md index 9d2ab3f..bc6680c 100644 --- a/docs/build/overview.md +++ b/docs/build/overview.md @@ -14,7 +14,7 @@ Fluence Cloud rents GPU and CPU compute from data centers around the world. You | Product | What you get | Billing | Guides | |---|---|---|---| -| GPU Cloud | GPU containers, GPU virtual machines, and bare metal servers | By the hour | [GPU Cloud concepts](./gpu_cloud/overview.md) | +| GPU Cloud | GPU containers, GPU virtual machines, and bare metal servers, on demand or as spot instances | Per minute or per hour, depending on the offer, with a minimum charge | [GPU Cloud concepts](./gpu_cloud/overview.md) | | CPU Cloud | Virtual machines with disks, public IP addresses, private networks, and security groups | By the second | [CPU Cloud concepts](./cpu_cloud/overview.md) | To reserve GPU clusters for a longer term, Fluence runs [GPU Cluster Auctions](https://auctions.fluence.ai), where providers bid for your request. These docs do not cover auctions. @@ -26,12 +26,11 @@ The sidebar of the Fluence Console has these sections: | Section | What it is for | |---|---| | **Public cloud** | CPU virtual machines on the **Compute** page, disks on **Storage**, and public IP addresses, VPCs, and security groups on **Network** | -| **GPU Marketplace** | Renting GPU containers, VMs, and bare metal servers | -| **GPU instances** | Managing the GPU instances you rented | +| **GPU cloud** | Renting GPU containers, VMs, and bare metal servers on the **Marketplace** page, and managing them on the **Instances** page | | **Billing** | Your balance, top-ups, promo codes, and usage. The balance in the page header also links here. | | **Settings** | Your profile, API keys, SSH keys, and the **MCP** page with configurations for AI clients | | **Support** | Opens an email to support@fluence.ai | ## Pay for compute -A new account can rent GPU and CPU compute right after registration. Rent is charged from your prepaid balance in USD, so top it up first. You can top it up by card or in crypto in the console. An AI agent can also top up over HTTP with USDC through [x402](./api/x402.md). See [Payment and balance management](./balance.md). +A new account can rent CPU compute right after registration. GPU Cloud is not open to every account: if the console shows **GPU Cloud is temporarily unavailable**, contact support. Rent is charged from your prepaid balance in USD, so top it up first. You can top it up by card or in crypto in the console. An AI agent can also top up over HTTP with USDC through [x402](./api/x402.md). See [Payment and balance management](./balance.md). diff --git a/docs/build/registration.md b/docs/build/registration.md index 52d5415..0a4ebcd 100644 --- a/docs/build/registration.md +++ b/docs/build/registration.md @@ -15,7 +15,7 @@ The Fluence Console uses one page to sign in and to create an account. After the first sign-in, the console asks for your name, last name, job role, and company. You can fill them in or click **Skip**. You can change them later in **Settings** > **Account**. -After you register, you can rent GPU and CPU compute. Rent is charged from a balance in USD, so top it up first. See [Payment and balance management](./balance.md). +After you register, you can rent CPU compute. GPU Cloud is not open to every account. See [Rent a GPU instance](./gpu_cloud/instance_rent.md). Rent is charged from a balance in USD, so top it up first. See [Payment and balance management](./balance.md). :::tip An AI agent or a script doesn't need to register. [Paying with x402](./api/x402.md) creates an account for the paying wallet.