Skip to content
355 changes: 264 additions & 91 deletions docs/build/api/gpu_cloud.md

Large diffs are not rendered by default.

17 changes: 11 additions & 6 deletions docs/build/api/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,11 @@ The Fluence API lets you do from code what you do in the Fluence Console. You ca
| Version | Paths | Use it for |
|---|---|---|
| Public v3 | `/v3/...` | New integrations: CPU virtual machines, disks, networks, public IP addresses, SSH keys, prices, account, API keys, top-ups, and promo codes |
| GPU v3 | `/v3/gpu/...` | New integrations: GPU offers, GPU containers, GPU virtual machines, and bare metal. See [GPU Cloud API](./gpu_cloud.md). |
| Public v1 and v2 | `/v1/...`, `/v2/...` | Existing integrations. These paths still work. |
| GPU | `/gpu/...` | GPU containers, GPU virtual machines, and bare metal. See [GPU Cloud API](./gpu_cloud.md). |
| Legacy GPU (deprecated) | `/gpu/...` | Existing GPU integrations. These paths still work. Move them to GPU v3. See [Legacy GPU API](./gpu_cloud.md#legacy-gpu-api). |

The GPU v3 endpoints have their own OpenAPI document: [api.fluence.dev/v3/gpu/openapi.json](https://api.fluence.dev/v3/gpu/openapi.json).

The x402 top-up and wallet sign-in endpoints use `/v2/x402/...` and `/v1/auth/siwe...`. They are not in the API reference. See [Pay with x402](./x402.md).

Expand Down Expand Up @@ -60,12 +63,13 @@ Each key has a set of permissions, called scopes, and an expiration date. The AP
| API keys | `api_key:list` | `api_key:create`, `api_key:remove` |
| Account | `account:read` | `account:write` |
| Top-ups and payments | `billing:get_history`, `billing:get_spending_history` | `billing:top_up` |
| GPU containers | | `akash:write` |
| GPU virtual machines and bare metal | | `spheron:write` |
| GPU instances and offers (`/v3/gpu`) | `gpu:read` | `gpu:write` |
| Legacy GPU containers (`/gpu`) | | `akash:write` |
| Legacy GPU virtual machines and bare metal (`/gpu`) | | `spheron:write` |

- A key can only get scopes that your account has.
- A `write` scope does not include the matching `read` scope. To create and then read a VM, a key needs both `vms:write` and `vms:read`.
- The GPU API does not accept read-only scopes. To list or read GPU instances, a key needs `akash:write` or `spheron:write`.
- A `write` scope does not include the matching `read` scope. To create and then read a VM, a key needs both `vms:write` and `vms:read`. The exception is `gpu:write`, which also allows reading `/v3/gpu`.
- A key with `akash:write` or `spheron:write` also works with `/v3/gpu`. The legacy GPU API does not accept read-only scopes: to list or read instances there, a key needs `akash:write` or `spheron:write`.
- In the Fluence Console, the **Read only** preset does not include `clusters:read`, `vm:list`, or `prices:read`. A key with only this preset cannot read the catalog or prices.

## Requests
Expand All @@ -87,6 +91,7 @@ Some v3 operations take an `Idempotency-Key` header. The key is 1 to 128 charact
- `POST /v3/vms` and `POST /v3/vms/{vm_id}/terminate` require the header.
- Creating a disk, a public IP address, a VPC, a subnet, a security group, or a draft VM's disk accepts the header.
- A different body with the same key returns `409` with `idempotency_conflict`.
- `/v3/gpu` has its own retry rules and codes. See [Retry a create request](./gpu_cloud.md#retry-a-create-request).
- A request that is still running returns `409` with `request_in_progress` and a `Retry-After` header.

Other changes, such as provisioning, restarting, editing, and creating SSH keys, API keys, or top-ups, have no `Idempotency-Key`. If the result of such a request is unknown, read the resource before you send it again. Repeating the request can, for example, restart a VM twice or create a second API key.
Expand All @@ -109,7 +114,7 @@ The API returns `2xx` on success and a JSON error body on failure:

Every v3 error body has a `code` field, such as `forbidden`, `vm_not_draft`, or `insufficient_balance`. Write your error handling against `code`. Other fields of the error body differ between operations. The API reference lists them for each operation.

The GPU endpoints return errors in a different format. See [GPU Cloud API](./gpu_cloud.md).
GPU v3 error bodies also have `code`, plus `error` and `details`. The legacy GPU API returns `detail`. See [GPU Cloud API errors](./gpu_cloud.md#errors).

## Pages in this section

Expand Down
2 changes: 1 addition & 1 deletion docs/build/api/x402.md
Original file line number Diff line number Diff line change
Expand Up @@ -198,4 +198,4 @@ For GPU containers, VMs, and bare metal servers, see the [GPU Cloud API](./gpu_c
- Terminating a VM stops billing for the VM only. Delete its disks with `DELETE /v3/storages/<storage_id>` and its public IP address with `DELETE /v3/public-ips/<public_ip_id>` to stop paying for them.
- A wallet account has no email address, so it gets no low-balance warnings. Watch `GET /v3/account/balances` and top up before the balance runs out.
- If the balance runs out and the debt exceeds 5 USD or stays unpaid for 3 days, Fluence removes your VMs. Your disks and public IP addresses stay. If the debt then reaches 20 USD or stays unpaid for 10 more days, Fluence deletes all your resources, including disks and public IP addresses.
- GPU instances are billed by the hour from the moment they start. When the balance runs out, Fluence ends them at the next hourly charge or earlier. See [GPU Cloud billing](../gpu_cloud/overview.md#billing-model).
- GPU instances are billed from the moment they start running, by the billing step of their offer, with a minimum charge. When the balance runs out, Fluence ends them within minutes. See [GPU Cloud billing](../gpu_cloud/overview.md#billing-model).
6 changes: 6 additions & 0 deletions docs/build/balance.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,12 @@ The **Top-up history** table on the **Billing** page lists your top-ups with the
| `Canceled` | You canceled the top-up |
| `Failed` | The payment did not go through |

## GPU Cloud usage

The **GPU Cloud usage** block on the **Billing** page lists your GPU resources for the selected calendar month in UTC. For each resource, it shows the name, type, hours, and cost.

The page of a GPU instance shows what the instance cost since launch. For an instance that runs across the start of a month, the two numbers differ.

## Promo codes

To activate a promo code, open the **Promo code** card on the **Billing** page, or the **Promo code** tab in the top-up dialog. Enter the code and click **Activate**.
Expand Down
80 changes: 50 additions & 30 deletions docs/build/gpu_cloud/instance_rent.md
Original file line number Diff line number Diff line change
@@ -1,53 +1,73 @@
---
description: Rent a GPU container, GPU VM, or bare metal server in the GPU Marketplace of the Fluence Console.
description: Rent a GPU container, GPU VM, or bare metal server, on demand or as a spot instance, in the GPU cloud section of the Fluence Console.
---

# Rent a GPU instance

This guide shows how to rent a GPU instance in the **GPU Marketplace** of the Fluence Console. For workload types, statuses, and billing, see [GPU Cloud concepts](./overview.md). To rent from code, see the [GPU Cloud API](../api/gpu_cloud.md).
This guide shows how to rent a GPU instance in the **GPU cloud** section of the Fluence Console. For workload types, spot instances, statuses, and billing, see [GPU Cloud concepts](./overview.md). To rent from code, see the [GPU Cloud API](../api/gpu_cloud.md).

Before you start, make sure your balance covers at least 2 hours of the instance's price. The console doesn't let you launch with less. See the [billing model](./overview.md#billing-model).
GPU Cloud is not open to every account. If the sidebar has no **GPU cloud** section, or the console shows **GPU Cloud is temporarily unavailable**, your account has no access to GPU Cloud yet. Contact support at support@fluence.ai.

## Choose the GPU and the configuration
Before you start, top up your balance. To launch, your balance must cover the minimum charge of the instance on top of what your running resources already hold. See the [billing model](./overview.md#billing-model).

These steps are the same for all workload types.
## Choose a GPU

1. Open **GPU Marketplace** and choose a tab: **Containers**, **VM**, or **Baremetal**.
2. In **Select GPU Model**, choose a row. The table shows each GPU model with its **vRAM**, **Interface** (PCIe or SXM), the lowest **Price** per hour, and a **Count** of GPUs. If a model comes with several vRAM sizes or interfaces, pick one in the row. Pick the number of GPUs in **Count**.
3. In the configuration table, choose an offer. Each row shows the **GPU**, its **Locations**, **vCPU**, **RAM**, **Storage**, and the **Price** per hour. If the offers are in several locations, filter them with **All locations**. To see only offers with NVLink between GPUs, turn on **NVLink supported**. The switch appears only when such offers exist.
4. Under **Configure deployment**, in **Basic information**, keep the generated name or enter your own.
1. In the sidebar, open **GPU cloud** > **Marketplace**. The table lists GPU models with their lowest **Price** per hour, the **GPUs per instance** you can get, the **Locations**, and the number of **Offers**.
2. To narrow the list, use the filters:
- The workload tabs: **All**, **Container**, **VM**, or **Bare metal**.
- GPU class, minimum VRAM, GPUs per instance, and country.
- **More filters**: **Max price**, **Min vCPU**, **Min RAM**, **Min local disk**, and **GPU interconnect**.
- To see spot prices too, turn on **Include spot**.
3. Click a GPU model. The row expands.
4. Choose the number of GPUs under **GPUs per instance**. If the model has spot offers, choose **On-demand** or **Spot** under **Rental**.
5. Choose an offer:
- To take the cheapest offer, click **Launch cheapest**.
- To choose a location, find the offer in the table and click **Launch** on its row. The table shows each offer's **Location**, **Workload**, **vCPU**, **RAM**, **Disk**, **Ready in**, and **Price**.

The next steps depend on the workload type.
An offer marked **Recently out of capacity** is still on sale. If the launch fails because the provider has no capacity, nothing is charged.

The console opens the **Create GPU instance** page.

## Configure the instance

## Configure a container
1. In **Instance name**, keep the generated name or enter your own: 1 to 25 lowercase letters, numbers, or hyphens.
2. In the **GPU** section, check the GPU, the number of GPUs, and the rental type. To pick another GPU, click **Change GPU**. The location field shows **Cheapest available** by default. To rent in a specific location, choose it from the list.
3. If the GPU comes in several workload types, choose **Container**, **VM**, or **Bare metal**.

The next steps depend on the workload type.

1. Under **Container image**, choose a predefined image in **Select image**, or click **Custom image** to use your own:
- In **Image**, enter the full image reference, for example `ghcr.io/owner/image:tag`.
- For an image in a private registry, turn on **Private mode**. Choose the registry, **Docker Hub - docker.io** or **GitHub container registry - ghcr.io**, and enter your **Username** and **Password** or access token.
- Click **Set image**.
2. Optional: in **Container Start Command**, enter a command that replaces the entry point of the image.
3. In **Port**, enter the ports to open, separated by commas:
- A port number, for example `443`. It opens the port for all protocols.
- A port with a protocol, for example `7090/tcp` or `53/udp`.
- A range, for example `5000-5005`.
### Configure a container

For a predefined image, the console adds port `22` for SSH. For a custom image, add every port yourself. Port `10250` is reserved. To serve HTTP on port 80, select **Enable http on port 80**.
4. Optional: to add environment variables, click **Add variable** and enter a **Name** and a **Value** for each.
5. Optional: in **SSH key (optional)**, choose one SSH key, or click **Add key** to add one. You cannot choose a key for a custom image in private mode.
1. In **Image**, choose a predefined image, or choose **Custom image** and enter the **Image reference**, for example `registry/image:tag`.
2. Optional: in **Startup command (optional)**, enter a command that replaces the entry point of the image.
3. Optional: in **SSH key**, choose a key, or click **Add SSH key**. The console passes the key to the container in the `SSH_PUBKEY` variable. Only images that read `SSH_PUBKEY` use it. Some predefined images need a key.
4. In **Exposed ports**, set the ports to open and their **Protocol**. The list offers the protocols that the location supports. Click **Add port** to add one. You cannot change the ports after launch.
5. Optional: in **Environment variables**, click **Add variable** and enter a **Name** and a **Value**. A name has uppercase letters, digits, and underscores, and starts with an uppercase letter. To hide a value after launch, select **Secret**. You cannot read a secret value back. Some locations do not accept environment variables. To set them, pick another location.
6. Optional: to pull the image from a private registry, select **Pull from a private registry** and enter the **Registry host**, **Registry username**, and **Registry password**. This option appears only where the offer supports it.

:::warning
With a predefined image, the platform uses the environment variable `SSH_PUBKEY` to pass your SSH key into the container. If you also define `SSH_PUBKEY`, the deployment fails.
The **SSH key** field sets the `SSH_PUBKEY` variable. Do not add `SSH_PUBKEY` to the environment variables yourself.
:::

## Configure a VM or a bare metal server
### Configure a VM or a bare metal server

1. In **OS image**, choose the image. The images come with GPU drivers.
2. In **SSH key**, choose the key to sign in with, or click **Add key** to add one. A VM and a bare metal server need an SSH key.
1. In **Operating system**, choose the OS image. The images come with GPU drivers.
2. In **SSH key**, choose the key to sign in with, or click **Add SSH key**. A VM and a bare metal server need an SSH key.

## Review and launch

For a container, the review panel shows the name, the location, and the image. For a VM or a bare metal server, it shows the name, the data center, and the type. It also shows the hardware: GPU, vRAM, interface, vCPU, RAM, and disk or storage. Under **Payment** it shows the **Hourly** price, the **Upfront** hours, and the **Total**: the hourly price times 2 hours.
The **Review** panel shows the instance settings: name, GPU, location, expected time to be ready, rental type, workload, image or OS, and SSH key. Below them, it shows the price:

- **Price per hour**: the hourly price of the instance.
- **Billing**: how often you are **Billed**, for example per minute, the **Minimum** time you pay for, and the amount **Reserved at launch**, which is the minimum charge.
- **If your balance runs out**: the instance ends within minutes, and its disk is erased.

If no offer matches your configuration, the panel says why and can suggest alternatives. Click **Use this** to switch to an alternative.

For a spot instance, select the confirmation checkbox. It confirms that the provider can take back the instance at any time. It also confirms that any stop ends the instance and erases its disk.

Click **Launch**. The console opens the instance page, where you can follow the status. See [Manage GPU instances](./manage_instances.md).

Click **Launch**. The console opens the **GPU instances** page, where you can follow the status. If your balance is lower than the total, the button reads **Top up balance** and opens the **Billing** page instead.
If your balance is too low, the console refuses the launch and shows a **Top up** link. Add funds, then launch again.

If the console cannot confirm that the instance was created, check the **GPU instances** page before you launch again. Otherwise you can end up with two instances.
If the console cannot confirm the launch, open **GPU cloud** > **Instances** and check your instances before you launch again. Otherwise you can end up with two instances.
Loading
Loading