Deploy a model

Deploying a model runs it in one of your projects’ namespaces, on the cluster that namespace lives on, and grants it to your tenant so its applications, agents and assistants can call it through the gateway. Tenant admins deploy; the wizard checks that the plan fits before it lets you start.

Who can do it

Role	What they can do
Admin	Deploy and undeploy models for the tenant, read the model server’s log
Builder	Follow a deployment’s status and its steps on the model’s page
Auditor	Follow a deployment’s status and its steps on the model’s page
Reviewer	No access
End user	No access
Platform admin	Holds every capability: also deploys for any tenant, and can Analyze model and Set parameter count when the fit check lacks them

Before you start

The model is in the catalog. A platform admin adds it. See The model catalog.
Your tenant holds capacity on a cluster. A platform admin grants GPUs, vCPUs and memory to the tenant on Customers › Tenants. See Tenants.
A project has a ready namespace there. A project gets its namespace on a cluster when a platform admin sets its quota on that cluster, on Customers › Projects. See Projects.
The cluster is connected. A plan can be checked against a cluster that is not connected, but not deployed to it.

The deploy wizard

Deploy, on a catalog card, a catalog row or the model’s page, opens Deploy model, for the tenant named in its description. It has five steps. On a platform that serves models without a cluster, Hardware and Serving are left out and there is no fit check.

1. Placement

In Project (namespace), choose where the model runs. Each entry reads project (environment) · cluster; a namespace that is not ready also shows being created, failed or being removed.
Select Next.

The wizard preselects the first namespace that is ready on a connected cluster. The model runs under that project’s quota and network policies.

What this step can tell you:

This tenant has nowhere to deploy yet. The tenant holds no capacity on any cluster, or none of its projects has a namespace. A platform admin sees Go to Tenants or Go to Projects.
That namespace is not ready yet. The deployment would be refused: it is never placed in another namespace instead. Wait, or choose another project.
This cluster is not connected. The plan is still checked, but Deploy stays unavailable.

2. Hardware

Under Hardware, choose GPU or CPU. The wizard starts on CPU when the tenant holds no GPUs on that cluster.
For a GPU plan:
- GPU: the GPU type, shown as card — memory · n held. A partitioned GPU shows its profile, such as H100 1g.10gb — 10 GiB.
- GPUs: how many each replica gets.
- Tensor parallel: how many GPUs each layer is sharded across. It must divide the model’s attention head count.
For a CPU plan:
- vCPUs per replica.
- Memory (GiB) per replica. The weights and the KV cache must fit in it, as they would in a GPU’s memory.
Select Next. The wizard checks the plan again and shows any problem with these fields on this step.

CPU serving suits small models, up to a few billion parameters, and light traffic. Chat models, embedding models and rerankers can all run on CPU.

3. Serving

Replicas: how many copies of the model server run. Each replica gets its own GPUs, or its own vCPUs and memory, and the tenant’s capacity is checked for all of them. With more than one, the step shows the total.
Max context length (tokens): the largest prompt plus answer served. This is usually what decides whether a model fits.
Concurrent sequences: how many requests run at once. The KV cache is reserved per sequence.
Select Next.

4. Pre-flight and review

The step summarizes the plan (Where it runs, Hardware, per replica, Replicas, Max context length (tokens), Concurrent sequences) and gives the verdict:

This plan should run, with needed of available.
This plan will not run, with the problems to resolve.

A bar shows where the memory goes: Weights, KV cache and Overhead, against Available. Memory is checked per replica: each replica loads the whole model.

Each problem appears as an error or a warning with the field it concerns and a suggested fix. Errors block the deployment; warnings do not. The checks include:

Finding	What to change
The plan does not fit in memory	A shorter context, fewer concurrent sequences, more GPUs per replica, or a larger GPU
The fit is tight (warning)	A larger target or a shorter context
Tensor parallel does not divide the attention heads	A divisor of the head count
More GPUs required than requested	More GPUs, or less parallelism
Tensor parallel across partitioned GPU slices	A whole card, or tensor parallel 1
FP8 weights on a GPU older than the Hopper generation	A Hopper-class or newer card, or a 16-bit model
Context longer than the model was trained for (warning)	A shorter Max context length
Parallelism on a CPU plan	Tensor parallel 1; add replicas instead
Not enough capacity left in the tenant’s grant on the cluster	Fewer replicas, smaller replicas, or more capacity from a platform admin
No CPU model server is registered	A platform admin registers one before the model can run on CPU

When the model has not been analyzed, the step says This model has not been analyzed yet: its parameter count and attention shape are unknown, so no fit can be checked. A platform admin selects Analyze model, or Set parameter count when the count is unknown. A tenant admin asks a platform admin to do it.

Select Deploy. It is available only when the verdict is This plan should run, every field holds a value, the namespace is ready and the cluster is connected.

5. Deployment

The wizard follows the rollout:

Plan checked
Placed on the cluster
Model weights located
Model server started
Access granted to the tenant
Published at the gateway
Ready to serve

With several replicas, progress reads k of n replicas ready with the stage of the slowest, and the rollout finishes when all are ready.

You can close the wizard at any time: the deployment continues in the background, and Back is unavailable once it has started. The wizard ends on Deployment complete (the model is callable through the gateway) or Deployment failed. Follow it on the model’s page, with the model server’s log opens the model’s page.

Check it worked

On Build › Models, the model’s status reads Active.
On the model’s page, every condition under Conditions reads Met.
Call it through the gateway with a tenant key. See Chat completions or Embeddings and reranking.

When it did not work

What you see	What it means
The deployment did not start, on the review step	The control plane refused the plan. The message gives the reason, such as a namespace that is not ready or capacity the tenant does not hold
Failed in the catalog	A rollout step was not met. The model’s page shows which, and why
Not served in the catalog	The model is running but the gateway is not publishing it. The reason is on the row
The model server keeps restarting in the log	The log shows the last lines of the previous run, which say why it stopped

A failed deployment can be deployed again. It keeps holding its capacity until it is removed, so undeploy it before you try a different plan.

Undeploy a model

Open the model’s page, or find its row in the catalog table.
Select Undeploy.
Confirm in Undeploy model.

The model stops serving for the tenant: requests to it fail and the tenant’s grant is revoked. Its status reads Removing, then Not deployed.

Limits

Field	Default	Range
GPUs per replica	1	1 to 64
Tensor parallel	1	1 to 64, and a divisor of the attention head count
vCPUs per replica	4	1 to 128
Memory (GiB) per replica	16	1 to 1,024
Replicas	1	1 to 32
Max context length (tokens)	8,192 on GPU, 4,096 on CPU	256 to 1,000,000, in steps of 256
Concurrent sequences	32 on GPU, 8 on CPU	1 to 4,096

Replicas is a fixed count.
A CPU deployment needs the tenant to hold both vCPUs and memory on the cluster. Capacity is a total: each new plan is added to every other deployment the tenant runs on that cluster, a failed one included until it is removed.
One deployment of a model runs per tenant. Undeploy before deploying it again with a different plan.
An external model is never deployed: its provider serves it.