Model registry

Platform › Model serving › Model registry lists the model weights this platform holds and the imports that put them there. Olympusx AI packages weights as container images in your own registry, so any cluster that can pull from that registry serves the model without reaching the internet, and nothing is downloaded when a model starts.

Who can open it

Platform administrators only. Importing writes to a shared registry and consumes real storage, so it is platform work. Tenants see the result in the model catalog, not here.

How weights reach a cluster

Model weights are not in the platform’s images, and they are far larger than everything else in an installation. Olympusx AI delivers them as an image that holds nothing but the model files. The kubelet pulls it like any other image and mounts it beside the inference runtime, which reads the files directly.

• Nothing is downloaded when a pod starts. A restart or a reschedule does not re-fetch tens of gigabytes.
• An air-gapped cluster can serve any model you have put in your registry.
• Weights and runtime are separate images. Upgrading the serving runtime does not rebuild the model, and the reverse.
• Weights are identified by digest. A tag can be moved; the digest is what names the weights a tenant actually serves.
• They travel through your own registry, with the mirroring, pull secrets and signing you already use for the platform’s images.

The cluster that imports need not be the cluster that serves. One connected cluster can fill a registry every air-gapped cluster reads from.

Before the first import

Importing needs a registry to push to, set at installation: spec.modelServing.importRegistry on the Olympus resource, or the equivalent chart value. Without it, the page shows Importing is not configured and no import can start. See Configuration.

Gated Hugging Face repositories also need a Hugging Face token configured at installation. Settings › Platform config shows under Secrets whether one is set.

What you see

1. The header, Model registry, with “The model weights this platform holds, packaged so clusters serve them without internet access.”
2. While an import runs, a progress card for each: the model, the stage (Downloading, Packaging or Uploading), bytes done of the total, and the time left.
3. Two tabs.

Packaged weights

What the registry holds. Every cluster serves from here. The toolbar has Search models, an Origin filter and Import a model.

Column	What it shows
Model	The Hugging Face id, linking to the weights’ page.
Size	The image’s size.
Origin	Imported (packaged by Olympusx AI) or Hand-carried (packaged elsewhere and verified here).
Digest	The image digest the weights are pinned to.
Actions	Remove weights.

An empty registry shows No models packaged yet with Import a model.

Recent imports

Every import this platform has run, newest first, with Search imports and a Status filter (Succeeded, Failed).

Column	What it shows
Status	Starting, Importing now, Succeeded or Failed.
Model	The model and revision.
Ran on	Cluster or Bastion, with the cluster’s name.
Started	When it started.
Detail	Why it failed, when it did.

When the registry cannot be read, the page says The model registry could not be read: what is packaged and what is importing are both unknown, which is not the same as an empty registry.

Import a model

1. Select Import a model. The Import a model dialog opens.

2. Model. Enter the Hugging Face id as organization/name, for example meta-llama/Llama-3.1-8B-Instruct, and optionally a Revision: a commit or tag. Empty uses the default branch. When the control plane can reach Hugging Face, the model card is shown; when it cannot, the import does not need it. Select Next.

3. Where to run. Choose Run the import on: a registered cluster. It does not have to be the cluster that will serve the model. When the cluster has a bastion agent, choose Fetch the weights using:

   • The cluster itself — it needs internet access, or
   • Its bastion agent — a machine outside the cluster. Use it when the cluster’s network should not carry the download.

   A cluster declared air-gapped can only fetch through its bastion. Select Next.

4. Review. Check the model, the revision, the cluster and what fetches the weights, then select Start import.

The import runs in the background. A short-lived job, on the cluster or on the bastion, fetches the weights, layers them onto a small base image, and pushes the result to the import registry. The result is recorded by digest. You are notified when it finishes.

Starting the same model again while it imports is refused with 409 import_in_progress: the running import is not duplicated.

Once the import succeeds, the weights appear under Packaged weights. To make them deployable, the model must also be in the catalog: see The model catalog. A deployment of that model then pulls the packaged image instead of fetching from Hugging Face.

When no cluster has internet access

Without a connected cluster or a bastion, Olympusx AI gives you the commands for the transfer across the air gap and records the result:

1. Ask the API for the commands, as a platform administrator: GET /api/v1/platform/model-artifacts/instructions?repoId=<org/name>&revision=<revision>. It returns the exact commands for a machine that has access, with pinned tools, and the image reference to push to.
2. Run them on that machine, and carry the image into your network by whatever means your policy allows.
3. Push it to the reference the instructions named.
4. Ask Olympusx AI to verify it: POST /api/v1/platform/model-artifacts/verify with {"repoId": "<org/name>", "revision": "<revision>", "clusterId": "<cluster id>"}. A job on that cluster inspects the pushed image and records its digest and size. It appears under Packaged weights as Hand-carried.

Both calls need the import registry configured (412 registry_not_configured without it). See Air-gapped installation for the rest of an air-gapped setup.

A model’s page

Select a model under Packaged weights.

• Overview: Model details with the Image reference, Digest, Size, Origin, Revision, Imported on and Imported by, and whether the model is In the catalog or “Not in the catalog, so no tenant can deploy it yet”. The Model card follows when Hugging Face can be reached; otherwise No model card available, which does not affect deploying.
• Used by: Deployments serving these weights, in every tenant, with the tenant and cluster of each.

Remove weights

1. On Packaged weights, select Remove weights in the row, or open the model and select Remove.
2. Confirm Remove model weights.

The platform stops offering these weights. The image stays in your registry: deleting it there follows your registry’s own retention policy.

Removal is refused with 409 artifact_in_use while any deployment serves the weights, because the deployment would fail on its next restart. Undeploy it first; the Used by tab lists them.

Sizing

An image is roughly the size of the weights.

Model	Approximate image
7–8B parameters, 16-bit	15–17 GB
8B parameters, 4-bit quantized	5–6 GB
70B parameters, 16-bit	130–140 GB

• Node disk. Every node that runs a model pulls its image once and caches it. Size node disk for the models that will run there.
• Import scratch space. An import stages the weights before layering them, so it briefly needs about twice the model’s size, on the cluster or on the bastion.
• Registry storage grows by the image size per model and revision.

Note

The images live in your registry; protect them with its own replication and retention. See Backup and restore.

Related

• Remote access for attaching a bastion agent
• Clusters for declaring and measuring a cluster’s internet access
• Runtimes for the serving images
• Deploy a model