How it fits together
An Olympusx AI installation is a small number of parts in one namespace on your platform cluster, plus the model servers it places on the clusters you register. This page names each part by what it does and says what crosses between them. It is for the platform team planning an install and for anyone who wants to know where a request goes.
The parts
Section titled “The parts”| Part | Where it runs | What it does |
|---|---|---|
| The console | Platform namespace, served on its own host name | The web application every person signs in to. It holds no data of its own: everything it shows comes from the control plane. |
| The control plane | Platform namespace | Holds the platform’s state and makes every decision: who a caller is, what they may do, which models a tenant may call, what a budget allows, where a model runs. It also runs the work that is not a single model call: knowledge pipelines, agent runs, assistant conversations, Code Assistant indexing and reviews, Security investigations. |
| The gateway | Platform namespace, served on its own host name | The one entrance for API traffic. It authenticates the key, asks the control plane for a decision, has the request scanned by the guardrails service, routes it to the model server for that tenant and model, and records the tokens used. |
| The guardrails service | Platform namespace | Scans prompts and answers for personal data, secrets, prompt injection, toxic content and banned topics, per the tenant’s policy. Optional: your platform team turns it on. |
| The database and the object store | Platform namespace (bundled), or ones you provide | The platform’s records: tenants, keys, budgets, usage, conversations, runs, the audit log, documents and model artifacts. |
| Vector databases | A cluster you choose, one per instance | A tenant’s knowledge: the chunks and embeddings of its collections. Each instance belongs to one tenant. |
| The inference runtime | The clusters you register, in each tenant project’s namespace on that cluster | The model servers. One deployment serves one model for one tenant, on GPU or CPU nodes. |
| Your identity provider | Yours, or the built-in identity service | Signs people in and tells Olympusx AI which tenant and role each account has. |
| The bastion | Optional, on a host you choose outside the cluster | Fetches model weights and runtime images from the internet for an air-gapped platform, and lets the console reach a registered cluster’s network. See Remote access. |
What crosses between them
Section titled “What crosses between them”- A person signing in. The console sends the browser to your identity provider’s sign-in page. The provider returns a token that names the person’s tenant and role. The console passes that token to the control plane on every call, and the control plane decides each call against the role. See Identity and sign-in.
- An application calling a model. The application sends an OpenAI-style request with an Olympusx AI API key to the gateway host. Before anything reaches a model, the control plane resolves the key to its tenant and project, checks that the tenant is granted the model and that the key’s allow-list includes it, and checks the budgets. The guardrails service scans the prompt. The gateway routes the request to that tenant’s deployment of that model, or to the connected cloud provider for an external model. The answer is scanned on the way back if the tenant’s policy says so, and its token counts go to the usage record. The key never reaches the model server. See The gateway.
- A person using the console’s own features. The Playground, assistant chat, knowledge queries and agent runs are started from the console and run in the control plane. They call models directly rather than through the gateway, and apply the same policy: model grant, budgets, guardrails and metering.
- A model being deployed. A tenant admin chooses a granted model and a cluster. The control plane checks the tenant’s capacity on that cluster, then creates the model server in the project’s namespace there and tells the gateway the new route. A deployment draws on capacity, never on a token budget.
- A cluster being registered. The platform administrator gives the control plane a credential for the cluster. From then on the control plane reads its nodes and GPUs and places workloads on it. See Clusters.
- Knowledge being built. A pipeline reads documents from an upload or a connector, the document converter turns them into text, an embedding model the tenant deployed turns chunks into vectors, and the vectors land in the tenant’s vector database.
- A code host or alerting system calling in. Webhooks from GitLab, GitHub or Bitbucket and from Alertmanager or a SIEM arrive at the console host and are checked against a secret before anything acts on them. See Webhooks.
Two host names
Section titled “Two host names”An installation publishes two addresses, each with its own certificate:
| Address | Who uses it |
|---|---|
https://<your-console-host> |
People, in a browser. Also the management API under /api/, assistant share links, and inbound webhooks. |
https://<your-gateway-host> |
Applications, editors and agents calling models, with an API key. The OpenAI-compatible API is under /v1. |