Chat completions

POST /v1/chat/completions sends a conversation to a chat model and returns its answer. It follows the OpenAI Chat Completions API. This page covers what Olympusx AI adds to it and the cases where its behaviour differs.

Request

POST https://<your-gateway-host>/v1/chat/completions
Authorization: Bearer <api-key>
Content-Type: application/json

Field	Required	Notes
model	Yes	The model’s name as the catalog shows it, and as GET /v1/models lists it. The tenant must be granted it, a deployment of it must be serving (or a cloud provider must serve it), and the key’s allow-list, if any, must include it.
messages	Yes	The conversation, as in the OpenAI API.
stream	No	true for server-sent events. See Streaming.
tools, tool_choice	No	Function calling. The model must support tool calls; the catalog and GET /v1/models say whether it does (supports_tools).
user	No	An opaque reference to the person this call is for. See The user field.
Other parameters	No	temperature, top_p, max_tokens, stop, response_format, seed and the rest are passed to the model’s runtime.

The prompt and the answer together must fit the model’s context window, which GET /v1/models reports as context_window.

Examples

curl

curl -s https://<your-gateway-host>/v1/chat/completions \
  -H "Authorization: Bearer $OLYMPUS_API_KEY" \
  -H 'content-type: application/json' \
  -d '{
    "model": "<model-name>",
    "messages": [
      {"role": "system", "content": "You answer in one short paragraph."},
      {"role": "user", "content": "What does a project quota limit?"}
    ],
    "user": "cust-4821"
  }'

Python, with the OpenAI SDK

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://<your-gateway-host>/v1",
    api_key=os.environ["OLYMPUS_API_KEY"],
)

reply = client.chat.completions.create(
    model="<model-name>",
    messages=[{"role": "user", "content": "What does a project quota limit?"}],
    user="cust-4821",
)
print(reply.choices[0].message.content)
print(reply.usage.total_tokens)

Response

The model’s runtime returns the OpenAI response object: id, object, created, model, choices with each message and finish_reason, and usage with prompt_tokens, completion_tokens and total_tokens. The token counts in usage are the counts Olympusx AI meters.

If the tenant’s guardrail policy is in Enforce mode and checks answers, personal data in the answer may arrive redacted.

Streaming

With "stream": true the answer arrives as server-sent events, one data: line per chunk, ending with data: [DONE].

stream = client.chat.completions.create(
    model="<model-name>",
    messages=[{"role": "user", "content": "Write a haiku about isolation."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:
        print("\n", chunk.usage.total_tokens, "tokens")

• The last chunk carries usage. Olympusx AI sets stream_options.include_usage on every streamed request, so a final chunk with an empty choices array and a usage object always arrives. Clients that iterate over choices[0] without checking the list is non-empty fail on it; check first, as above.
• Answers checked by guardrails arrive in one piece. When the tenant’s policy checks answers on the gateway, Olympusx AI holds the answer until it is complete, checks it, then sends it. The stream still has the streaming format, but its chunks arrive together at the end.
• A long silence ends the call. A call that sends nothing for the platform’s idle limit (300 seconds by default) is cut. A whole call is cut after the platform’s call limit (1,800 seconds by default). Reasoning models that think for minutes before answering are why these are generous; your platform administrator can change both. See Limits.

Tool calls

Tool calls work as in the OpenAI API: send tools, read tool_calls from the answer, run the tool in your application, and send its result back in a tool message. Tools that Olympusx AI runs for you belong to agents. See Agents, tools and skills.

The user field

Send user with the opaque handle your application already uses for the person the call is for: a customer number, an account reference, a session key.

• It makes captured calls erasable by person. When the tenant samples gateway traffic for review, the captured turn is stored with this reference, and Govern › Erasure can find and erase that person’s turns. Without it, a captured turn can be erased only when its retention window closes. See Erasure.
• It must not be personal data. A value containing @, or longer than 200 characters, is not stored: Olympusx AI drops it silently and the request goes on. A name, a national identifier or an account number would defeat the point of erasing anything.

Text completions

POST /v1/completions takes a prompt instead of messages and follows the OpenAI Completions API. Everything above applies to it: authentication, the model check, budgets, guardrails, streaming with usage, and the user field.

Errors

Status and code	Cause
400 model_required	No model in the body.
401 invalid_key, key_expired, key_disabled	The key is wrong, expired or revoked.
403 model_not_granted	The tenant is not granted the model.
403 model_not_allowed	The key’s allow-list does not include the model.
403 guardrails_blocked	The policy blocked the prompt or the answer.
404 model_not_deployed	Granted, and nothing serves it for this tenant.
429 budget_exceeded, model_budget_exceeded	A budget is spent for the period.
503 guardrails_unavailable	The scanner could not answer, and the platform fails closed.

Errors the model’s runtime returns, such as a prompt longer than the context window, pass through with the runtime’s status and message. The full list is in Errors.

Related

• API overview
• Models
• Guardrails