Skip to content
Olympusx AIOlympusx AIOlympusx AI documentation

Embeddings and reranking

Embedding models turn text into vectors for search and clustering; rerankers score a set of documents against a query so you can keep the most relevant. Both are called through the gateway with an API key, like any model, and both are metered and checked against budgets. Olympusx AI uses the same models for knowledge collections; this page is for applications that call them directly.

Embeddings

Section titled “Embeddings”

POST /v1/embeddings follows the OpenAI Embeddings API.

Field Required Notes
model Yes An embedding model the tenant is granted and serves.
input Yes A string, or an array of strings.
encoding_format No float (the default) or base64.
dimensions No Only for models that support shortened vectors.
user No As in chat completions. 
curl -s https://<your-gateway-host>/v1/embeddings \
  -H "Authorization: Bearer $OLYMPUS_API_KEY" \
  -H 'content-type: application/json' \
  -d '{"model":"<embedding-model>","input":["Refunds are issued within 14 days.","Claims need a policy number."]}'
from openai import OpenAI
import os


client = OpenAI(base_url="https://<your-gateway-host>/v1", api_key=os.environ["OLYMPUS_API_KEY"])
result = client.embeddings.create(
    model="<embedding-model>",
    input=["Refunds are issued within 14 days.", "Claims need a policy number."],
)
vectors = [item.embedding for item in result.data]
print(len(vectors), len(vectors[0]), result.usage.prompt_tokens)

The response is the OpenAI list object: data holds one embedding per input, in input order, and usage holds the tokens metered.

Vectors from different embedding models cannot be compared. Store the model name with every vector you keep, and embed queries with the same model as the documents.

An embedding model a cloud provider serves is called the same way, and its vectors are returned in the same shape.

Reranking

Section titled “Reranking”

A reranker scores documents against a query. Call it at the gateway’s root, not under /v1:

POST https://<your-gateway-host>/rerank
Field Required Notes
model Yes A reranker the tenant is granted and has deployed. The gateway checks and routes on it like any model.
query Yes The question.
texts Yes The documents to score, as an array of strings. 
curl -s https://<your-gateway-host>/rerank \
  -H "Authorization: Bearer $OLYMPUS_API_KEY" \
  -H 'content-type: application/json' \
  -d '{
    "model": "<reranker-model>",
    "query": "How long do refunds take?",
    "texts": ["Claims need a policy number.", "Refunds are issued within 14 days."]
  }'
import os, requests


resp = requests.post(
    "https://<your-gateway-host>/rerank",
    headers={"Authorization": f"Bearer {os.environ['OLYMPUS_API_KEY']}"},
    json={
        "model": "<reranker-model>",
        "query": "How long do refunds take?",
        "texts": ["Claims need a policy number.", "Refunds are issued within 14 days."],
    },
    timeout=30,
)
for hit in resp.json():
    print(hit["index"], hit["score"])

The answer is an array of {"index", "score"}, best first, where index points into texts and a higher score is more relevant.

Errors

Section titled “Errors”

The errors are the same as for chat completions: 400 model_required without a model, 403 model_not_granted for a model the tenant is not granted, and 404 model_not_deployed when nothing serves it. See Errors.

Section titled “Related”