Shared LLM API

Use one OpenAI-compatible base URL for pooled local models.

The shared LLM API exposes a stable OpenAI-compatible surface under this SaaS domain. Today it is backed by the broker-managed Qwen3 pool, and future local models can be added behind the same `/api/llm/v1` base URL so callers do not need to re-integrate.

GET/api/llm/v1/models

List live models

Returns the models currently exposed by the shared local LLM stack. Use this when you want runtime discovery instead of hard-coding one model forever.

POST/api/llm/v1/chat/completions

Create a chat completion

Accepts OpenAI-style `model`, `messages`, `temperature`, and `max_tokens` fields and forwards the request into the pooled local model router.

GET /api/llm/v1/models

Sample

curl -sS \
  -H "x-api-key: YOUR_API_KEY" \
  "https://kaleidovid.com/api/llm/v1/models"

Models response example

Sample

{
  "object": "list",
  "data": [
    {
      "id": "qwen3-4b",
      "object": "model",
      "owned_by": "kaleidovid-local"
    }
  ]
}

POST /api/llm/v1/chat/completions

Sample

curl -sS \
  -X POST \
  -H "Content-Type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{
    "model": "qwen3-4b",
    "messages": [
      { "role": "system", "content": "You are a helpful assistant." },
      { "role": "user", "content": "Explain why a shared local LLM endpoint is useful." }
    ],
    "temperature": 0.4,
    "max_tokens": 256
  }' \
  "https://kaleidovid.com/api/llm/v1/chat/completions"

Python OpenAI SDK example

Sample

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://kaleidovid.com/api/llm/v1",
)

response = client.chat.completions.create(
    model="qwen3-4b",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain the API in one short paragraph."},
    ],
    temperature=0.4,
    max_tokens=256,
)

print(response.choices[0].message.content)

The current production model exposed by this shared route is qwen3-4b. The important contract is the base URL, not the model family: as more local models are added, callers should discover them through /api/llm/v1/models and then choose a model id dynamically.