Skip to content

Deployments

Deploy as an OpenAI-compatible endpoint

A deployment serves a model version, or a base model with a system prompt, at its own chat-completions URL. Point any OpenAI SDK at it. Every response carries a prediction id, so feedback from your users flows back into the next training run.

Deploy an endpoint free

Free to start · Pay as you go · No card needed

What it does

Drop-in for the OpenAI SDK

Real SSE streaming, tools and tool_calls, response_format, stop, seed and the usual sampling parameters are forwarded to the model; errors use OpenAI's error shape.

Checked before it serves

Deploying makes a real test call first. If it fails, the deployment is marked failed with the error instead of serving a broken endpoint. Start and Stop control serving.

Revisions, rollback and canaries

Every change to what's served is a revision. Roll back to any earlier one, or send a percentage of traffic to another revision and compare their error rates and latency.

Keys and privacy per deployment

Deployment keys (kojen_dk_…) can call only their own endpoint and are metered separately. Choose what each request logs (nothing, metadata or redacted excerpts) and how long excerpts are kept.

Retraining from feedback

Rate a prediction with an optional correction. With auto-retraining on, a rising error rate or share of negative feedback starts a prompt-tuning run on the corrections; the result waits for review and is promoted only if it beats what's serving.

How it works

  1. 1Pick a model version (or a base model and system prompt) and deploy it.
  2. 2Create a deployment key and call the endpoint with any OpenAI client.
  3. 3Send feedback with each response's prediction id.
  4. 4Watch per-revision metrics, roll back or canary, and promote retrained versions.
Python, OpenAI SDK
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://kojen.ai/api/v1/endpoints/<deployment-id>",
    api_key=os.environ["KOJEN_DEPLOYMENT_KEY"],   # kojen_dk_… for this deployment (or an owner key)
)

reply = client.chat.completions.create(
    model="ignored",   # the deployment decides the model and system prompt
    messages=[{"role": "user", "content": "I was charged twice"}],
    temperature=0.2,
    max_tokens=256,
)
print(reply.choices[0].message.content)
prediction_id = reply.model_extra["kojen"]["prediction_id"]

Limits, plainly

  • 120 requests a minute per key and 600 per deployment; max 4,096 output tokens per request.

Questions

Is the endpoint really OpenAI-compatible?
It accepts the chat-completions request shape and returns chat.completion responses (or chat.completion.chunk events when streaming), plus a kojen object with the prediction id and revision. The deployment decides the model and system prompt, so the model field is ignored.
Can I roll back a bad release?
Yes. Every change creates a revision, and rolling back serves an earlier revision again. A canary split lets you try a new one on part of the traffic first.
Can LoRA adapters be deployed?
Not yet: deployments serve hosted models with their tuned prompts. Adapters from Text LoRA are saved as draft model versions.

Read more

More of Kojen

Deploy an endpoint free

Free to start · Pay as you go · No card needed