Skip to content

OpenAI-compatible endpoint

Use a Kojen deployment with the Vercel AI SDK

Call a Kojen-trained model from the Vercel AI SDK with @ai-sdk/openai-compatible: generateText, streamText, tool calling and useChat, with exact code.

Before you start

  • Deploy a model in Kojen (Deployments → New deployment). Its id goes in the base URL: https://kojen.ai/api/v1/endpoints/<deployment-id>.
  • Create a deployment key (kojen_dk_…) under Deployments → Learning, revisions & keys. It can only call that one deployment, so it's the right key to ship in an app. An account key (kojen_…, Settings → API Keys & MCP) of the deployment's owner with the read or write scope also works.
  • Keep the key server-side, e.g. in KOJEN_DEPLOYMENT_KEY. Anyone holding it can call the deployment and spend its owner's token allowance.

Create the provider

Install
npm install ai @ai-sdk/openai-compatible zod

Kojen deployments speak OpenAI's chat completions, so the AI SDK's OpenAI-compatible provider is all you need. Point baseURL at the deployment, without /chat/completions.

TypeScript
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { generateText } from "ai";

const kojen = createOpenAICompatible({
  name: "kojen",
  baseURL: "https://kojen.ai/api/v1/endpoints/<deployment-id>",
  apiKey: process.env.KOJEN_DEPLOYMENT_KEY, // kojen_dk_…
  includeUsage: true, // ask for the final usage chunk when streaming
});

// The model id is ignored by Kojen: the deployment decides what is served.
const { text } = await generateText({
  model: kojen("deployed"),
  prompt: "I was charged twice",
});
console.log(text);

Stream tokens

Streaming is real: the endpoint pipes the model's SSE through as it's generated, so streamText and useChat render tokens as they arrive.

TypeScript
import { streamText } from "ai";

const result = streamText({
  model: kojen("deployed"),
  messages: [{ role: "user", content: "App crashes on login" }],
});

for await (const delta of result.textStream) process.stdout.write(delta);

// In a route handler, return it to useChat instead:
// return result.toUIMessageStreamResponse();

Tool calling

tools and tool_choice are forwarded to the deployed model and its tool_calls come back unchanged, so multi-step tool use works as with any provider. Whether the model calls tools well depends on the model you deployed.

TypeScript
import { generateText, stepCountIs, tool } from "ai";
import { z } from "zod";

const { text, steps } = await generateText({
  model: kojen("deployed"),
  prompt: "Is order 1042 refundable?",
  tools: {
    getOrder: tool({
      description: "Look up an order by number",
      inputSchema: z.object({ orderNumber: z.string() }),
      execute: async ({ orderNumber }) => ({ orderNumber, status: "delivered", daysAgo: 12 }),
    }),
  },
  stopWhen: stepCountIs(5), // let the model call the tool, then answer
});

What the endpoint supports

  • model is ignored: the deployment decides the model and system prompt. Any string works; SDKs that require one just need something.
  • stream: true streams real SSE chunks ending in [DONE]; stream_options: { include_usage: true } adds a final usage chunk.
  • tools, tool_choice, parallel_tool_calls and response_format (text, json_object, json_schema) are forwarded to the model, and its tool_calls come back unchanged, so tool calling and structured output work through the SDK as usual.
  • Also forwarded: temperature, top_p, n (up to 4), stop, seed, the penalties and max_tokens / max_completion_tokens (up to 4,096).
  • Only POST /chat/completions and POST /feedback exist under the base URL. There is no /models, /embeddings or Responses API, so use the SDK's chat-completions client.
  • Each reply carries kojen.prediction_id (also the X-Kojen-Prediction-Id header). Post it to /feedback to rate a prediction and feed continuous learning.
  • Errors use OpenAI's { error: { message, type, param, code } } shape; 429s send Retry-After.

Get an API key

Kojen is free to start, no card needed. Train and evaluate an agent, deploy it behind an OpenAI-compatible endpoint, and connect your tools over MCP.

Create a free account

Also see: LangChain · LlamaIndex · OpenAI SDK · Claude Code · Cursor · Claude Desktop