Skip to content

OpenAI-compatible endpoint

Use a Kojen deployment with LangChain (Python and JS)

Point LangChain's ChatOpenAI at a Kojen deployment's OpenAI-compatible base URL, in Python and JavaScript, with streaming, tool calling, structured output and the Kojen MCP tools.

Before you start

  • Deploy a model in Kojen (Deployments → New deployment). Its id goes in the base URL: https://kojen.ai/api/v1/endpoints/<deployment-id>.
  • Create a deployment key (kojen_dk_…) under Deployments → Learning, revisions & keys. It can only call that one deployment, so it's the right key to ship in an app. An account key (kojen_…, Settings → API Keys & MCP) of the deployment's owner with the read or write scope also works.
  • Keep the key server-side, e.g. in KOJEN_DEPLOYMENT_KEY. Anyone holding it can call the deployment and spend its owner's token allowance.

Python

Python (langchain-openai)
pip install langchain-openai

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="deployed",  # ignored: the deployment decides the model
    base_url="https://kojen.ai/api/v1/endpoints/<deployment-id>",
    api_key=os.environ["KOJEN_DEPLOYMENT_KEY"],  # kojen_dk_…
    temperature=0.2,
)

print(llm.invoke("I was charged twice").content)

for chunk in llm.stream("App crashes on login"):
    print(chunk.content, end="", flush=True)

Tools and structured output

Because tools and response_format are forwarded, bind_tools and with_structured_output behave as they do against OpenAI, as far as the deployed model supports them.

Python
from pydantic import BaseModel, Field
from langchain_core.tools import tool

@tool
def get_order(order_number: str) -> dict:
    """Look up an order by number."""
    return {"order_number": order_number, "status": "delivered", "days_ago": 12}

# Tool calling: tools and tool_calls pass straight through the endpoint.
reply = llm.bind_tools([get_order]).invoke("Is order 1042 refundable?")
print(reply.tool_calls)

# Structured output: response_format (json_schema) is forwarded to the model.
class Ticket(BaseModel):
    category: str = Field(description="billing, bug or how-to")
    urgent: bool

print(llm.with_structured_output(Ticket).invoke("I was charged twice and need it fixed today"))

JavaScript / TypeScript

In LangChain.js the base URL goes in configuration, which is passed to the OpenAI client underneath.

TypeScript (@langchain/openai)
npm install @langchain/openai @langchain/core

import { ChatOpenAI } from "@langchain/openai";

const llm = new ChatOpenAI({
  model: "deployed", // ignored by Kojen
  apiKey: process.env.KOJEN_DEPLOYMENT_KEY, // kojen_dk_…
  configuration: { baseURL: "https://kojen.ai/api/v1/endpoints/<deployment-id>" },
});

const reply = await llm.invoke("I was charged twice");
console.log(reply.content);

for await (const chunk of await llm.stream("App crashes on login")) process.stdout.write(String(chunk.content));

Give a LangChain agent the Kojen tools

The other direction: let a LangChain or LangGraph agent drive the dojo itself (create datasets, tune prompts, run evals) through Kojen's MCP server. Use an account key (kojen_…) here, not a deployment key.

Python (langchain-mcp-adapters)
pip install langchain-mcp-adapters

import os
from langchain_mcp_adapters.client import MultiServerMCPClient

client = MultiServerMCPClient({
    "kojen": {
        "transport": "streamable_http",
        "url": "https://kojen.ai/api/mcp",
        "headers": {"Authorization": f"Bearer {os.environ['KOJEN_KEY']}"},  # kojen_… account key
    }
})
tools = await client.get_tools()  # Kojen's dojo tools, ready for any LangChain/LangGraph agent

What the endpoint supports

  • model is ignored: the deployment decides the model and system prompt. Any string works; SDKs that require one just need something.
  • stream: true streams real SSE chunks ending in [DONE]; stream_options: { include_usage: true } adds a final usage chunk.
  • tools, tool_choice, parallel_tool_calls and response_format (text, json_object, json_schema) are forwarded to the model, and its tool_calls come back unchanged, so tool calling and structured output work through the SDK as usual.
  • Also forwarded: temperature, top_p, n (up to 4), stop, seed, the penalties and max_tokens / max_completion_tokens (up to 4,096).
  • Only POST /chat/completions and POST /feedback exist under the base URL. There is no /models, /embeddings or Responses API, so use the SDK's chat-completions client.
  • Each reply carries kojen.prediction_id (also the X-Kojen-Prediction-Id header). Post it to /feedback to rate a prediction and feed continuous learning.
  • Errors use OpenAI's { error: { message, type, param, code } } shape; 429s send Retry-After.

Get an API key

Kojen is free to start, no card needed. Train and evaluate an agent, deploy it behind an OpenAI-compatible endpoint, and connect your tools over MCP.

Create a free account

Also see: Vercel AI SDK · LlamaIndex · OpenAI SDK · Claude Code · Cursor · Claude Desktop