Use a Kojen deployment with LlamaIndex
Call a Kojen-trained model from LlamaIndex with OpenAILike in Python or the OpenAI LLM in LlamaIndex.TS, including chat, streaming and function-calling agents.
Before you start
- Deploy a model in Kojen (Deployments → New deployment). Its id goes in the base URL:
https://kojen.ai/api/v1/endpoints/<deployment-id>. - Create a deployment key (
kojen_dk_…) under Deployments → Learning, revisions & keys. It can only call that one deployment, so it's the right key to ship in an app. An account key (kojen_…, Settings → API Keys & MCP) of the deployment's owner with thereadorwritescope also works. - Keep the key server-side, e.g. in
KOJEN_DEPLOYMENT_KEY. Anyone holding it can call the deployment and spend its owner's token allowance.
Python: OpenAILike
Use OpenAILike, not OpenAI: it doesn't validate the model name against OpenAI's list. Set is_chat_model=True, since the endpoint serves chat completions only, and set context_window to your deployed model's, because LlamaIndex can't look it up.
pip install llama-index-llms-openai-like
import os
from llama_index.llms.openai_like import OpenAILike
from llama_index.core.llms import ChatMessage
llm = OpenAILike(
model="deployed", # ignored by Kojen
api_base="https://kojen.ai/api/v1/endpoints/<deployment-id>",
api_key=os.environ["KOJEN_DEPLOYMENT_KEY"], # kojen_dk_…
is_chat_model=True, # Kojen serves chat completions only
is_function_calling_model=True, # tools are forwarded to the model
context_window=128000, # set to your deployed model's window
)
print(llm.chat([ChatMessage(role="user", content="I was charged twice")]))
for chunk in llm.stream_complete("App crashes on login"):
print(chunk.delta, end="", flush=True)Function-calling agents
With is_function_calling_model=True, LlamaIndex agents send tools and read the model's tool_calls, which the endpoint passes through.
from llama_index.core.agent.workflow import FunctionAgent
def get_order(order_number: str) -> dict:
"""Look up an order by number."""
return {"order_number": order_number, "status": "delivered", "days_ago": 12}
agent = FunctionAgent(tools=[get_order], llm=llm)
print(await agent.run("Is order 1042 refundable?"))TypeScript: LlamaIndex.TS
LlamaIndex.TS has no OpenAILike; its OpenAI class takes any model name and a baseURL (or the OPENAI_BASE_URL environment variable), and supports tool calls for any model name.
npm install llamaindex @llamaindex/openai
import { OpenAI } from "@llamaindex/openai";
const llm = new OpenAI({
// Kojen ignores the model name, but LlamaIndex.TS looks it up for the context window
// (unknown names fall back to 1,024 tokens), so pass an OpenAI model name.
model: "gpt-4o-mini",
baseURL: "https://kojen.ai/api/v1/endpoints/<deployment-id>",
apiKey: process.env.KOJEN_DEPLOYMENT_KEY, // kojen_dk_…
});
const reply = await llm.chat({ messages: [{ role: "user", content: "I was charged twice" }] });
console.log(reply.message.content);What the endpoint supports
modelis ignored: the deployment decides the model and system prompt. Any string works; SDKs that require one just need something.stream: truestreams real SSE chunks ending in[DONE];stream_options: { include_usage: true }adds a final usage chunk.tools,tool_choice,parallel_tool_callsandresponse_format(text,json_object,json_schema) are forwarded to the model, and itstool_callscome back unchanged, so tool calling and structured output work through the SDK as usual.- Also forwarded:
temperature,top_p,n(up to 4),stop,seed, the penalties andmax_tokens/max_completion_tokens(up to 4,096). - Only
POST /chat/completionsandPOST /feedbackexist under the base URL. There is no/models,/embeddingsor Responses API, so use the SDK's chat-completions client. - Each reply carries
kojen.prediction_id(also theX-Kojen-Prediction-Idheader). Post it to/feedbackto rate a prediction and feed continuous learning. - Errors use OpenAI's
{ error: { message, type, param, code } }shape; 429s sendRetry-After.
Get an API key
Kojen is free to start, no card needed. Train and evaluate an agent, deploy it behind an OpenAI-compatible endpoint, and connect your tools over MCP.