The core difference: a library versus a loop
DSPy asks you to restructure your pipeline into signatures and modules, write a metric function, and run an optimizer such as GEPA or MIPROv2 over a training set. In return it can optimize multi-step programs end to end, which no hosted tool matches today. GEPA, its reflective optimizer, reported beating GRPO reinforcement learning by 6% on average while using up to 35 times fewer rollouts.
Kojen works one level up. You bring a task description and test cases; the Forge's prompt tuning rewrites the system prompt, scored by an LLM judge on those cases, and model search tries the same task across 11 hosted models to find the cheapest one that holds up. The winning version is saved, can be evaluated with confidence intervals and several samples per case (pass@k and pass^k), red-teamed for prompt injection and data exfiltration, and served behind an OpenAI-compatible endpoint.
Weight training
DSPy's BootstrapFinetune distils a program's traces into fine-tuned weights, and its GRPO path via Arbor is labelled experimental; both run on GPUs you provide. Kojen's text LoRA trains a real adapter on Qwen3-8B or Llama-3.1-8B-Instruct on rented GPUs, paid in credits for the time used, but it is early access for enabled accounts and the adapters can't serve evaluations or deployments yet. Neither is a full managed fine-tuning service.
DSPy at a glance
A Python framework for programming LLM pipelines as modules and optimizing them against a metric you write. Its optimizers rewrite instructions (MIPROv2, COPRO, GEPA, SIMBA), pick few-shot demos (the Bootstrap family) and can fine-tune weights (BootstrapFinetune, experimental GRPO via Arbor). Version 3.5.0 shipped on November 10, 2026.
- The widest set of optimizers anywhere, covering instructions, few-shot demos and weights.
- Optimizes whole multi-step programs, not one prompt at a time.
- Strong research lineage: GEPA, now a built-in optimizer, was an ICLR 2026 oral.
- A real learning curve: you restructure code into signatures and modules and write your own metrics.
- No hosted UI, managed training or serving; RL needs your own GPUs and is experimental.
- Defaults change between releases (3.5.0 changed the default adapter), so optimized programs need re-evaluating.
Kojen at a glance
A hosted dojo for AI agents: LLM-judged evaluations with confidence intervals and pass@k / pass^k, red-team and stress simulations, prompt tuning and model search, LoRA training in early access, and OpenAI-compatible endpoints, all driveable over MCP and REST.
- Measures and improves in one loop: evaluations, red-teaming, prompt tuning, model search and deployment share the same project and test cases.
- Honest statistics by default: every rate has a confidence interval, tuned prompts must beat the current one in a paired test, and pass@k / pass^k come from several samples per case.
- Agent-native: the full loop is exposed over an MCP server and a REST API with an OpenAPI spec, with the same 24 tools as the in-app agent.
- Young product: weight training is early access on 8B models, LoRA adapters can't be served yet, and full fine-tuning and RLHF are previews.
- No tracing SDK for apps running elsewhere, so it is not a replacement for an observability platform.
- Closed source and hosted only; no self-hosting.