Skip to content

Best of · Prompt optimization

The best prompt optimization tools in 2026

The best prompt optimization tool for engineers is DSPy, whose optimizers (GEPA, MIPROv2 and others) rewrite instructions and few-shot examples across whole multi-step programs; for a hosted option without code, Kojen tunes a system prompt against your test cases and searches 11 models, and Google's Vertex AI Prompt Optimizer serves Gemini users. DeepEval, AdalFlow, PromptWizard and TextGrad are good open-source libraries for narrower jobs.

One hosted option is on its way out: OpenAI's dataset-backed prompt optimizer is being retired along with its Evals platform.

By Will SchulzUpdated 29 sources

The list

  1. 01DSPy

    dspy.ai

    Best for: engineers optimizing multi-step LLM programs

    The widest set of optimizers anywhere, for instructions (GEPA, MIPROv2, COPRO, SIMBA), demonstrations (Bootstrap family) and weights (BootstrapFinetune). GEPA, now stable in DSPy 3.5.0, reported beating GRPO by 6% on average with up to 35 times fewer rollouts. You restructure code and write the metric.

    Pricing: Free, MIT-licensed library. You pay your own model and GPU costs. Source · Open source: yes (MIT)

    Strengths

    • The widest set of optimizers anywhere, covering instructions, few-shot demos and weights.
    • Optimizes whole multi-step programs, not one prompt at a time.
    • Strong research lineage: GEPA, now a built-in optimizer, was an ICLR 2026 oral.

    Limits

    • A real learning curve: you restructure code into signatures and modules and write your own metrics.
    • No hosted UI, managed training or serving; RL needs your own GPUs and is experimental.
    • Defaults change between releases (3.5.0 changed the default adapter), so optimized programs need re-evaluating.
  2. 02Kojenour product

    kojen.ai

    Best for: hosted prompt tuning and model choice without code

    The Forge rewrites the system prompt against your test cases, scored by an LLM judge, accepts a new prompt only when a paired bootstrap test says it beats the current one, and can search 11 hosted models for the best fit. The result is evaluated with confidence intervals, red-teamed and deployable. It tunes single-prompt tasks, not multi-module programs. We make Kojen.

    Pricing: Free to start, no card needed; then pay as you go. GPU training (LoRA) is paid in credits billed by the second at the GPU's live price. Team and volume pricing by email. Source · Open source: no

    Strengths

    • Measures and improves in one loop: evaluations, red-teaming, prompt tuning, model search and deployment share the same project and test cases.
    • Honest statistics by default: every rate has a confidence interval, tuned prompts must beat the current one in a paired test, and pass@k / pass^k come from several samples per case.
    • Agent-native: the full loop is exposed over an MCP server and a REST API with an OpenAPI spec, with the same 24 tools as the in-app agent.

    Limits

    • Young product: weight training is early access on 8B models, LoRA adapters can't be served yet, and full fine-tuning and RLHF are previews.
    • No tracing SDK for apps running elsewhere, so it is not a replacement for an observability platform.
    • Closed source and hosted only; no self-hosting.
  3. Best for: Gemini and Google Cloud users

    A zero-shot mode that needs only your prompt and a data-driven mode that scores rewrites against labelled samples and custom metrics, including small on-device models. Tied to Google Cloud; the SDK is labelled experimental.

    Pricing: Pay as you go: billed as the Gemini calls and Vertex jobs it runs. No separate published rate found. Source · Open source: no

    Strengths

    • Both a no-data zero-shot mode and a metric-driven batch mode.
    • Custom metrics supported.
    • Covers small on-device models such as Gemini Nano and Gemma 3n.

    Limits

    • Tied to Google Cloud and mostly Gemini models.
    • The SDK is labelled experimental and pricing is folded into other services.
  4. 04DeepEval / Confident AI

    www.confident-ai.com

    Best for: optimizing prompts inside an existing eval suite

    DeepEval's PromptOptimizer runs GEPA and MIPROv2 against the same metrics you already evaluate with, so optimization and testing share one definition of good.

    Pricing: Free (2 seats, 1 project); Starter $200/month; Team $2,000/month; Team++ and Enterprise custom. Priced per organization, not per seat. Source · Open source: yes (Apache-2.0)

    Strengths

    • One of the largest metric libraries, including agent, multi-turn and MCP metrics.
    • Prompt optimizers (GEPA, MIPROv2) built in.
    • A hosted MCP server and tracing platform on top of the open-source library.

    Limits

    • Red-teaming is a separate library, and in the hosted product it is Enterprise-only.
    • Steep jump between paid tiers, from $200 to $2,000 a month.
  5. Best for: combining several optimization methods in one loop

    Textual-gradient instruction tuning, few-shot bootstrapping and an OPRO-style history of past prompts in a PyTorch-like library, actively released. Smaller community than DSPy.

    Pricing: Free, MIT-licensed library. Source · Open source: yes (MIT)

    Strengths

    • Combines three optimization ideas in one loop.
    • Actively released (v1.1.4, November 2026).
    • Handles multi-component pipelines such as RAG and agents.

    Limits

    • A much smaller community than DSPy.
    • Library only; you build evaluators and infrastructure yourself.
  6. Best for: single prompts with little or no labelled data

    Microsoft's agent-driven optimizer refines instructions and examples together and can synthesize examples aimed at a prompt's weak spots. One prompt at a time.

    Pricing: Free, MIT-licensed library. You pay model API costs. Source · Open source: yes (MIT)

    Strengths

    • Optimizes instructions and examples together, and can synthesize examples aimed at a prompt's weak spots.
    • Works with little or no labelled data.
    • MIT-licensed, from Microsoft.

    Limits

    • One prompt at a time, not multi-step pipelines.
    • Set up mainly around Q&A and reasoning benchmarks, with low recent activity.
  7. Best for: research and inference-time refinement

    Published in Nature in 2025, TextGrad treats LLM critiques as gradients for any text variable. No commits since mid-2025.

    Pricing: Free, MIT-licensed library. You pay model API costs. Source · Open source: yes (MIT)

    Strengths

    • Peer-reviewed method, published in Nature (2025).
    • Optimizes any text, including single answers at inference time.
    • Familiar PyTorch-style API for ML engineers.

    Limits

    • No commits since July 2025.
    • Every step costs extra LLM calls, and there is no evaluation or production tooling around it.

Side-by-side comparison

"Partial" means the feature exists with real limits, given in the note. "Not verified" means we couldn't confirm it from the vendor's own material.

The best prompt optimization tools in 2026: features and pricing, checked November 2026
Checked Nov 15, 2026DSPyKojenVertex AI Prompt OptimizerDeepEval / Confident AIAdalFlowPromptWizard
What it isPrompt optimization

Open-source project started at Stanford NLP

Training dojo

Kojen (that's us)

Prompt optimization

Google Cloud

Evaluation framework

Confident AI

Prompt optimization

SylphAI

Prompt optimization

Microsoft

PricingFree, MIT-licensed library. You pay your own model and GPU costs. SourceFree to start, no card needed; then pay as you go. GPU training (LoRA) is paid in credits billed by the second at the GPU's live price. Team and volume pricing by email. SourcePay as you go: billed as the Gemini calls and Vertex jobs it runs. No separate published rate found. SourceFree (2 seats, 1 project); Starter $200/month; Team $2,000/month; Team++ and Enterprise custom. Priced per organization, not per seat. SourceFree, MIT-licensed library. SourceFree, MIT-licensed library. You pay model API costs. Source
Open sourceYes, MIT RepoNoNoYes, Apache-2.0 RepoYes, MIT RepoYes, MIT Repo
Prompt optimization

Automatically rewrites instructions or examples to raise a metric.

Yes

GEPA, MIPROv2, COPRO, SIMBA, Bootstrap few-shot and more

Yes

Prompt tuning plus search across 11 hosted models

Yes

Zero-shot and data-driven modes

Yes

PromptOptimizer with GEPA and MIPROv2

Yes

Text gradients + few-shot bootstrap + OPRO-style history

Yes

Instructions and few-shot examples together

Evaluations on datasets

Run a model or app over a set of test cases and score the results.

Yes

dspy.Evaluate over a devset with your metric

Yes

Test cases scored for accuracy, hallucination, bias and latency

Partial

Through Vertex AI's evaluation service

Yes
Partial

Evaluators inside the training loop

No
LLM-as-judge scoring

A model grades outputs against a rubric or reference.

Partial

Built-in LLM metrics (SemanticF1, CompleteAndGrounded); otherwise you write the judge

Yes

Judge kept out of the target's model family in simulations

Partial

Vertex evaluation metrics

Yes

G-Eval, DAG and dozens of built-in metrics

Partial
Partial

LLM critiques drive the refinement

Confidence intervals

Rates reported with an interval or a significance test, not just a point estimate.

No

Reports a single score

Yes

Wilson and bootstrap intervals on every rate

Not verified
No

Not in the official docs

No
No
Fine-tuning / LoRA

Trains model weights or adapters.

Partial

BootstrapFinetune; RL via Arbor is labelled experimental; bring your own GPUs

Partial

Text LoRA on Qwen3-8B / Llama-3.1-8B, early access; adapters can't be served yet; full fine-tuning and RLHF are previews

Yes

Separate Vertex AI tuning service

No
No
No
Model endpoints

Serves a model or tuned version behind an API you call in production.

No

Library; deploy in your own code

Yes

OpenAI-compatible, SSE streaming, tool calls forwarded

Yes

Vertex AI endpoints

No
No
No
Self-host

You can run it on your own infrastructure.

Yes

Runs wherever your Python runs

No

Hosted only

No
Partial

Libraries run locally; platform on-prem is Enterprise

Yes

Python library

Yes

Python library

Check the improvement is real

Every optimizer will find a prompt that scores higher on the examples it saw. The risk is overfitting: the new prompt wins on the training cases and loses elsewhere. Hold out cases the optimizer never sees, and compare old and new prompts on the same held-out cases with a paired test, not two averages. On small test sets, differences of a few points are usually noise.

Optimizing the prompt first is also the cheaper lever. Research such as OPRO showed LLM-proposed instructions beating human-written ones by up to 8% on GSM8K and up to 50% on Big-Bench Hard tasks, before any weights change.

How we made this list

  • We compared what each tool changes (instructions, examples, weights, model choice), what signal it needs (a metric, labelled data, an LLM judge), whether it checks the improvement is real, and how much code it takes.
  • Order reflects usefulness for production prompt optimization in November 2026.
  • Every feature claim links to the vendor's own pricing page, docs or repository, checked on November 15, 2026. Where we couldn't confirm a feature, the table says "Not verified".
  • We make Kojen. We include it where it does the job and say where others are stronger; it is not placed first by default.
  • No vendor paid to be included, and there are no affiliate links.

Frequently asked questions

What is automatic prompt optimization?
Using an algorithm, usually an LLM in a loop, to rewrite a prompt's instructions or examples and keep the versions that score higher on a metric over a set of examples.
What is the best free prompt optimization tool?
DSPy (MIT) for multi-step programs and the widest choice of optimizers; DeepEval's PromptOptimizer if you already use DeepEval; AdalFlow or PromptWizard (both MIT) for single prompts.
Is OpenAI's prompt optimizer going away?
The dataset-backed version is: OpenAI says it is deprecating it as part of the Evals platform, which became read-only on October 31, 2026 and is scheduled to shut down on November 30, 2026.
Does prompt optimization replace fine-tuning?
Often, for format, tone and policy following; not for teaching new knowledge or skills a model lacks. Try prompt tuning and model choice first and fine-tune only when a measured gap remains.

Keep reading

Sources

Checked November 2026.

  1. [1]stanfordnlp/dspy. GitHub.
  2. [2]DSPy 3.5.0 release notes. GitHub.
  3. [3]dspy.Evaluate. DSPy docs.
  4. [4]BootstrapFinetune. DSPy docs.
  5. [5]Tutorial: Online RL over a multi-module DSPy program. DSPy docs.
  6. [6]Use MCP in DSPy. DSPy docs.
  7. [7]GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning. arXiv (Agrawal et al.).
  8. [8]Prompt optimizer. Google Cloud docs.
  9. [9]Data-driven prompt optimizer. Google Cloud docs.
  10. [10]Pricing. Confident AI.
  11. [11]confident-ai/deepeval. GitHub.
  12. [12]confident-ai/deepteam. GitHub.
  13. [13]Evaluation flags and configs. DeepEval docs.
  14. [14]Prompt optimization: MIPROv2. DeepEval docs.
  15. [15]MCP reference. Confident AI docs.
  16. [16]SylphAI-Inc/AdalFlow. GitHub.
  17. [17]Auto-Differentiating Any LLM Workflow: A Farewell to Manual Prompting. arXiv.
  18. [18]microsoft/PromptWizard. GitHub.
  19. [19]PromptWizard: Task-Aware Prompt Optimization Framework. arXiv.
  20. [20]zou-group/textgrad. GitHub.
  21. [21]Optimizing generative AI by backpropagating language model feedback. Nature 639, 609–616 (2025).
  22. [22]Kojen for Agents: REST API & MCP Server. Kojen.
  23. [23]llms.txt. Kojen.
  24. [24]Pricing. Kojen.
  25. [25]Large Language Models as Optimizers (OPRO). arXiv (Yang et al., ICLR 2024).
  26. [26]Prompt optimizer (deprecation notice). OpenAI docs.
  27. [27]Deprecations: Evals platform (2026-06-03). OpenAI docs.
  28. [28]promptim / promptimizer. GitHub.
  29. [29]Loop. Braintrust docs.

Try it on your own test cases.

Run an LLM-judged evaluation with confidence intervals, a red-team pass and a prompt-tuning run on your agent. Free to start, no card needed, then pay as you go.

Start training free