Skip to content

Best of · Red-teaming

The best AI red-teaming tools in 2026

The best AI red-teaming tools in 2026 are Promptfoo for the broadest attack library with OWASP, NIST and MITRE ATLAS presets, NVIDIA garak for scanning models for known vulnerabilities, Microsoft PyRIT for expert multi-turn attacks, DeepEval's DeepTeam for an open-source red-team library tied to an eval platform, and Giskard for EU-hosted continuous red-teaming. Kojen ranks lower on attack coverage but reports robustness with a confidence interval and lets you fix and retest in the same place.

Most of these are free and open source, so the choice is mainly about coverage, workflow and who owns the tool.

By Will SchulzUpdated 25 sources

The list

  1. Best for: the broadest attack coverage, free

    Dozens of plugins combined with strategies like Crescendo, GOAT, tree search and Best-of-N, plus presets for the OWASP LLM and agentic Top 10s, NIST AI RMF, MITRE ATLAS and the EU AI Act, and MCP red-teaming. Free up to 10k probes a month. OpenAI announced its acquisition in March 2026; it remains MIT-licensed.

    Pricing: Community: free forever, including self-hosting and red-teaming up to 10k probes a month. Enterprise and On-Premise: custom pricing. Source · Open source: yes (MIT)

    Strengths

    • The widest red-team coverage available, with presets for OWASP LLM and agentic Top 10s, NIST AI RMF, MITRE ATLAS and the EU AI Act.
    • MIT-licensed config-as-code that lives in your repo and runs in CI.
    • OpenAI now names it as the migration path off OpenAI's own hosted Evals, which shut down on November 30, 2026.

    Limits

    • No statistics built in: no confidence intervals and no pass@k metric.
    • Now owned by OpenAI, which some teams testing other vendors' models will weigh.
    • Doesn't improve the model it finds problems in; there is no tuning or serving.
  2. Best for: scanning a model for known vulnerabilities

    NVIDIA's probe-and-detector scanner, often compared to nmap for LLMs, with a large research-backed probe set, many target types, OWASP tags and AVID reports. Tests models rather than your app's cases.

    Pricing: Free, Apache-2.0. No paid tier. Source · Open source: yes (Apache-2.0)

    Strengths

    • A large, research-backed probe library.
    • Talks to many targets: Hugging Face, OpenAI, Bedrock, NIM, LiteLLM, REST and local gguf models.
    • OWASP tagging and AVID reporting.

    Limits

    • Scans a model or endpoint rather than your app's own test cases.
    • CLI only, with no team platform.
  3. Best for: expert, multi-turn red-teaming

    Microsoft AI Red Team's composable toolkit for adaptive multi-turn attacks (Crescendo, TAP, PAIR, many-shot) across text, image, audio and video. More code, more depth.

    Pricing: Free, MIT. Source · Open source: yes (MIT)

    Strengths

    • Strong multi-turn, adaptive attacks.
    • Composable building blocks suited to expert, human-in-the-loop red teams.
    • Maintained by Microsoft's AI Red Team (moved to github.com/microsoft/PyRIT in 2026).

    Limits

    • Code-heavy, with more setup than a scanner.
    • No general evaluation workflow and no published compliance mapping that we found.
  4. 04DeepEval / Confident AI

    www.confident-ai.com

    Best for: red-teaming next to an eval platform

    DeepTeam, from the DeepEval team, covers 50+ vulnerabilities and multi-turn attacks with OWASP, NIST and MITRE ATLAS mappings. In the hosted Confident AI product, red-teaming is Enterprise-only.

    Pricing: Free (2 seats, 1 project); Starter $200/month; Team $2,000/month; Team++ and Enterprise custom. Priced per organization, not per seat. Source · Open source: yes (Apache-2.0)

    Strengths

    • One of the largest metric libraries, including agent, multi-turn and MCP metrics.
    • Prompt optimizers (GEPA, MIPROv2) built in.
    • A hosted MCP server and tracing platform on top of the open-source library.

    Limits

    • Red-teaming is a separate library, and in the hosted product it is Enterprise-only.
    • Steep jump between paid tiers, from $200 to $2,000 a month.
  5. Best for: continuous red-teaming with EU hosting

    An open-source vulnerability scan across OWASP LLM Top 10 categories and a Hub with continuous red-teaming, offered in the EU, the US, private cloud or on-prem. Enterprise pricing isn't public.

    Pricing: Open-source library free; Hub Enterprise by quote. SaaS in the EU or US, private cloud or on-prem. Source · Open source: yes (Apache-2.0)

    Strengths

    • Quality checks and red-teaming in one tool.
    • EU data residency and on-prem options.
    • A Hub playground non-developers can use.

    Limits

    • Enterprise pricing isn't published.
    • The v2 to v3 rewrite dropped some features (v2 is no longer maintained).
  6. 06Kojenour product

    kojen.ai

    Best for: red-teaming that feeds straight into fixing and retesting

    Five attack categories (prompt injection, jailbreak, data exfiltration, harmful content, PII leak), a judge from a different model family than the target, errors counted as failures, and a robustness score with a confidence interval. Generated attacks are kept for reruns after you tune the prompt or switch models. Narrower coverage than the tools above. We make Kojen.

    Pricing: Free to start, no card needed; then pay as you go. GPU training (LoRA) is paid in credits billed by the second at the GPU's live price. Team and volume pricing by email. Source · Open source: no

    Strengths

    • Measures and improves in one loop: evaluations, red-teaming, prompt tuning, model search and deployment share the same project and test cases.
    • Honest statistics by default: every rate has a confidence interval, tuned prompts must beat the current one in a paired test, and pass@k / pass^k come from several samples per case.
    • Agent-native: the full loop is exposed over an MCP server and a REST API with an OpenAPI spec, with the same 24 tools as the in-app agent.

    Limits

    • Young product: weight training is early access on 8B models, LoRA adapters can't be served yet, and full fine-tuning and RLHF are previews.
    • No tracing SDK for apps running elsewhere, so it is not a replacement for an observability platform.
    • Closed source and hosted only; no self-hosting.

Side-by-side comparison

"Partial" means the feature exists with real limits, given in the note. "Not verified" means we couldn't confirm it from the vendor's own material.

The best AI red-teaming tools in 2026: features and pricing, checked November 2026
Checked Nov 15, 2026PromptfoogarakPyRITDeepEval / Confident AIGiskardKojen
What it isRed-teaming

OpenAI (acquisition announced March 9, 2026); still MIT open source

Red-teaming

NVIDIA

Red-teaming

Microsoft AI Red Team

Evaluation framework

Confident AI

Red-teaming

Giskard (France)

Training dojo

Kojen (that's us)

PricingCommunity: free forever, including self-hosting and red-teaming up to 10k probes a month. Enterprise and On-Premise: custom pricing. SourceFree, Apache-2.0. No paid tier. SourceFree, MIT. SourceFree (2 seats, 1 project); Starter $200/month; Team $2,000/month; Team++ and Enterprise custom. Priced per organization, not per seat. SourceOpen-source library free; Hub Enterprise by quote. SaaS in the EU or US, private cloud or on-prem. SourceFree to start, no card needed; then pay as you go. GPU training (LoRA) is paid in credits billed by the second at the GPU's live price. Team and volume pricing by email. Source
Open sourceYes, MIT RepoYes, Apache-2.0 RepoYes, MIT RepoYes, Apache-2.0 RepoYes, Apache-2.0 RepoNo
Red-teaming

Generates adversarial attacks (prompt injection, jailbreaks, data leaks) and scores them.

Yes

Dozens of plugins; Crescendo, GOAT, tree and Best-of-N strategies; OWASP/NIST/MITRE presets

Yes

Probe/detector scanner with OWASP tags and AVID reports

Yes

Crescendo, TAP, PAIR, many-shot, Skeleton Key; text, image, audio and video converters

Yes

DeepTeam library; hosted red-teaming is Enterprise-only

Yes

Vulnerability scan across OWASP LLM Top 10 categories; continuous red-teaming in the Hub

Yes

Prompt injection, jailbreak, data exfiltration, harmful content, PII leak

LLM-as-judge scoring

A model grades outputs against a rubric or reference.

Yes

Model-graded assertions

Yes

ModelAsJudge, refusal and jailbreak judges

Yes

True/false, Likert and classification scorers

Yes

G-Eval, DAG and dozens of built-in metrics

Yes

Groundedness, conformity, LLM judge checks

Yes

Judge kept out of the target's model family in simulations

pass@k / pass^k

Several samples per case, reported as any-success (pass@k) or all-success (pass^k).

Partial

eval --repeat n, but no pass@k metric

Partial

--generations outputs per prompt (default 10); pass counts, no pass@k

Not verified
Partial

Repeat runs per case; no pass@k metric

Not verified
Yes

1–10 samples per case

Confidence intervals

Rates reported with an interval or a significance test, not just a point estimate.

No

Not in the CLI docs

Not verified
Not verified
No

Not in the official docs

Not verified
Yes

Wilson and bootstrap intervals on every rate

Evaluations on datasets

Run a model or app over a set of test cases and score the results.

Yes

YAML test cases and assertions, runs in CI

Partial

Runs its own probe sets, not your test cases

Partial

Scenarios and datasets for repeatable assessments

Yes
Yes
Yes

Test cases scored for accuracy, hallucination, bias and latency

Prompt optimization

Automatically rewrites instructions or examples to raise a metric.

Partial

promptfoo optimize: one prompt against one provider

No
No
Yes

PromptOptimizer with GEPA and MIPROv2

No
Yes

Prompt tuning plus search across 11 hosted models

Official MCP server

An MCP server from the vendor that lets agents drive the product.

Yes

promptfoo mcp, plus MCP red-team testing

No
Not verified
Yes

Hosted MCP server

No
Yes

ai.kojen/kojen in the MCP Registry, OAuth sign-in

Self-host

You can run it on your own infrastructure.

Yes

Free in the Community tier

Yes
Yes
Partial

Libraries run locally; platform on-prem is Enterprise

Yes

Library; Hub on-prem by quote

No

Hosted only

Scanner, toolkit or loop

Red-teaming tools fall into three shapes. Scanners such as garak run a fixed probe library against a model and report what got through. Toolkits such as PyRIT and Promptfoo let you compose attacks against your own app and its tools. Loops such as Kojen generate attacks for your task, then keep them so the same attacks can be rerun after every fix.

Whatever you pick, judge results carefully: an LLM judge grading its own model family can be lenient, and a response that errored is not a defended attack. Report robustness with an interval, because 4 successful attacks out of 20 and 40 out of 200 are both 80% defended but carry very different certainty.

How we made this list

  • We compared attack coverage (plugins, strategies, multi-turn), mapping to OWASP, NIST and MITRE frameworks, how results are judged and reported, and what you can do with a finding.
  • Order mostly reflects attack coverage, the main job of a red-teaming tool.
  • Every feature claim links to the vendor's own pricing page, docs or repository, checked on November 15, 2026. Where we couldn't confirm a feature, the table says "Not verified".
  • We make Kojen. We include it where it does the job and say where others are stronger; it is not placed first by default.
  • No vendor paid to be included, and there are no affiliate links.

Frequently asked questions

What is the best free AI red-teaming tool?
Promptfoo's free Community tier has the broadest coverage (up to 10k probes a month); garak (Apache-2.0) and PyRIT (MIT) are fully free and open source.
Which red-teaming tools map to the OWASP LLM Top 10?
Promptfoo has OWASP LLM and agentic presets; DeepTeam maps to OWASP LLM Top 10 2025 and OWASP agentic risks; garak tags probes by OWASP category; Giskard's scan covers OWASP LLM Top 10 categories.
Can these tools red-team MCP servers?
Promptfoo has dedicated MCP plugins and an MCP security testing guide. We didn't find equivalent MCP-specific attacks in the other tools' docs.
Who owns Promptfoo now?
OpenAI. Promptfoo announced on March 9, 2026 that it was joining OpenAI, and says the project remains MIT-licensed open source.

Keep reading

Sources

Checked November 2026.

  1. [1]Promptfoo is joining OpenAI. Promptfoo blog.
  2. [2]OpenAI acquires Promptfoo to secure its AI agents. TechCrunch.
  3. [3]Pricing. Promptfoo.
  4. [4]promptfoo/promptfoo. GitHub.
  5. [5]Red team configuration. Promptfoo docs.
  6. [6]Command line. Promptfoo docs.
  7. [7]MCP server. Promptfoo docs.
  8. [8]Tracing. Promptfoo docs.
  9. [9]NVIDIA/garak. GitHub.
  10. [10]garak CLI reference. garak docs.
  11. [11]garak judge detectors. garak docs.
  12. [12]microsoft/PyRIT. GitHub.
  13. [13]PyRIT documentation 1.1.0. Microsoft.
  14. [14]Pricing. Confident AI.
  15. [15]confident-ai/deepeval. GitHub.
  16. [16]confident-ai/deepteam. GitHub.
  17. [17]Evaluation flags and configs. DeepEval docs.
  18. [18]Prompt optimization: MIPROv2. DeepEval docs.
  19. [19]MCP reference. Confident AI docs.
  20. [20]Pricing. Giskard.
  21. [21]Giskard. Giskard.
  22. [22]Giskard-AI/giskard-oss. GitHub.
  23. [23]Kojen for Agents: REST API & MCP Server. Kojen.
  24. [24]llms.txt. Kojen.
  25. [25]Pricing. Kojen.

Try it on your own test cases.

Run an LLM-judged evaluation with confidence intervals, a red-team pass and a prompt-tuning run on your agent. Free to start, no card needed, then pay as you go.

Start training free