AI agent red teaming tools in 2026: what actually catches agent failures

An AI agent handed over personnel records for more than 150 people to someone holding one ordinary user credential. Another produced a signed bank guarantee for 125,000 euros that had never been issued, in a named bank's name, and stripped the marking that flags a file as generated when asked to. A third let a file containing 30 named people and a confidential purchase agreement be downloaded with no login at all.

None of those failures is predictable from a feature list. That is the problem with the way AI red teaming tools are usually compared. You get a table of checkmarks, and the checkmarks describe what a tool probes for rather than what it can actually see happening inside your agent.

This article compares seventeen tools a different way. It sorts them by the job they do, takes a position on what each is good at and where it falls down, and asks of each one the only question that matters: would it have caught something like the above?

Why comparing these tools is harder than it looks

Three things make an honest comparison difficult. The tools do different jobs, so a single ranking is misleading. The "agentic" claims have run well ahead of the capability, so the marketing rarely tells you what a tool actually observes. And in the last two years roughly a third of the category changed owner, so some names on 2025-era lists no longer exist independently.

On that last point, two years of acquisitions:

An AI security tool owned by a frontier model vendor has a structural conflict when you use it to test that vendor's models, or to choose between vendors. That lands on Promptfoo, the most-used tool in the category, now owned by OpenAI. Promptfoo says it will keep supporting a diverse range of providers. That is a promise, not a governance structure. It also remains open source, so the fork option is real. Weigh it; it is a question, not a disqualification.

Most of these tools are not competing with each other

Work out which of four jobs you are hiring for before you look at any names.

Open-source probe libraries

Free, model-level, you bring the engineering. Garak, PyRIT, DeepTeam, and UK AISI Inspect. The best value in the category if you have a security engineer with time.

Developer testing in CI

Red teaming wired into pull requests. Promptfoo. The lowest setup cost and the strongest ergonomics here.

Agent adversarial testing platforms

Testing is the product. Enoki, Mindgard, General Analysis, Adversa AI, Giskard, SPLX, and Straiker.

Runtime protection with a scanner attached

You buy them to block attacks in production, and a testing module comes along. Lakera and Check Point, Lasso, HiddenLayer, Cisco AI Defense, and Prisma AIRS.

If what you need is a firewall in the request path, do not buy a testing platform; go look at Lakera. If what you need is confidence your agent holds before launch, a runtime guardrail does not give you that.

The five questions that separate them

If you read nothing else, take these into every vendor call. They separate a tool that tests an agent from a scanner that tests a model with agentic marketing on top.

1. Does it reach past the model to the tools, memory, and permissions? The worst agent failures are things the agent did, not things it said. A tool that only reads the model's text output cannot see them.
2. Can an attack arrive through a channel the user never typed into? Real attacks ride in on a document, an email, or a tool output. PyRIT's XPIA workflow plants the payload in real external storage, Garak has a latent-injection family, and most commercial tools only simulate it inside the prompt.
3. Does it hold state across one real conversation, or restart every turn? The dangerous jailbreaks build over several turns. Garak's agent_breaker, for one, starts a fresh conversation each turn and misses them.
4. Can you fix a finding and re-run the exact attack to confirm it held? A one-shot report is worth little if you cannot verify the fix and catch it when it comes back.
5. Who owns it, and do they also sell what you are testing? Independence is now a purchasing variable, not a footnote.

Seventeen AI red teaming tools in 2026

One read per tool: what it is good at, where it falls down, and who owns it.

1. Enoki (Belgium)

• Good at: driving a tool-using agent end to end. It generates attacks for that specific agent, runs them multi-turn, and shows which tools fired, across both security and safety.
• Not for: blocking attacks in production. It is a testing platform, not a runtime guardrail, and it is the newest and smallest name here.
• Owner: independent.

2. Mindgard (Boston and London)

• Good at: the strongest agentic story of the commercial set, with autonomous attack workflows and MCP and A2A discovery.
• Not for: buyers who want to scope before a sales call. No published probe count and no standards mapping.
• Owner: independent ($30M Series A, Aug 2026).

3. General Analysis (San Francisco)

• Good at: the most credible agent methodology on the list. It maps the tool graph and builds multi-step exploits; of the 55 live customer-service agents it tested, its adversarial agent talked 50 into handing over more than $10M in fabricated perks; only 5 refused.
• Not for: teams that need scale or a big-vendor name behind them. Small team, $10M seed.
• Owner: independent (YC S24).

4. Adversa AI (Tel Aviv)

• Good at: adversarial AI with a research pedigree. One of the earliest names in the space, with published LLM and multimodal attacks behind a red-teaming offering.
• Not for: teams that want a track record on tool-using agents. The research pedigree is long, the agentic platform is recent.
• Owner: independent.

5. Giskard (Paris)

• Good at: the best EU open-source starting point, with real standards work at AFNOR, CEN-CENELEC, and ISO, and customers including AXA and BNP Paribas.
• Not for: tool-using agents. Its own scope is conversational, text-to-text, tested through an API endpoint, and the free tier stops at 2024-era techniques.
• Owner: independent (open-source library plus a platform).

6. Promptfoo (San Francisco, now OpenAI)

• Good at: developer ergonomics. The best red teaming you can wire into CI, with a large plugin ecosystem and the biggest community in the category (24k+ stars).
• Not for: proving a tool-using agent holds. Its MCP plugin's depth is unclear from public docs, so ask on a call.
• Owner: OpenAI (acquired Mar 2026), still MIT. Owned by a model vendor, which is a conflict when you test or choose models; the fork option is real.

7. NVIDIA Garak (Santa Clara)

• Good at: the best free breadth at model level, 180 probe classes across 41 families, including a strong latent-injection family.
• Not for: agents. Its agent_breaker probe restarts a fresh conversation each turn, and a proposal for real tool-calling probes went stale and was auto-closed.
• Owner: NVIDIA (Apache 2.0, free).

8. Microsoft PyRIT (Redmond)

• Good at: the best indirect-injection design anywhere. Its XPIA workflow plants the payload in real external storage, not just inside the prompt.
• Not for: teams that want a product. It talks to your agent as a black box over HTTP, has no idea which tools fired, and expects you to write code.
• Owner: Microsoft (MIT, free). The repo moved to microsoft/PyRIT, so most online tutorials are stale.

9. Confident AI DeepTeam (San Francisco)

• Good at: the richest agentic taxonomy of the open-source set, with a callback that can read back tools_called and retrieval_context, the closest any open-source tool gets to instrumentation.
• Not for: hands-off use. It never invokes a tool itself; you write the adapter and supply the execution.
• Owner: Confident AI (Apache 2.0, free).

10. UK AISI Inspect

• Good at: being the one option here with nothing to sell you. A neutral, government-backed open-source evaluation framework from the UK AI Security Institute, with real support for agent evaluations.
• Not for: teams wanting a turnkey attacker. It is a framework you build evaluations on, aimed at evaluation more than generating attacks.
• Owner: UK AI Security Institute (open source).

11. Straiker (Mountain View)

• Good at: an agent-native platform (discover, test, protect) with the deepest pockets of the pure-plays.
• Not for: buyers who want capability detail up front. Public detail is thin; expect a call.
• Owner: independent ($64M Series A, $85M total).

12. SPLX (now Zscaler)

• Good at: the broadest compliance mapping of anyone here, covering MITRE ATLAS, NIST AI RMF, OWASP, the EU AI Act, ISO 42001, and DORA.
• Not for: tool-using agents, or buyers who need certainty they can still buy it standalone. The probe product's scope is conversational; the agentic framing is Zscaler's PR.
• Owner: Zscaler (acquired Nov 2025).

13. Cisco AI Defense (formerly Robust Intelligence)

• Good at: being the safe enterprise default, with a free Explorer Edition (Mar 2026) that added agentic framework support and adaptive multi-turn tests.
• Not for: teams wanting a specialist. It is broad and bundled rather than deep.
• Owner: Cisco (acquired Robust Intelligence, Aug 2024).

14. Lakera (San Francisco, now Check Point)

• Good at: runtime detection. The reference implementation, with >98% detection, sub-50ms latency, and 80M+ adversarial patterns from Gandalf.
• Not for: pre-deployment agent testing. Red teaming is the junior sibling; its strength is the request path.
• Owner: Check Point (acquired Sep 2025).

15. Prisma AIRS (formerly Protect AI, Palo Alto Networks)

• Good at: breadth across the AI supply chain, with the well-known Protect AI open-source tools (LLM Guard, ModelScan, Rebuff) and a bug-bounty community behind it.
• Not for: deep, agent-specific adversarial testing. Its strength is scanning and guardrails, not driving a tool-using agent.
• Owner: Palo Alto Networks (acquired Protect AI in 2025; now Prisma AIRS).

16. HiddenLayer (Austin)

• Good at: the deepest model-file scanning on the list, covering malware, backdoors, and serialization attacks.
• Not for: teams whose main need is agent red teaming. It is a model-artifact scanner first; the agentic red teaming is a newer addition.
• Owner: independent ($50M Series A in 2023, no announced follow-on).

17. Lasso Security (Tel Aviv)

• Good at: the most explicit multi-turn agentic testing language of any vendor, with a claimed 3,000+ attack types.
• Not for: buyers who take claims at face value. It has the highest claim-to-capital ratio on the list, so pressure-test it.
• Owner: independent ($6M seed, Nov 2023, no round since).

Adjacent, and why they are not on the list

A few names you might expect are missing on purpose. Zenity and Noma do posture management and runtime enforcement, not red teaming. Pillar Security, Virtue AI, and Repello are real testing products, left off here on size or recency. And if you arrive from an older list: CalypsoAI is now F5 AI Red Team, Robust Intelligence is now Cisco AI Defense, and Aim Security is inside Cato.

Comparison matrix

In list order, not ranked. "Indirect injection" is whether a tool can plant an attack in a real external channel, like a document or an email, or only simulates one inside the prompt. "Unverified" means we could not confirm it from public sources.

When to definitely use Enoki

These are the cases we are built for.

• Your agent can call tools, reach data, or act on a user's behalf, and you need to know what it does under attack, not just what it says.
• You want attacks built for your specific agent, run multi-turn and adaptively, rather than a fixed checklist pulled from a library.
• You need to reproduce a finding, fix it, and re-run the exact attack to confirm it held.
• You ship changes often and want the agent tested on every release, not once at launch.
• You care about both sides: security failures like leaks and tool abuse, and safety failures like advice the agent should not give.

When not to use Enoki

And the cases where we are the wrong choice. If one of them is you, go elsewhere.

• Testing a chatbot or a RAG app with no tools, on no budget: download Garak or Giskard's open-source library. Free will do the job.
• Wiring tests into pull requests as a developer workflow: Promptfoo has better ergonomics and a bigger community than anything here, us included.
• Needing something in the request path blocking attacks in production today: that is a guardrail, and Lakera or Lasso is the shape of product you want.
• Deploying third-party or open-weight model files and worried about what is inside them: HiddenLayer first, and Prisma AIRS's ModelScan if you are already on Palo Alto.
• Procurement that will not approve a company founded in 2026: fair, and Cisco or Zscaler is the low-risk answer.

What we would ask every vendor on this list, including us

Paste these into a demo call. They are the fastest way to tell a real agent tester from a text-to-text scanner with agentic marketing.

1. Test a tool-calling agent with persistent memory, live, on the call. Show me which tools fired.
2. Show me an attack that arrives through a document or an email, not through the chat box.
3. Show me the same conversation across six turns, not six fresh conversations.
4. Show me a finding with reproduction steps, then show me the re-run after a fix.
5. How many of your attacks are written for my agent versus pulled from a fixed library?
6. Who owns you, and do they sell a model I might be testing?

Where this category goes next

Three things. The agentic marketing and the agentic capability will converge, because buyers are starting to test the claim on demo calls. Consolidation is not finished, and independence is now a purchasing variable. And testing is moving from a launch gate to a per-release step.