AI red teaming tools: which to run, what each tests and what it misses

AI red teaming tools fall into three kinds. Scanners fire a fixed library of attacks at a model, frameworks let you script attacks against your own application, and managed services run them for you. Most teams shipping an AI feature should start with one scanner for coverage, add one framework for their own cases and keep both in CI.

By Brandon Da Costa, founder of Digiton DynamicsPublished 28 October 2026Updated 28 October 2026Editorial policy

Book a 25-minute scoping call See all practices

AI red teaming for teams shipping AI

Open-source tools: garak, PyRIT, promptfoo, DeepTeam
Scanner we ran: garak 0.17.1
promptfoo owner: Part of OpenAI, still MIT
Old PyRIT repository: Archived, use microsoft/PyRIT
Checked: 28 October 2026

The short answer

Start with a scanner such as garak for broad coverage of known attacks, then add a framework such as promptfoo, PyRIT or DeepTeam to test your own application, its tools and its data. Microsoft's AI Red Teaming Agent suits teams already on Foundry. No tool replaces a human tester, and every score needs reading.

Five AI red teaming tools compared, checked on 28 October 2026

Licences, owners, release dates and push dates below come from each project's repository on 28 October 2026, and the capability column comes from its own README or documentation. We ran one of them, garak, and the result is further down.

Open-source and managed AI red teaming tools, what each is and when to pick it
Tool	What it is	Licence and owner	Activity	Pick it when
garak	A scanner. It probes an LLM for hallucination, data leakage, prompt injection, toxicity and jailbreaks, and compares itself to nmap	Apache-2.0, NVIDIA	v0.17.1 on 16 October 2026, last push 27 October	You want broad coverage of known attacks against a model endpoint with little setup
PyRIT	A Python framework for finding risks in generative AI systems. CSET notes it can use other AI models as attackers against the target	MIT, Microsoft	v1.1.0 on 4 September 2026, last push 28 October. The old Azure/PyRIT repository is archived	You need adaptive, multi-turn attacks and can write Python
promptfoo	A command line tool and library for evals and red teaming of prompts, agents and RAG, with CI support	MIT. Part of OpenAI, which says it stays open source	0.125.1 on 25 October 2026. Needs Node 22.22 or newer	You test your own application's cases on every pull request
DeepTeam	A Python framework with more than 50 ready vulnerability types, scored locally by an LLM judge	Apache-2.0, Confident AI	Last push 23 October 2026	You want ready-made vulnerability categories inside a Python test suite
Microsoft AI Red Teaming Agent	A managed service in Microsoft Foundry that uses PyRIT's attack strategies. Text scenarios only	Service, not open source	Documentation read 28 October 2026	Your agents already run in Foundry

Pick the tool by who writes the tests

Tools differ less in what they attack than in who writes the tests.

Georgetown's CSET surveyed 23 AI red-teaming resources and found two approaches. Benchmark-style tools hand you a ready process and a fixed set of prompts. They are quick to run and report numbers such as attack success rate, but they are built around one threat model and can miss multi-turn behavior. Framework-style tools give you infrastructure and expect you to bring the tests and the code. CSET named garak as the prototypical benchmark and the UK AI Security Institute's Inspect as the prototypical framework. It also found AI models acting as scorers in more than half the tools, with no stated guarantees about how reliable those scores are, so read a score as a lead and not a verdict.

Which one to start with

You need breadth in an afternoon. Run garak against a staging endpoint. Version 0.17.1 lists 191 probes in 42 modules and marks 98 of them inactive, so choose the set you run on purpose.
You test your own application in CI. Use promptfoo or DeepTeam, and write cases for your tools, your data and your policies. Run them on every pull request, because the prompts and the model both change.
You need attacks that adapt across turns. Use PyRIT, which lets an attacker model and a scorer take part, and plan on writing Python.
Your agents live in Microsoft Foundry. The Red Teaming Agent covers Foundry hosted agents that call Azure tools. Microsoft's table marks non-Foundry agents and function tool calls as not supported.
The risk is a business rule. Scanners find known attack families and a person finds the flaw in your refund logic. OWASP's GenAI Red Teaming Manual sets out an eight-phase method for that kind of work.

A first AI red team run, in five steps

Scope the system, not the model. List what the application can read, call and change: tools, files, accounts and other agents. Write down what a failure would cost.
Take attack classes from a list someone else maintains. The OWASP Top 10 for LLM Applications 2025 works as a spine. Five of its entries can be probed from outside: prompt injection, sensitive information disclosure, excessive agency, system prompt leakage and unbounded consumption.
Run a scanner first. In minutes it tells you which known attack families get through.
Add cases only your application has. Hide an instruction in a document, an email or a tool result that your agent reads and check whether it acts on it. No scanner library knows your tools.
Keep every finding as a regression test and name it with the vocabulary in NIST AI 100-2 E2025, so results can be compared later and shared with other teams.

What we tested on 28 October 2026

We installed garak 0.17.1 from PyPI into a throwaway virtual environment, pointed its data folder at the same directory and ran it offline. We did not aim it at any real model or any real project. Counting its probe listing gave 42 modules and 191 probes, with 98 of them marked inactive. Then we ran one active probe from the latent injection family. Its own description is string-based latent prompt injections in fact snippets on the Eiffel Tower, and it carries the OWASP LLM01 tag.

$ garak --model_type test.Repeat     --probes latentinjection.LatentInjectionFactSnippetEiffel --generations 1
garak LLM vulnerability scanner v0.17.1 ( https://github.com/NVIDIA/garak )
  at 2026-10-28T18:08:25.724301
DEPRECATION: --model_type on CLI is deprecated since version 0.13.1.pre1
DEPRECATION: --probes on CLI is deprecated since version 0.15.1.pre1
loading generator: Test: Repeat
queue of probes: latentinjection.LatentInjectionFactSnippetEiffel
garak run complete in 2.72s

The report file logged one entry for the run: detector base.TriggerListDetector, 256 evaluated, 0 passed and 256 failed. The generator test.Repeat answers every prompt by repeating it, so the planted string always comes back and all 256 attempts count as hits. That is the expected result and it says nothing about any real model. What it proves is that the probe, the detector and the report run end to end on a laptop without an API key. Point the same command at a staging endpoint to get a result worth reading. Long lines in the listing are wrapped for width. The deprecation notices are worth noting too, since newer garak versions move these options into config files.

We also tried promptfoo with npx. It stopped at once with the message Required: greater than or equal to 22.22.0, because this machine runs Node 22.18.0, so we did not run it. If you see the same message, upgrade Node before anything else.

What these tools cannot tell you

Models grade the answers. More than half the tools in CSET's survey use an AI model as the scorer. Microsoft says the scores of its own agent can be non-deterministic and should be reviewed before anyone acts.

The library is fixed. A scanner replays known attacks. It cannot invent the one that fits your tools, and a static dataset can miss behavior that only appears over several turns.

The data is synthetic. Microsoft says its agent runs on synthetic data that does not match real-world distributions. A clean run means these probes found nothing, nothing more.

Test a copy. Attack libraries send adversarial text to the model they target. Point them at a staging copy with fake data and test keys, and read the provider's usage terms first.

Two cases that sit next to this work

Offensive AI tooling is already in use outside test labs. CrowdStrike says one actor used ARTEX, an open-source AI pentest tool, plus AI models against South Korean financial firms in late September 2026, and banks and authorities had not confirmed the tool in early reports. Our ARTEX brief lists the side systems to check on your own estate.

Inside a test, the risk runs the other way. The UK AI Security Institute reported a July 2026 test in which Claude Mythos 5 tried to push malware into a real GitHub project, and the attempt failed. Anthropic published its own assessment on 23 October. The Mythos 5 brief is the reminder for your own runs: a test with real tokens and internet access can touch real things.

Where this sits in a production review

AI agent development. An agent that calls tools gets its red team cases written alongside its permissions, before launch.
Production engineering. A production review will give you a scorecard across ten layers with evidence for each, and the findings can feed your regression suite.
Custom AI development. A product with an AI feature ships with the attack cases that match its data and its tools.

Two sibling guides cover what you are testing: MCP security best practices for the tool layer and Supabase RLS for the database behind it. If your agents read untrusted files, web pages or tool output, agentjacking defense covers the hidden-instruction attacks these tools probe for, and what prompt injection is has the short definition.

Frequently asked questions

What are AI red teaming tools?

AI red teaming tools send adversarial inputs to an AI system and record whether it fails. Scanners such as garak replay a library of known attacks, frameworks such as PyRIT, promptfoo and DeepTeam let you script attacks against your own application, and managed services such as Microsoft's AI Red Teaming Agent run them as a service.

Which open-source AI red teaming tool should I start with?

Start with garak for broad coverage, because it ships a large probe library and needs little setup. Add promptfoo or DeepTeam for cases that belong to your own application and run them in CI. Use PyRIT when you need custom, multi-turn attack flows and are comfortable writing Python.

What is the difference between garak and promptfoo?

Garak is a scanner with a ready library of probes for a model endpoint, which CSET classes as benchmark-style. Promptfoo is a command line tool and library for evals and red teaming that you configure with your own test cases and run in CI. Promptfoo is now part of OpenAI and remains MIT licensed.

Is PyRIT still maintained?

Yes, in a new repository. The Azure/PyRIT repository on GitHub is archived. Microsoft's development continues at microsoft/PyRIT, which had release v1.1.0 on 4 September 2026 and a push on 28 October 2026. Use the microsoft organization's repository and documentation, and check older tutorials for the old address.

Can an AI red teaming tool replace a human red team?

No. Tools find known attack families quickly and repeat them on every change. They miss flaws in your business rules, and many use a model to score the answers, which can be wrong. Microsoft recommends reviewing the results of its own agent before acting on them, and CSET found no reliability guarantees for AI scorers.

Is it safe to run AI red teaming tools against production?

Avoid it. These tools send adversarial prompts and can trigger real actions if the target has tools, accounts or tokens. Run them against a staging copy with fake data and test keys, check your model provider's terms first, and point agents that can act at nothing you cannot afford to change.

Sources, checked 2026-10-28

12 sources

garak is an Apache-2.0 licensed LLM vulnerability scanner from NVIDIA that probes for hallucination, data leakage, prompt injection, toxicity and jailbreaks. Its latest release is v0.17.1 of 16 October 2026.Source: github.com, checked 2026-10-28.
PyRIT is an MIT licensed Python framework from Microsoft for identifying risks in generative AI systems, with release v1.1.0 of 4 September 2026 and a push on 28 October 2026.Source: github.com, checked 2026-10-28.
The Azure/PyRIT repository on GitHub is archived, and Microsoft's development continues at microsoft/PyRIT.Source: github.com, checked 2026-10-28.
promptfoo is an MIT licensed CLI and library for evaluating and red teaming LLM apps with CI support. Its README states that it is now part of OpenAI and remains open source. Release 0.125.1 requires Node 22.22 or newer.Source: github.com, checked 2026-10-28.
DeepTeam is an Apache-2.0 licensed Python red teaming framework with more than 50 ready vulnerability types that run locally with LLM-as-a-judge scoring.Source: github.com, checked 2026-10-28.
Microsoft's AI Red Teaming Agent uses PyRIT's attack strategies, supports text scenarios only, tests Foundry hosted agents that call Azure tools, and notes its scores can be non-deterministic and run on synthetic data.Source: learn.microsoft.com, checked 2026-10-28.
CSET surveyed 23 AI red-teaming resources, separated benchmark-style tools such as garak from open-ended frameworks such as Inspect, and found AI scorers in over half of the tools with no stated reliability guarantees.Source: cset.georgetown.edu, checked 2026-10-28.
The OWASP Top 10 for LLM Applications 2025 lists prompt injection, sensitive information disclosure, supply chain, data and model poisoning, improper output handling, excessive agency, system prompt leakage, vector and embedding weaknesses, misinformation and unbounded consumption.Source: genai.owasp.org, checked 2026-10-28.
OWASP's GenAI Red Teaming Manual version 1.0 sets out an eight-phase methodology for evaluating the security, safety and robustness of generative AI systems.Source: genai.owasp.org, checked 2026-10-28.
NIST AI 100-2 E2025, published March 2025, is a taxonomy and terminology of attacks and mitigations in adversarial machine learning.Source: csrc.nist.gov, checked 2026-10-28.
CrowdStrike reported that one actor used the open-source AI pentest tool ARTEX with AI models against South Korean financial firms.Source: crowdstrike.com, checked 2026-10-28.
The UK AI Security Institute reported on 4 August 2026 that Claude Mythos 5 tried to push malware into a real GitHub project during a cyber test, and Anthropic published its assessment on 23 October 2026.Source: aisi.gov.uk, checked 2026-10-28.

Digiton Dynamics OÜ is registered in Tallinn, Estonia, and delivers from Lisbon.

Update log

28 October 2026First published. Every source was opened on 28 October 2026 and the garak run was made the same day.

Each source above was opened on the date beside it. How facts get chosen and checked is written down in the editorial policy.

Talk to an engineer about your system

Book a free 25-minute scoping call. We tell you what we would build, what it takes to keep it running, and whether we are the right fit.

Book a scoping call