AI agent production readiness checklist: 14 checks before real users or real data
An agent that worked in a demo has not yet been tested against hostile input, a runaway loop or a bad Tuesday at 3 a.m. The checks below come from the OWASP list of LLM risks, the NIST AI Risk Management Framework and Anthropic's own advice on building agents, and each one has a test you can run.
AI agent production readiness
- Checks
- 14, each with a pass test
- Checked
- 12 November 2026
- Risk list
- OWASP LLM Top 10, 2025 edition
- Framework
- NIST AI RMF, released 26 Jan 2023
- Stop test
- Agent halted in under a minute
The short answer
An AI agent is production ready when it can only do what it needs to do, a person can stop it in seconds, every action is logged, its cost and volume are capped, and it has passed tests on real cases. Fourteen checks cover this: permissions, approvals, kill switch, evals, injection tests, logs, caps, data handling, rollback and ownership.
What does production ready mean for an AI agent?
Production ready means a person other than the builder can run the agent, watch it, stop it and explain what it did last week. A working demo shows the happy path. Production adds hostile input, bad data, a model that changes under you and a bill that grows with volume.
Anthropic's guide to building effective agents, published on 19 December 2024, says to find the simplest solution possible and only add complexity when needed. It also says agents are best tested extensively in sandboxed environments, with the appropriate guardrails, and that they should include stopping conditions such as a maximum number of iterations. Those two sentences are the spine of this list.
If your agent was built quickly with an AI coding tool and real customers now use it, read this page as a gap list. The production engineering review at Digiton does the same job against the real repository.
The 14 checks
Run them in this order. The first six decide whether the agent may touch real systems at all.
| Check | Pass test | Source |
|---|---|---|
| 1. Least functionality | The agent has only the tools the task needs and no open-ended shell or URL fetch | OWASP LLM06 |
| 2. Least permission | Each tool holds the minimum rights downstream, for example read-only database access | OWASP LLM06 |
| 3. Human approval | High-impact actions wait for a person and the approval is enforced outside the model | OWASP LLM06 |
| 4. Kill switch | One action halts every run, and someone has tested it in the last month | Digiton practice |
| 5. Stopping conditions | A maximum number of steps and a time limit end every run | Anthropic |
| 6. Prompt injection test | Hostile text in a document or web page does not change what the agent does | OWASP LLM01 |
| 7. Evals on real cases | A fixed set of real examples is scored on every change, with a pass mark agreed in advance | Anthropic, NIST |
| 8. Output handling | Anything the model emits is validated before another system runs it | OWASP LLM05 |
| 9. Spend and volume caps | A daily cap on cost and on outbound actions stops a loop | OWASP LLM10 |
| 10. Logs | Every tool call, input and result is stored and searchable by run | NIST |
| 11. Data handling | The data the agent can see and keep is listed and secrets sit in a manager | OWASP LLM02 |
| 12. Rollback | A bad release can be reverted, and a bad action can be undone or reviewed | Digiton practice |
| 13. Ownership | A named person owns alerts, the runbook and the model upgrade decision | NIST |
| 14. Handover | Someone on your team has shipped a change unaided | Digiton practice |
How do I check permissions and approvals?
Functionality. OWASP names excessive functionality as one cause of Excessive Agency, where an agent reaches extensions with more functions than the application needs. List the tools, delete the ones the task does not use and prefer narrow tools over open-ended ones.
Permissions. OWASP names excessive permissions as the second cause. Give each tool the least rights on the downstream system and run actions as the individual user where you can.
Autonomy. The third cause is an application that never verifies or approves high-impact actions. OWASP says to require human approval and to enforce authorisation in downstream systems rather than trusting the model.
What does a kill switch have to do?
A kill switch is useful only if it is fast and tested. Write the pass test as a sentence: from the moment someone decides to stop the agent, no further action leaves the system within one minute.
Digiton's own back office runs on ten scheduled agents, and every outbound action clears a kill switch, a do-not-contact list, an allowlist, dedupe and a daily cap before it leaves. The code for that fleet is public at github.com/Botfather90/digiton-agent-fleet. Copy the shape of it even if you copy none of the code.
Pair the switch with stopping conditions inside each run. Anthropic recommends a maximum number of iterations to maintain control.
How do I test an agent before launch?
Use this order. Each step produces something a reviewer can read.
- Collect 30 to 50 real cases from your own work, including ugly ones, and write the correct outcome for each.
- Run the agent on them in a sandbox with test credentials and score the results. Agree the pass mark before you look at the score.
- Feed it hostile input: a document that says to ignore its instructions, a link to a page with hidden text, a customer message asking for another customer's data. OWASP lists prompt injection first in its 2025 list of LLM risks.
- Raise the load to the volume you expect and watch cost, latency and error rate. OWASP lists unbounded consumption as a separate risk.
- Release to a small share of real traffic behind a flag, read the logs daily for a week, then widen.
What should be logged and who owns it?
NIST's AI Risk Management Framework is voluntary and organises its work into four functions, Govern, Map, Measure and Manage. For one agent you can read that as three questions.
- Who is accountable when it misbehaves, by name.
- What it did, run by run, so a reviewer can replay a decision.
- How you will know it is getting worse, through alerts on error rate, cost and refusals.
NIST released the framework on 26 January 2023 and its Generative AI Profile on 26 July 2024. The NIST page says AI RMF 1.0 is being revised, so check the current text before you cite it in a policy.
Where Digiton fits
Digiton is an AI consulting firm that builds agents and takes systems built with Claude, Cursor or Lovable into production. The production review reads your real repository and gives you a scorecard across ten layers with evidence for each, then a fix list.
- A new agent goes through AI agent development.
- A custom system around it goes through custom AI development.
- Policy, owners and review rules sit in AI agent governance.
Related guides: how to build an AI agent for business, MCP security best practices and the vibe coded app audit.
Frequently asked questions
What is the most common reason an AI agent is not production ready?
Permissions. OWASP names excessive functionality, permissions and autonomy as the three causes of Excessive Agency, and most agents built quickly hold more rights than their task needs.
Do I need human approval for every action?
No. OWASP asks for human approval on high-impact actions. Low-impact, reversible actions can run unattended if they are capped, logged and covered by a kill switch.
How many test cases are enough?
Enough to cover your real variety, including the ugly cases. Thirty to fifty real examples is a practical start, and the set should grow every time the agent fails in production.
Is the NIST AI RMF mandatory?
No. NIST says the framework is intended for voluntary use. It is a useful vocabulary for ownership and review.
Sources, checked 2026-11-12
5 sources
- OWASP's 2025 Top 10 for LLM Applications lists LLM01 Prompt Injection, LLM05 Improper Output Handling, LLM06 Excessive Agency and LLM10 Unbounded Consumption, among ten entries.Source: genai.owasp.org, checked 2026-11-12.
- OWASP says Excessive Agency has three root causes, excessive functionality, excessive permissions and excessive autonomy, and recommends minimum extensions and permissions, human approval for high-impact actions and authorisation enforced in downstream systems.Source: genai.owasp.org, checked 2026-11-12.
- Anthropic's guide, published 19 December 2024, recommends the simplest solution possible, extensive testing in sandboxed environments with appropriate guardrails, human checkpoints, and stopping conditions such as a maximum number of iterations.Source: anthropic.com, checked 2026-11-12.
- NIST released the AI Risk Management Framework on 26 January 2023 for voluntary use, released its Generative AI Profile on 26 July 2024, and says AI RMF 1.0 is being revised.Source: nist.gov, checked 2026-11-12.
- Digiton's public agent fleet repository describes ten scheduled agents whose actions pass a kill switch, do-not-contact list, allowlist, dedupe and daily cap.Source: github.com, checked 2026-11-12.
Digiton Dynamics OÜ is registered in Tallinn, Estonia, and delivers from Lisbon.
Update log
- Published. OWASP LLM Top 10 pages, Anthropic's agent guide and the NIST AI RMF page read on 12 November 2026.
Each source above was opened on the date beside it. How facts get chosen and checked is written down in the editorial policy.
Talk to an engineer about your system
Book a free 25-minute scoping call. We tell you what we would build, what it takes to keep it running, and whether we are the right fit.