How to vet an AI agency and spot red flags

Verify production work first. Ask for live AI systems you can inspect, a named technical owner, written answers on data handling, and a scoped quote with build versus running costs. Then start with a small paid project before committing to a large one. Builders welcome this; vendors who only demo will resist it.

Guaranteed results, no verifiable references, vague data policies, pressure to sign fast, buzzwords with no plain explanation, and no plan for what happens when the AI is wrong. Any single one warrants caution. Two or more together usually mean the agency cannot deliver production software and is selling a demo.

Scammy AI agencies promise certainty, refuse to show live systems, and rush you past diligence. They charge premium fees for thin wrappers around public APIs, withhold code and data ownership, and disappear after launch. Legitimate agencies show shipped work, explain their stack plainly, and put measurable outcomes in writing.

Ask: What is one AI system you built that is live today? What breaks when it fails? Where does my data go and is it used for training? Who is the technical owner? What does this cost to build versus run? Do I own the code and prompts? Clear, specific answers signal a real builder.

No. AI outputs are probabilistic, so any guarantee of specific results or revenue is a red flag. A credible agency commits instead to measurable targets, accuracy ranges, evaluation methods, and timelines in writing. They will say what success looks like and how it is measured, not promise outcomes no honest builder can control.

Cost depends on scope, but it should map to defined deliverables and milestones, not to the word 'AI'. Expect a written split between one-time build cost and ongoing run cost (models, infrastructure, maintenance). Be wary of flat retainers with no defined output or quotes that hide recurring fees you cannot predict or control.

A demo works on cherry-picked inputs in a controlled setting. A real build runs in production: it handles bad inputs, monitors itself, logs errors, recovers from failure, and is tested on your actual data. Ask any agency to run a small test on your inputs, not theirs. Demos impress; production systems endure.

Ask for live systems and named contacts, then verify independently. Talk to a technical stakeholder, not just the buyer, and ask what broke and how the agency responded. Search for the agency's own products in production. Track record across multiple markets or industries signals they have shipped beyond a single lucky project.

A clear scope with deliverables and milestones, code and data ownership, a data processing agreement, defined build and run costs, acceptance criteria, and a documented handover. It should state what happens to the system if you stop paying. If ownership and exit terms are missing, you are renting a dependency, not buying an asset.

Yes, when jargon replaces clarity. Terms like agentic, RAG, and LLMOps are legitimate, but a good agency explains in plain language what they will build, why, and what it will do for you. If they cannot translate the buzzwords into outcomes and a concrete plan, the complexity is hiding a lack of substance.

Critical. AI systems process your data, so confirm in writing where it is stored, which model providers see it, whether it trains their models, and how it is deleted. For EU operations, require GDPR alignment and a data processing agreement. Vague answers are a red flag, since experienced teams have addressed these questions many times.

Start small. A scoped paid pilot with clear acceptance criteria lets you test the agency's communication, code quality, and delivery before a large commitment. Builders welcome a small first step that proves value. Vendors who push for a big upfront contract and resist a smaller proof are showing you a red flag.

Look for evidence of operational maturity: monitoring, error handling, evaluation pipelines, and uptime they can speak to. Ask how they handle a model giving a wrong answer in production. Agencies that ship their own products, like Digiton with its Parci real-estate platform deployed across markets, have lived with the realities of running AI, not just building it.

It shows live, verifiable systems, names a technical owner, explains its stack and data handling plainly, prices by scope with build and run costs separated, commits to measurable outcomes in writing, and gives you ownership of code and data. It is comfortable starting small and honest about what AI cannot do. Substance over slogans.