AI, explained
How do I control AI costs?
Almost every surprise AI bill comes from the same place, which is a workflow nobody put a ceiling on.
Measure per task, not per month
Total monthly spend is the least useful number you have. It moves with volume, so it tells you nothing about whether the system got worse. The number that matters is cost per completed unit of work: per resolved ticket, per document processed, per qualified lead. Track it from day one and a regression becomes visible immediately. A prompt change that quietly doubled the context, or a retry loop that now fires on 8 percent of requests, shows up as a step change in that ratio while the monthly total still looks normal.
The four levers, in order of payoff
- Ceilings per workflow. Every automated workflow gets a hard budget and a defined behaviour when it hits: stop, queue, or degrade to a cheaper path. Without this, one bad loop can spend a quarter's budget over a weekend.
- Routing by difficulty. Most production traffic is easy. Classify first with a small fast model, escalate only what genuinely needs the larger one. This is usually the single biggest saving available and it rarely costs accuracy if the escalation rule is honest.
- Caching. Prompt caching on the stable part of a long system prompt or document cuts repeated input cost substantially. A plain result cache on identical questions, common in support and internal search, is cheaper still because it skips the call entirely.
- Context discipline. Retrieval that returns twelve chunks when four would do is paying for tokens on every request forever. Trimming what goes in beats negotiating what comes out.
Two alerts worth having
Alert on cost per task crossing a threshold, and on any single workflow exceeding its daily budget by a set margin. Total-spend alerts fire too late to help. Add a per-user or per-tenant rate limit on anything customer-facing, because bots and enthusiastic power users are a real line item, not a theoretical one.
The trap
Teams optimise the model price and ignore the architecture. Moving to a model 40 percent cheaper saves 40 percent. Removing a redundant call from a three-step chain, or caching the retrieval that runs on every message, often saves more. Look at the call graph before you look at the price list. A short AI audit against your actual traffic tends to find both.
Frequently asked questions
How do I control AI costs?
Put a hard spend ceiling on each workflow with a defined fallback when it is hit, route easy tasks to a cheaper model and escalate only what needs the larger one, cache stable prompt context and repeated answers, and monitor cost per completed task so regressions surface within hours rather than at invoice time.
What drives most AI spend in production?
Input tokens usually dominate, because long system prompts, retrieved documents and conversation history are resent on every single call while outputs stay short. Retry loops and over-broad retrieval are the two most common hidden multipliers, and both are architectural problems rather than pricing problems.
Does a cheaper model always save money?
Not reliably. A weaker model that needs two attempts, produces work a human has to correct, or triggers more escalations can cost more end to end than one good call. Compare cost per successfully completed task, including rework, rather than comparing the price per million tokens.
Related
Ready to put AI to work?
Book a discovery audit and we will map the highest-ROI AI agents and automations for your business.
Book a discovery audit →