AI, explained
How do I migrate off an AI vendor?
Leaving an AI vendor is easy or impossible, and which one it is was decided when the contract was signed, not when you decided to leave.
The four assets that make migration possible
- Prompts and configuration. Every system prompt, tool definition, routing rule and threshold, in a repository you control.
- Evaluation sets. The questions with known correct answers, plus the recorded results. Without these you cannot prove the replacement is as good, so the migration becomes a matter of opinion and stalls.
- Source data and the retrieval index. The documents are usually yours. The chunking, embeddings and metadata schema often are not, and rebuilding them is the slow part.
- Traces. Historic inputs, retrieved context and outputs. These are both a regulatory record and the best material for building an evaluation set if you never made one.
The sequence that works
Export first, while the relationship is still cordial and the export clause still means something. Then rebuild behind the same internal interface so downstream systems do not change. Then run both systems on the same live traffic in shadow mode, comparing against the evaluation set, for long enough to cover the awkward cases. Then cut over one workflow at a time, keeping the old path available for a defined period. Big-bang cutovers on AI systems fail in the tail, which is exactly where the complaints come from.
What usually goes wrong
Three things. The evaluation set does not exist, so nobody can say whether quality dropped, and the migration is judged on anecdotes. The retrieval index metadata was implicit in the vendor platform and has to be reverse engineered. And the contract has no export clause for prompts and traces, only for raw data, which leaves you with documents and no system.
Preventing it next time
Put four lines in the next contract: you own the prompts and configuration, you receive evaluation sets and results, you can export traces in a machine readable format on demand, and termination assistance includes a documented index schema. Build behind your own abstraction so a model or vendor change is a configuration change plus a regression run. Digiton builds this way as standard, and an AI audit on an existing system will tell you which of the four assets you currently do not have.
Frequently asked questions
How do I migrate off an AI vendor?
Recover prompts and configuration, evaluation sets with results, source data and the retrieval index, and historic traces. Rebuild behind the same internal interface, run both systems in shadow mode against the evaluation set, then cut over workflow by workflow while keeping the old path available for a defined period.
What causes AI vendor lock-in?
Rarely the model. It is the assets around it: prompts held in the vendor platform, evaluation sets that were never yours, retrieval index metadata that exists only inside their product, and traces you cannot export. Without those, switching means rebuilding and revalidating from scratch rather than transferring.
How long does an AI vendor migration take?
For one well-instrumented workflow with existing evaluation sets, a few weeks including a shadow period. Without evaluation sets, add time to build them from historic traces before anything can be judged. The rebuild is usually not the slow part. Proving quality did not drop is.
Related
Ready to put AI to work?
Book a discovery audit and we will map the highest-ROI AI agents and automations for your business.
Book a discovery audit →