AI, explained

What is a vector database?

It is a database that searches by meaning instead of by matching words, which is what makes an AI assistant able to answer from your own documents.

Traditional search is literal. Ask a keyword system for "how much holiday do I get" and it looks for those words. If your handbook says "annual leave entitlement", the search fails, even though the document plainly contains the answer. A vector database fixes this by storing meaning rather than spelling.

How it actually works

An embedding model reads a chunk of text and outputs a vector, typically several hundred to a few thousand numbers. Similar meanings produce similar vectors. The database indexes those vectors so it can find nearest neighbours quickly, usually with an approximate algorithm such as HNSW, because comparing a query against millions of vectors exactly would be slow. At query time you embed the question, ask for the closest chunks, and hand those chunks to the language model as context. That pipeline is what people mean by RAG, retrieval augmented generation.

When you genuinely need one

• Large or growing corpora. Thousands of pages of policy, contracts, tickets, or product documentation that will not fit in a prompt.
• Answers that must cite a source. Retrieval gives you the passage the answer came from, which is what makes an assistant auditable.
• Content that changes. Re-embedding a changed document is cheap. Retraining a model is not.

When you do not

Plenty of projects reach for a vector database and do not need it. If your whole knowledge base is fifty pages, put it in the prompt. If your questions are structured ("what did we invoice this client in June"), you want SQL, not similarity search. And if your documents are already well organised with reliable metadata, a keyword search combined with filters often beats a naive vector setup. The strongest production systems usually run hybrid retrieval, combining keyword and vector scores, because each catches what the other misses.

Choosing one

The options divide into dedicated engines and extensions to databases you already run. Postgres with pgvector is the pragmatic default for most businesses: no new system to operate, transactional consistency with the rest of your data, and performance that is more than adequate at typical corpus sizes. Dedicated vector engines earn their place at very large scale or when you need specialised indexing behaviour. The honest guidance is to start with the database you already have and move only when you can measure a reason to.

The harder problem is rarely the database. It is chunking, metadata, and evaluation: how you split documents, what you attach to each chunk, and how you measure whether retrieval is actually returning the right passage. Teams that skip evaluation ship a system that sounds confident and retrieves the wrong page. A short AI audit usually surfaces whether your content is ready for retrieval before anyone builds the pipeline.

Frequently asked questions

Related