A vector database or a library plus your own storage
You need nearest-neighbour search over embeddings. Do you run a dedicated vector database, or add vectors to the store you already have?
The short answer
Start in the database you already run. Move to a dedicated store when scale or filtered search forces it, and not because the architecture diagram looked bare.
Your existing database
Good at
- One system to back up, secure and operate, not two.
- Filters and joins alongside the vector search, in the same transaction and query language.
- No sync problem: the document and its embedding cannot drift apart.
- Postgres with pgvector handles low millions of vectors comfortably on ordinary hardware.
- The team already knows how to run it.
Costs you
- Index build and recall tuning are less mature, and HNSW builds can be slow.
- Vector search competes with your transactional workload for the same memory and CPU.
- Scaling past tens of millions of vectors gets uncomfortable and then painful.
- Fewer built-ins: hybrid search, reranking and metadata filtering may be yours to write.
Pick this when: Under roughly ten million vectors, you already run Postgres, and you want one thing to operate. This is most teams.
A dedicated vector database
Good at
- Hundreds of millions of vectors, with sharding and replication that are someone else's problem.
- Fast filtered search, which is where naive implementations fall apart.
- Hybrid dense-plus-sparse retrieval and reranking as features rather than projects.
- Rebuilding or swapping indexes without downtime.
Costs you
- A second datastore: its own operations, backups, access control and bill.
- A sync pipeline between source of truth and index, and the staleness it introduces.
- Lock-in to a query model and an index format that are not portable.
- Cost that is often dominated by keeping the index resident in memory.
Pick this when: Scale, filtered search at scale, or hybrid retrieval you do not want to build and maintain yourself.
A library in your own process
Good at
- Prototypes and offline evaluation, where a FAISS index in memory is the whole answer.
- Small static corpora — tens of thousands of chunks needs no server at all.
- Zero network latency and complete control over the index.
Costs you
- No durability, no replication, no concurrent writers: it is a data structure, not a database.
- Updates usually mean rebuilding, which stops being cheap fast.
- Every instance holds its own copy, so memory scales with replicas.
- You are now writing the persistence and filtering layer nobody budgeted for.
Pick this when: Prototypes, evaluation harnesses, and small corpora that change rarely.