A vector database stores embeddings (lists of numbers that represent the meaning of text, images or code) and quickly finds the stored vectors most similar to a query vector. It is how semantic search and retrieval augmented generation (RAG) find relevant passages by meaning rather than by exact keywords.
MongoDB's vector search overview describes the core idea well: "Unlike traditional full-text search which finds text matches, vector search finds vectors that are close to your search query in multi-dimensional space." This guide covers how that works, how the indexes behind it are built, and when you need a dedicated system versus a column in the database you already run.
What is a vector database?
A vector database is a store that treats vectors as first-class data. It does three jobs:
- Stores each vector with an ID and usually some metadata, such as the source file, a tenant ID or a date.
- Indexes the vectors so that finding near neighbors does not mean scanning everything.
- Queries by similarity: given a query vector, return the top matches, optionally filtered by metadata.
The term covers a range of products. Some are purpose-built services; others are extensions or features added to general databases. Supabase's AI docs make the case for the second camp in one line: "The best vector database is the database you already have."
How do embeddings and similarity search work?
An embedding model turns content into a fixed-length vector so that similar content lands close together. Our embeddings guide explains how those models work and how to pick one.
Similarity search then measures distance between vectors. The common measures are cosine distance, inner product (dot product) and Euclidean (L2) distance. The pgvector README lists its operators:
<->for L2 distance<#>for negative inner product<=>for cosine distance
A minimal pgvector setup looks like this:
CREATE EXTENSION vector;
CREATE TABLE chunks (
id bigserial PRIMARY KEY,
source text,
content text,
embedding vector(1536)
);
-- Five nearest chunks by cosine distance
SELECT id, source, content
FROM chunks
ORDER BY embedding <=> '[0.12, -0.03, ...]'
LIMIT 5;
Use the same distance function your embedding provider recommends, and the same one for your index and your queries.
What are vector indexes like HNSW and IVFFlat?
Without an index, the database compares the query with every vector. That is exact nearest neighbor search. The pgvector README says it "provides perfect recall," but cost grows with every row you add.
An index switches to approximate nearest neighbor (ANN) search. In pgvector's words, an index "trades some recall for speed," and "you will see different results for queries after adding an approximate index." Two index types dominate.
HNSW (Hierarchical Navigable Small Worlds). pgvector says an HNSW index "creates a multilayer graph." A search enters at a sparse top layer, hops toward the query, and drops into denser layers to refine the result. The README says it has better query performance than IVFFlat, in terms of the speed-recall tradeoff, "but has slower build times and uses more memory," and it can be built on an empty table because there is no training step. Tuning knobs in pgvector:
m: max connections per layer (16 by default).ef_construction: candidate list size while building (64 by default); higher gives better recall but slower builds.hnsw.ef_search: candidate list size at query time (40 by default); higher gives better recall but slower queries.
IVFFlat. pgvector says it "divides vectors into lists, and then searches a subset of those lists that are closest to the query vector." It builds faster and uses less memory than HNSW but has lower query performance. The README suggests starting with rows / 1000 lists for up to 1M rows and probing sqrt(lists) at query time.
HNSW is common across products. MongoDB says its vector search "supports approximate nearest neighbor (ANN) search with the Hierarchical Navigable Small Worlds algorithm and exact nearest neighbor (ENN) search."
-- HNSW index for cosine distance, with pgvector's defaults spelled out
CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops)
WITH (m = 16, ef_construction = 64);
What should you watch for with filters and limits?
Approximate indexes interact with filters in ways that surprise people. pgvector notes that with approximate indexes, "filtering is applied after the index is scanned." With HNSW and the default ef_search of 40, a filter that matches 10 percent of rows returns only about 4 results on average. Since version 0.8.0, pgvector offers iterative index scans that keep scanning until enough rows match.
Dimension limits also matter. pgvector can index vector columns of up to 2,000 dimensions, halfvec up to 4,000, and binary vectors up to 64,000. MongoDB's docs say its vector search supports embeddings of up to 8,192 dimensions. If your model outputs more dimensions than your index supports, request shorter vectors from the model or use half precision.
For multi-tenant apps, pgvector warns that sharing one approximate index means "vectors from one tenant can affect recall (and speed) for other tenants," and suggests partitioning by tenant.
When do you need a vector database, and when not?
Many projects need less than they think.
| Situation | Reasonable choice |
|---|---|
| A few hundred to a few thousand chunks, one process | Vectors in memory (NumPy or similar) with exact search |
| Knowledge base small enough to fit in the prompt | No retrieval; send the whole thing |
| You already run Postgres and want joins, transactions and filters | pgvector in your existing database |
| You already run MongoDB | MongoDB Vector Search on your existing data |
| Large corpus, many tenants, or no database team | A managed search service |
The "fit in the prompt" row is real. Anthropic's Contextual Retrieval post says: "If your knowledge base is smaller than 200,000 tokens (about 500 pages of material), you can just include the entire knowledge base in the prompt that you give the model, with no need for RAG or similar methods." With today's larger context windows, that threshold can be higher, though quality and cost still favor sending less.
What are pgvector and the hosted options?
Descriptions below come from each project's own docs.
- pgvector. "Open-source vector similarity search for Postgres." It supports exact and approximate search, single-precision, half-precision, binary and sparse vectors, and several distance functions, "Plus ACID compliance, point-in-time recovery, JOINs, and all of the other great features of Postgres."
- Supabase. Its docs describe "an open source toolkit for developing AI applications using Postgres and pgvector," with client libraries to store, index and query embeddings.
- MongoDB Vector Search. Lets you "search and index your vector data alongside your other MongoDB data," with HNSW-based ANN and exact search.
- Pinecone. Its docs describe Pinecone Database as "a search engine for AI agents and applications." Indexes can hold full-text, dense and sparse fields, namespaces keep each tenant's data separate, and with integrated embedding "Pinecone generates the vectors from your text server-side."
Pick based on what you already operate, how much data you have, and whether you need hybrid keyword plus vector search. Benchmark recall on your own queries rather than trusting a generic leaderboard.
How does a vector database fit into RAG?
In a RAG pipeline, the vector database is the retrieval step. You chunk documents, embed each chunk, and store the vectors. At question time you embed the query, fetch the nearest chunks, and put them in the model's prompt. Our RAG guide walks through the full loop.
Vectors alone can miss exact terms. Anthropic's Contextual Retrieval post notes: "While embedding models excel at capturing semantic relationships, they can miss crucial exact matches." Error codes, function names and version numbers are the classic misses. Pairing vector search with keyword search (BM25) and a reranker covers both.
Where should your retrieval prompts live?
The prompt that turns retrieved chunks into an answer deserves the same care as the index. Good ones tell the model to answer only from the supplied passages, cite them, and say when the answer is not there. Kipt keeps prompts like that as Markdown in a private GitHub repo you own, with fill-in blanks for the question and the retrieved context.
What should you read next?
Read what embeddings are to choose a model and dimension size, then what RAG is to see the database inside a full pipeline. If you are deciding between retrieval and training, see fine-tuning vs RAG.