AI Glossary 06 - Embeddings and Vector Search

Ask a search system for pages about “hiring policy” and it needs some way to match that phrase against documents that never use those words. Embeddings are that bridge, and this sixth glossary lesson collects the embedding terms around them: what the vectors are, how search runs over them, and the reranking vocabulary wrapped around the results. The site covers the storage layer in depth in How Vector Databases Work; the entries here stay at a sentence or two each.

The core representation

Four terms for the vectors themselves and how models learn to place them.

• Embedding: A list of numbers that positions a piece of text as a point in a space with hundreds or thousands of dimensions, built so meaning decides distance: related passages land near each other. Search, clustering, and classification all turn into geometry over these points.
• Embedding model: A network, usually a transformer encoder, trained to produce those vectors with a fixed output dimension such as 384 or 1,536. It encodes each passage on its own, which is what makes indexing a corpus ahead of time possible; the MTEB benchmark suite tracks how the current crop scores across tasks.
• Contrastive learning: The training scheme behind them: pull a passage and its matching query together in the space while pushing thousands of mismatches apart. No labels beyond pairs are required, and the query and document encoders can even share weights.
• Hard negatives: Training pairs chosen because they look relevant without being the answer: same vocabulary, wrong meaning. A model trained mostly on easy negatives scores well in the lab and then retrieves superficially similar junk in production.

From vectors to search

Four terms for the query path: measuring nearness, storing at scale, and keeping it fast.

• Similarity metric: The function that scores two vectors against each other: cosine similarity measures angle, dot product also rewards length, Euclidean distance measures the straight line. With normalized vectors the three nearly agree, which is why the choice rarely leads a design discussion.
• Vector database: A store built around nearest-neighbor queries: insert millions of vectors, get the closest handful back in milliseconds, with metadata filtering and persistence attached. Indexing, updates, and hybrid query support are the product surface.
• Approximate nearest neighbor search: Trading a little accuracy for orders of magnitude in speed, by organizing the collection so only promising regions ever get compared. Exact search scores everything, which stops being an option once the corpus is measured in millions.
• HNSW: The dominant structure for that approximation: a layered graph whose upper layers carry long-range links and whose lower layers refine locality, so a query descends like an express elevator with local stops. Its main rival clusters vectors and compresses each cluster with product quantization, and production systems often combine the two.

The retrieval stack around embeddings

Four terms for the stages that sit before and after the vector query in a real pipeline.

• Cross-encoder: A network that processes the query and the candidate passage as a single joined input and returns one relevance score, so the two texts interact token by token. It beats separate encoding on accuracy, but nothing can be precomputed, which confines it to shortlists. ColBERT’s late-interaction design splits the difference by keeping per-token vectors and comparing them in batches.
• Reranker: The stage that applies a cross-encoder to the top few hundred candidates from vector search and reorders them. It is the standard remedy when recall looks healthy but the first result keeps being wrong.
• Sparse retrieval: Classic keyword matching, with BM25 as the scoring workhorse: exact terms, inverted indexes, no training required. It still beats dense vectors on rare strings, product IDs, and names that an embedding would blur into the neighborhood.
• Hybrid search: Running sparse and dense retrieval together and merging the ranked lists, most often through reciprocal rank fusion, which combines positions rather than incomparable raw scores. Robustness is the appeal: each method’s weak spots are the other’s strengths.

Common confusions

Three pairs that decide real architecture choices when they get mixed up.

• Embedding model vs cross-encoder: An embedding model encodes texts separately, so the corpus can be indexed ahead of time. A cross-encoder scores pairs jointly, which is sharper and slower by orders of magnitude. Retrieval uses the first; reranking uses the second.
• Sparse vs dense retrieval: Sparse retrieval matches exact terms through an inverted index. Dense retrieval matches meaning through vectors. The hybrid setups exist because a question containing both a product ID and a paraphrase needs the two at once.
• Reranker vs retriever: The retriever scans the whole corpus and proposes candidates, favoring coverage. The reranker deep-reads a shortlist and fixes the ordering. Swapping the jobs in either direction wastes the design.

Further reading

Three papers behind the stack above:

• The GPU-based billion-scale nearest-neighbor paper (Johnson et al.), the study that made approximate search at that scale routine.
• The ColBERT paper, late interaction as a middle ground between bi-encoders and cross-encoders.
• The MTEB paper, the benchmark suite embedding releases now report against.

This is lesson 06 of the AI glossary. Lesson 05 covered Prompting and In-Context Learning, and embeddings are what let those prompts arrive already carrying the right documents. For the layer this lesson stops short of, How Vector Databases Work continues the storage story, and the AWS vector database guide shows the same stack on managed infrastructure. The next lesson assembles all of it into retrieval-augmented generation.