Retrieval

Retrieval is the process of searching a corpus of documents, data, or knowledge to find the pieces most relevant to a given query — it's a decades-old information-retrieval concept (the same core idea behind classic search engines) that has become central to modern AI product architecture as the first step of a RAG pipeline. Retrieval can happen through several distinct mechanisms, and the choice matters for quality: lexical/keyword retrieval (like classic BM25 or Elasticsearch full-text search) matches based on exact or near-exact word overlap between the query and documents, fast and precise for queries with specific terminology but blind to meaning (searching "cancel subscription" won't match a document that only says "terminate your plan"); semantic/vector retrieval embeds both the query and documents into vector space and finds nearest neighbors by meaning, catching paraphrases and synonyms that keyword search misses, but sometimes less precise on queries with specific exact terms (product codes, error messages) that benefit from literal matching; and hybrid retrieval combines both approaches (often via reciprocal rank fusion), typically producing the best real-world results by capturing both exact-match and semantic-match relevance. This matters for SaaS builders because retrieval quality is the ceiling on RAG quality — no amount of prompt engineering on the generation side fixes a system that retrieved the wrong documents in the first place, making retrieval tuning (chunking strategy, embedding model choice, metadata filtering, hybrid search weighting, reranking) often the highest-leverage place to invest engineering time in a RAG system. A concrete worked example: a developer-documentation search assistant receives the query "why is my webhook not firing." Pure keyword retrieval might miss a relevant troubleshooting doc titled "Debugging silent event delivery failures" because it shares no exact words with the query; semantic retrieval catches it because the embeddings recognize the conceptual overlap; a hybrid system also correctly prioritizes a doc that literally contains "webhook" in its title when that document is genuinely the best match. Production retrieval systems commonly add a reranking step after initial retrieval — a more computationally expensive but more accurate relevance model that re-scores the initial candidate set before the top few are sent to the LLM. Retrieval quality is also affected by document preparation choices that happen well before any query is issued: how documents are chunked (splitting on paragraph boundaries generally preserves more coherent meaning than splitting on a fixed character count that can cut a sentence or table mid-way), what metadata is attached (source, date, access permissions, section headers), and whether duplicate or near-duplicate content is deduplicated before indexing all directly affect what a retrieval system can find and how relevant it is — meaning a meaningful fraction of RAG quality work happens in data preparation, not in the retrieval algorithm itself.