Retrieval-Augmented Generation (RAG)

Retrieval-Augmented Generation (RAG) is a technique that enhances a language model's output by retrieving relevant information from an external knowledge source at inference time, then conditioning the model's generation on the retrieved context. Rather than relying solely on knowledge encoded in the model's parameters, RAG dynamically accesses an external database or document store, allowing the model to cite up-to-date, domain-specific, or proprietary information without retraining.

A typical RAG pipeline has three stages: retrieval, augmentation, and generation. First, the user's query is used to retrieve relevant documents from a vector database (using embedding similarity search) or a keyword-based index. Second, the retrieved documents are concatenated with the original query to form an augmented prompt. Third, the language model generates a response conditioned on this augmented context. The retrieval step uses an embedding model to convert text into dense vectors and a similarity metric (such as cosine similarity) to find the most relevant passages.

RAG addresses several limitations of parametric knowledge: it enables access to information beyond the training cutoff, reduces hallucination by grounding responses in retrieved evidence, allows knowledge updates without retraining, and provides source attribution. However, RAG quality depends heavily on the retrieval component — if relevant documents are not retrieved, the model cannot use them. Challenges include retrieval precision, context window limitations, handling conflicting sources, and maintaining the vector index as the knowledge base evolves.