Start a project
Techniques Reviewed 2 Oct 2026 · 19 sources

Retrieval-Augmented Generation

RAG /rɪˈtriː.vəl ɔːɡˈmɛn.təd ˌdʒɛn.əˈreɪ.ʃən/ noun Abbreviation RAG · adjectives retrieval-augmented, RAG-based
Definition

A technique that searches a set of documents for passages relevant to a question and places them in a language model’s prompt, so the answer draws on them.

Examples
  • Perplexity’s answer engine, which pulls live search results into its answers
  • Google’s AI Overviews, which a search expert said likely used RAG
  • Microsoft’s Azure AI Search, for chatbots that answer from company files
  • Legal research tools from LexisNexis and Thomson Reuters
A question card moves past a filing cabinet, which hands over two pages. A model reads the card and the pages together and writes an answer with footnote numbers pointing to the pages.
Inkling · The answer is written from pages fetched for the questionFig. R1
Earliest AI use
2020
Paper posted to arXiv on 22 May 2020 and accepted at NeurIPS 2020.
Current meaning
2023
Search results fed through the prompt of an unchanged, finished model.
Coined by
Lewis et al.
Patrick Lewis and 11 co-authors at Facebook AI Research, UCL and NYU.
Opposite
Closed-book answer
Answering from trained parameters alone, with nothing looked up.
§01

In plain words

Retrieval-augmented generation (RAG) is a technique that has a language model answer from documents found by a search, not from its training alone.

A large language model answers from what it absorbed in training. That knowledge stops at a knowledge cutoff and never covered private material, such as a firm’s contracts or support tickets. Retrieval-augmented generation adds a lookup. Before the model replies, a search step finds passages that match the question and places them in the prompt. The model then writes from those passages.

The 2020 paper that named the technique presented it as a way to update a model’s knowledge and show where answers come from.1 Its authors described two stores of knowledge: the model’s parameters and a searchable collection of text. A model that uses only the first is said to answer closed-book.2 The lead author has said the team planned a nicer name but had none when the paper was due.3

RAG is often confused with fine-tuning, which changes the model’s weights through further training. In a 2023 Microsoft comparison, retrieval beat unsupervised fine-tuning at adding knowledge, including facts new to the model.4 It also overlaps with grounding, the aim of tying answers to sources. RAG is one way to reach that aim.

It sits behind many answer engines and workplace assistants. A 2026 court order, summarizing the publishers’ complaint, describes Perplexity as using it to bring live search results into its answers. Cloud platforms such as Microsoft’s Azure sell it for answering from a company’s own files. It reduces hallucination without ending it. In 2024 Google’s AI Overviews, which a search expert said likely relied on retrieval, advised adding glue to pizza. A Stanford and Yale study that year found that leading legal research tools still hallucinated between 17% and 33% of the time.5

§02

How it works

Before any question arrives, the documents are prepared. They are cut into short chunks. Each chunk becomes an embedding, a list of numbers that places texts with similar meanings near each other, and is stored in a vector database.

At question time there are three steps. To retrieve, the system turns the question into an embedding and fetches the nearest chunks. Some systems add a keyword search, which catches exact strings such as an error code. To augment, it places those chunks in the prompt beside the question. To generate, the model writes the reply. Its weights do not change; only its prompt does.

Each step can fail. A chunk cut loose from its document may no longer say which company or quarter a figure belongs to. Anthropic’s fix adds a line of context to every chunk. With keyword search and reranking, a second scoring pass, it cut failed retrievals by 67% in Anthropic’s tests.6

Newer systems often hand the search to the model. Microsoft now calls the single-search pipeline classic RAG and recommends agentic retrieval, in which a model plans several searches. The creator of Claude Code says it dropped a local vector database for a related approach, agentic search, in which the model runs its own searches.

Fig. R2 · One question, three stepsIllustrative, not measured

How many days of parental leave do new employees get?

Retrieve
The three handbook passages closest in meaning to the question
Augment
Instructions, the three passages and the question, in one prompt
Generate
An answer drawn from the passages, citing handbook section 4.2
Retrieval finds passages, augmentation adds them to the prompt, and generation writes from them. The question and the handbook are invented.
§03

Origin and first known use

The name joins retrieval, finding documents in a collection, to generation, a model writing text. Patrick Lewis and 11 co-authors at Facebook AI Research, University College London and New York University coined it in a paper posted in May 2020. Precursors include retrieve-then-read question answering from 2017 and REALM, posted in February 2020. The original RAG was one model whose retriever and writer were trained together. After ChatGPT, the term spread to any system that feeds search results into the prompt of an unchanged model. By 2026 Microsoft’s documentation called that setup classic RAG, to set it apart from searches planned by the model.

  1. 2017 Chen and colleagues answer open-domain questions by first retrieving Wikipedia articles, then reading them with a neural network. A retrieve-then-read precursor.
    Chen, Fisch, Weston and Bordes, “Reading Wikipedia to Answer Open-Domain Questions”, ACL 2017 arXiv 1704.00051 →: source for 2017
  2. 2020 REALM adds a learned retriever to language model pre-training, so the model can fetch and attend over Wikipedia documents.
    Guu, Lee, Tung, Pasupat and Chang, “REALM: Retrieval-Augmented Language Model Pre-Training” arXiv 2002.08909 →: source for 2020
  3. 2020 Lewis and 11 co-authors coin “retrieval-augmented generation (RAG)” for a text generator that consults a searchable index of Wikipedia.
    Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks”, NeurIPS 2020 arXiv 2005.11401 →: source for 2020
  4. 2020 Facebook AI releases RAG as a component of the Hugging Face Transformers library, open to outside developers.
    Riedel, Kiela, Lewis and Piktus, “Retrieval Augmented Generation: Streamlining the creation of intelligent natural language processing models”, Facebook AI blog ai.meta.com →: source for 2020
  5. 2023 A survey dates a shift to ChatGPT’s arrival: RAG comes to mean feeding retrieved chunks to a large model through its prompt at answer time.
    Gao et al., “Retrieval-Augmented Generation for Large Language Models: A Survey” arXiv 2312.10997 →: source for 2023
  6. 2024 Microsoft researchers report that RAG fails on questions about a whole corpus, like its main themes, and propose GraphRAG, which builds an entity graph first.
    Edge et al., “From Local to Global: A Graph RAG Approach to Query-Focused Summarization” arXiv 2404.16130 →: source for 2024
  7. 2024 Stanford and Yale researchers test legal research tools marketed as RAG-based and find them far from hallucination-free.
    Magesh et al., “Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools” arXiv 2405.20362 →: source for 2024
  8. 2026 Boris Cherny, who created Anthropic’s Claude Code, says its early RAG setup with a local vector database gave way to agentic search, which worked better.
    Boris Cherny, post on X, 1 February 2026 x.com →: source for 2026
§04

In the wild

10 sightings · 2020–2026 · oldest first
  1. Paper22 May 2020

    “We explore a general-purpose fine-tuning recipe for retrieval-augmented generation (RAG) … models which combine pre-trained parametric and non-parametric memory for language generation.”

    Lewis et al., NeurIPS 2020 arXiv 2005.11401 →: Lewis et al., NeurIPS 2020, 22 May 2020Archived →: copy of Lewis et al., NeurIPS 2020, 22 May 2020
  2. Blog28 Sep 2020

    “With RAG, we control what it knows simply by swapping out the documents it uses for knowledge retrieval.”

    Riedel, Kiela, Lewis and Piktus, Facebook AI blog ai.meta.com →: Riedel, Kiela, Lewis and Piktus, Facebook AI blog, 28 Sep 2020Archived →: copy of Riedel, Kiela, Lewis and Piktus, Facebook AI blog, 28 Sep 2020
  3. Blog19 Mar 2024

    “A typical RAG system today uses a frozen off-the-shelf model for embeddings, a vector database for retrieval, and a black-box language model for generation…”

    Contextual AI, company blog contextual.ai →: Contextual AI, company blog, 19 Mar 2024Archived →: copy of Contextual AI, company blog, 19 Mar 2024
  4. Headline4 May 2024

    “Why RAG won’t solve generative AI’s hallucination problem”

    TechCrunch techcrunch.com →: TechCrunch, 4 May 2024Archived →: copy of TechCrunch, 4 May 2024
  5. Paper30 May 2024

    “Recently, certain legal research providers have touted methods such as retrieval-augmented generation (RAG) as “eliminating” (Casetext, 2023) or “avoid[ing]” hallucinations (Thomson Reuters, 2023)…”

    Magesh et al., arXiv preprint arXiv 2405.20362 →: Magesh et al., arXiv preprint, 30 May 2024Archived →: copy of Magesh et al., arXiv preprint, 30 May 2024
  6. Headline31 May 2024

    “Why Google’s AI Overviews gets things wrong”

    MIT Technology Review technologyreview.com →: MIT Technology Review, 31 May 2024Archived →: copy of MIT Technology Review, 31 May 2024
  7. Headline6 Jun 2024

    “Can a technology called RAG keep AI models from making stuff up?”

    Ars Technica arstechnica.com →: Ars Technica, 6 Jun 2024Archived →: copy of Ars Technica, 6 Jun 2024
  8. Post1 Feb 2026

    “Early versions of Claude Code used RAG + a local vector db, but we found pretty quickly that agentic search generally works better.”

    Boris Cherny, on X x.com →: Boris Cherny, on X, 1 Feb 2026Archived →: copy of Boris Cherny, on X, 1 Feb 2026
  9. Docs4 Aug 2026

    “Retrieval-augmented generation (RAG) is a pattern that extends LLM capabilities by grounding responses in your proprietary content.”

    Microsoft, Azure AI Search documentation learn.microsoft.com →: Microsoft, Azure AI Search documentation, 4 Aug 2026Archived →: copy of Microsoft, Azure AI Search documentation, 4 Aug 2026
  10. Court record2 Sep 2026

    “Perplexity uses the RAG method to connect its LLMs “to external sources of information, such as live search results…”

    Magistrate Judge Sarah L. Cave, order in Encyclopaedia Britannica v. Perplexity Order →: Magistrate Judge Sarah L. Cave, order in Encyclopaedia Britannica v. Perplexity, 2 Sep 2026
§05

Not to be confused with

  • Fine-tuning

    Fine-tuning changes a model’s weights through further training on new examples. RAG leaves the weights alone and supplies relevant documents with each question, so what it knows changes when the documents do.

    See fine-tuning →

  • Grounding

    Grounding is the goal of tying each claim in an answer to a source the model was given. RAG is one method for reaching it, by supplying those sources through a search.

    See grounding →

  • Web search

    A search engine returns a ranked list of links for a person to read. In RAG the results go to a model, which writes one answer from them, often from private documents rather than the web.

  • Long context window

    A long context window lets a model read a whole document set in one prompt, with no search step. RAG selects a few passages instead, which suits collections too large for any window.

    See context window →
§06

Questions

  • What does RAG stand for in AI?

    RAG stands for retrieval-augmented generation. Before a language model answers, a search step looks through a chosen set of documents, such as a help center or a legal database. The passages that match the question are added to the prompt. The model then writes its answer from those passages rather than from its training alone.

  • Is RAG better than fine-tuning?

    They do different jobs. Fine-tuning changes a model’s weights through further training. RAG hands the model relevant documents each time it answers, and those documents can be swapped at any time. For adding knowledge, a 2023 Microsoft study found that RAG consistently beat unsupervised fine-tuning. That held both for facts the model had seen in training and for entirely new ones.

  • Does RAG stop AI hallucinations?

    No. It reduces them when the search finds the right passages, but it does not remove them. Some legal research vendors promoted retrieval as a way to avoid or eliminate hallucinations. A 2024 study by Stanford and Yale researchers then tested tools from LexisNexis and Thomson Reuters. Each still hallucinated between 17% and 33% of the time.

  • Do long context windows make RAG unnecessary?

    Sometimes. Anthropic says a knowledge base under 200,000 tokens, about 500 pages, can go into the prompt whole, with no retrieval step. Larger collections still need a search that picks the relevant passages. More and more, the model runs that search itself. The creator of Anthropic’s Claude Code says early versions used RAG with a local vector database, but agentic search worked better.

  • Who coined the term retrieval-augmented generation?

    Patrick Lewis and 11 co-authors at Facebook AI Research, University College London and New York University coined the term. Their paper was first posted to arXiv on 22 May 2020 and accepted at NeurIPS 2020. Lewis later told NVIDIA that the team had planned a nicer-sounding name but had no better idea when the paper was written.

§07

Watch

§09

Cite this entry

“Retrieval-Augmented Generation.” SUPERR_dictionary, Superrduperr, 2 Oct. 2026, superrduperr.com/dictionary/retrieval-augmented-generation.

First published 24 Sep 2026 · Last reviewed 2 Oct 2026 · Report an error →