Large language models are fluent but forgetful — they only know what was in their training data, frozen at a point in time. Retrieval-Augmented Generation fixes part of that.
The mechanism
Before answering, the system searches a document store for content relevant to the question, then feeds that content to the model alongside the question itself. The model answers from what it was just shown, not just from memory.
Why it matters
This is what lets an AI assistant answer questions about your company’s internal documentation, or a platform’s own content, without needing to retrain the underlying model.
The failure mode to watch for
RAG reduces fabrication but does not eliminate it. A model can still misread the retrieved context or blend it with unrelated training knowledge. Good systems show their sources so a reader can verify.