What is RAG and what does it solve?
RAG reduces unsupported answers by retrieving enterprise knowledge the model has never seen, or that has gone out of date, at question time. Fine-tuning changes a model's behavior and language; RAG keeps knowledge outside the model, where it can be updated and cited.
- Freshness: content is updated without retraining
- Attribution: every answer cites the documents it relies on
- Declining to answer is designed behavior, not failure
What are the core components?
An enterprise RAG pipeline consists of five main components, and the chain is only as strong as its weakest link. Answer quality is usually decided in chunking and retrieval, not in the generation layer.
- Source connectors: collecting content and metadata from documents, wikis and business systems
- Chunking: segmentation that preserves heading and table structure
- Embedding and indexing: vector representations combined with keyword indexes
- Retrieval layer: hybrid search, permission filtering, reranking
- Generation layer: bounded prompts that turn context into cited answers
How are permissions preserved?
A RAG system must inherit access rules from its source systems; if the index carries no permission data, the model can leak content a user is not allowed to see. Permission checks are enforced at query time; data classes are labeled before indexing, and every question-answer flow is logged.
- Permission inheritance: access lists travel into the index and the query
- Data classes: exclusion by confidentiality label
- Query-time filtering: caching must not bypass permission checks
- Audit trail: who asked, which sources were retrieved, what was answered
How is quality measured?
RAG quality is measured with a repeatable evaluation set across versions, not by impression. A set drawn from real user questions scores retrieval accuracy, answer correctness and citation alignment separately. When chunking, indexing or the model changes, the same set is rerun; changes do not reach production while regressions remain.
- Evaluation set: real questions with expected sources
- Citation audit: whether each claim appears in its cited source
- Regression testing: rerunning the same set on every change
- Refusal behavior: testing safe responses to unknown questions