RAG (Retrieval-Augmented Generation) Cheat Sheet
Design retrieval-augmented pipelines covering chunking strategies, hybrid search, reranking, and evaluation of grounded LLM answers.
3 PagesIntermediateFeb 25, 2026
Chunk Documents for Retrieval
Split long text into overlapping chunks sized for the embedding model's context window.
python
from langchain_text_splitters import RecursiveCharacterTextSplittersplitter = RecursiveCharacterTextSplitter( chunk_size=800, chunk_overlap=120, separators=["\n\n", "\n", ". ", " ", ""],)chunks = splitter.split_text(long_document)print(f"{len(chunks)} chunks, avg len {sum(len(c) for c in chunks)//len(chunks)}")
Hybrid Search (Dense + Sparse)
Combine keyword (BM25) and vector search scores for more robust retrieval.
python
from rank_bm25 import BM25Okapibm25 = BM25Okapi([c.split() for c in chunks])bm25_scores = bm25.get_scores(query.split())vector_scores = vector_index.similarity_scores(query, top_k=len(chunks))# reciprocal rank fusion of the two rankingsdef rrf_score(rank, k=60): return 1.0 / (k + rank)combined = {}for rank, idx in enumerate(np.argsort(-bm25_scores)): combined[idx] = combined.get(idx, 0) + rrf_score(rank)for rank, idx in enumerate(np.argsort(-vector_scores)): combined[idx] = combined.get(idx, 0) + rrf_score(rank)
Rerank Retrieved Chunks
Use a cross-encoder to rescore the top candidates before sending them to the LLM.
python
from sentence_transformers import CrossEncoderreranker = CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2")pairs = [(query, chunk) for chunk in top_20_chunks]scores = reranker.predict(pairs)ranked = [c for _, c in sorted(zip(scores, top_20_chunks), reverse=True)]top_5 = ranked[:5]
Build a Grounded Prompt
Assemble retrieved context into a prompt that instructs the model to only answer from context.
python
SYSTEM = """Answer only using the CONTEXT below. If the answer isn'tcontained in the context, say you don't know. Cite the [source] for each claim."""context = "\n\n".join(f"[{c.metadata['source']}] {c.text}" for c in top_5)messages = [ {"role": "system", "content": SYSTEM}, {"role": "user", "content": f"CONTEXT:\n{context}\n\nQUESTION: {query}"},]
Common RAG Failure Modes
The most frequent ways a RAG pipeline breaks in production.
- Chunk too large- dilutes relevance signal, buries the useful sentence in noise
- Chunk too small- loses surrounding context needed to answer correctly
- No reranking- top-k vector search alone often misses the best passage
- Missing metadata filters- retrieval mixes documents across tenants/versions/dates
- Stale index- source docs changed but the vector store was never re-embedded
- No answer refusal- model hallucinates instead of saying context is insufficient
Pro Tip
Evaluate retrieval and generation separately — measure context recall (did we fetch the right chunk?) before measuring answer quality, since a perfect generator can't fix bad retrieval.
Was this cheat sheet helpful?
Explore Topics
#RAGRetrievalAugmentedGeneration#RAGRetrievalAugmentedGenerationCheatSheet#DataScience#Intermediate#ChunkDocumentsForRetrieval#Hybrid#Search#Dense#MachineLearning#DevOps#CheatSheet#SkillVeris
