Ruka hadi maudhui

Maudhui makuu ya ukurasa huu ni kwa Kiingereza.Tafsiri ukurasa huu

AI na mada zinazovuma

RAG: Retrieval-Augmented Generation roadmap

RAG is the most common AI feature in business: an assistant that answers questions from company documents, with sources. Learn how it works and build one step by step, from a simple prototype to a measured, production-ready system.

Muda
⏱ 3–4 weeks
Level
Intermediate
Steps
6 steps
Architecture ↓Cheat sheet ↓8 interview Q&As ↓
Shiriki

The roadmap

Click a step to open the full guide.

Best architecture: a production RAG system

Separate ingestion and query paths, sharing one index with permissions.

Two paths, one index

Ingestion (offline)

  • Sources (docs, wiki, PDFs)
  • Parse & clean
  • Chunk + metadata + permissions
  • Embed
  • Vector + keyword index

Query (online)

  • Question (+ rewrite)
  • Hybrid search + permission filter
  • Re-rank
  • LLM with citations
  • Answer + feedback + evals

Why it's built this way

  • Ingestion runs separately and only re-processes changed documents.
  • Permissions are stored with chunks and filtered at query time.
  • Hybrid search + re-ranking gives the best retrieval quality.
  • Evals and feedback close the loop.

RAG: Retrieval-Augmented Generation cheat sheet

The essentials on one screen. Bookmark it for revision.

Pipeline

LoadText + source + page
ChunkBy headings; ~500–800 words; 10–15% overlap
EmbedSame model for docs and questions
StoreVector + text + metadata (pgvector or vector DB)
RetrieveHybrid search → re-rank → top 4–8
TengenezaAnswer only from context; cite; say 'I don't know'

Fixes for common problems

Right info not foundBetter chunking, metadata, hybrid search
Exact codes/names missedAdd keyword (BM25) search
Too much irrelevant contextRe-rank; reduce top-k
Made-up answersStricter prompt; require citations
Follow-ups failRewrite the query with chat history
Outdated answersRe-index changed docs; filter by date

Improve performance

  • Use an ANN index (e.g. HNSW) once you have more than a few thousand chunks.
  • Filter by metadata before or during vector search to shrink the search space.
  • Re-rank only the top 20–50 candidates, not everything.
  • Cache embeddings for repeated questions and answers for frequent questions.
  • Send fewer, better chunks to the model: shorter prompts are faster and cheaper.
  • Embed documents in batches and only re-embed documents that changed.

Security checklist

  • Enforce permissions in retrieval: filter chunks by the user's access before they reach the model.
  • Treat document content as untrusted: it may contain prompt-injection text.
  • Remove or mask personal data (PII) before indexing when it isn't needed.
  • Log which documents were used for each answer for auditing.
  • Delete chunks when source documents are deleted (right to be forgotten).
  • Keep the vector database private; never expose it directly to the internet.

Most important interview questions

Click a question to see a strong answer. Each step also has its own questions.

1RAG ni nini?

Retrieval-augmented generation: retrieve relevant passages from your data and give them to an LLM so it answers from them, with citations, without retraining the model.

2Describe a production RAG architecture.

Ingestion (parse, clean, chunk, embed, store with metadata and permissions) → query (rewrite, hybrid search, filter, re-rank) → generation (grounded prompt, citations) → evaluation and monitoring, with caching and re-indexing of changed documents.

3How do you choose an embedding model?

Test a few on your own retrieval eval set; consider language support (e.g. Indian languages), vector size (storage cost), speed, price and whether it can run privately.

4the 'lost in the middle' problem ni nini?

Models can pay less attention to information buried in the middle of long contexts. Send fewer, well-ranked chunks and put the most relevant ones first.

5How do you reduce hallucinations in RAG?

Improve retrieval quality, instruct the model to use only the context and say when the answer is missing, require citations, and evaluate faithfulness automatically.

6How do you handle multilingual documents?

Use a multilingual embedding model, store the language in metadata, and either search across languages or translate the query; answer in the user's language.

7RAG vs long context windows: is RAG still needed?

Yes for large or changing data: retrieval is cheaper, faster, respects permissions and gives citations. Long contexts help when a few documents fit entirely.

8How would you evaluate a RAG system?

Golden question set → measure retrieval (recall@k, MRR) and generation (correctness, faithfulness, completeness) separately → automate and track over time → add real user questions.

Ramani Related

6

AI Engineer

6 steps · 4–6 months · Intermediate

Build products on top of large language models.

View roadmap →
6

PostgreSQL & SQL

6 steps · 4–5 weeks · Beginner

The database most new apps choose.

View roadmap →
6

AI Agents

6 steps · 3–4 weeks · Intermediate

Build AI that plans, uses tools and gets work done.

View roadmap →
5

MCP: Model Context Protocol

5 steps · 2–3 weeks · Intermediate

Connect AI apps to tools and data with one open standard.

View roadmap →
1

RAG: Retrieval-Augmented Generation · Step 1 of 6 · 2 days

How RAG works

Retrieve, then generate.

RAG has two phases. Indexing: split documents into chunks, turn each into an embedding (a list of numbers capturing meaning) and store them. Querying: embed the user's question, find the most similar chunks and pass them to the LLM with an instruction to answer only from them and cite sources.

This keeps answers grounded in your data, lets you update knowledge without retraining and shows users where answers came from.

What you'll learn

  • Why LLMs need retrieval
  • Indexing vs querying
  • Embeddings
  • Vector similarity
  • The prompt with context
  • Citations

Without vs with RAG

Plain LLM

  • Answers from training data only
  • Can't see your documents
  • May guess (hallucinate)
  • No sources

LLM + RAG

  • Searches your documents first
  • Answers from real passages
  • Says 'I don't know' when missing
  • Shows sources

Key concepts

Retrieval
Finding the passages most relevant to a question.
Augmented generation
The model writes its answer using the retrieved passages.
Embedding
Numbers that represent meaning, so 'refund' and 'money back' end up close together.

Step by step

  1. 1.1

    See the problem

    Ask a chatbot about your private document. It can't know it, or it guesses.

  2. 1.2

    Learn the two phases

    Indexing prepares documents once; querying runs for every question.

  3. 1.3

    Play with embeddings

    Embed a few sentences and compare similarity scores to build intuition.

INDEXING:  documents → clean → chunk → embed → vector store
QUERYING:  question → embed → top-k similar chunks → (re-rank)
           → prompt = instructions + chunks + question → LLM → answer + sources

Notes

  • RAG adds knowledge; fine-tuning changes behaviour.

Tips & tricks

  • Draw the pipeline: documents → chunks → embeddings → vector store → retrieve → prompt → answer.

Interview questions

Explain RAG to a non-technical person.

It's like an open-book exam: before answering, the AI looks up the relevant pages in your documents and answers from them, showing which pages it used.

What are the two phases of RAG?

Indexing (clean, chunk, embed and store documents) and querying (embed the question, retrieve similar chunks, then generate an answer from them).

The quick way to study this

Understand the two phases and the role of embeddings; that is the whole idea.

2

RAG: Retrieval-Augmented Generation · Step 2 of 6 · 3 days

Build a minimal RAG app

Working prototype in under 100 lines.

Start without frameworks so you understand every step. Load a few documents, split them into chunks of a few hundred words with some overlap, embed them and store them.

At question time, embed the question, take the top few chunks by cosine similarity and ask the model to answer using only that context, saying 'I don't know' if the answer is missing.

What you'll learn

  • Loading text and PDFs
  • Fixed-size chunking with overlap
  • Creating embeddings
  • A simple vector store
  • Prompting with retrieved context

Minimal RAG pipeline

  1. Load
  2. Chunk
  3. Embed
  4. Store
  5. Retrieve top-k
  6. Prompt LLM
  7. Answer

Key concepts

Chunk
A small passage of a document that is retrieved as a unit.
Overlap
Repeating a few sentences between chunks so ideas at the edges aren't cut in half.
Cosine similarity
A score from −1 to 1 for how close two embeddings point; higher = more similar.
Top-k
How many chunks you retrieve. Start with 4–8.

Step by step

  1. 2.1

    Load documents

    Read text and PDFs into plain text with their source name and page.

  2. 2.2

    Chunk

    Split into ~500–800-word chunks with ~10–15% overlap.

  3. 2.3

    Embed

    Call an embedding model for each chunk; store vector + text + metadata.

  4. 2.4

    Answer

    Embed the question, take the top 4 by cosine similarity, prompt the model with them.

# Minimal RAG (pseudo-code; use your provider's embedding and chat APIs)
chunks = split(load_docs("docs/"), size=800, overlap=100)
vectors = [embed(c.text) for c in chunks]

def answer(question: str) -> str:
    q = embed(question)
    top = sorted(zip(chunks, vectors), key=lambda cv: -cosine(q, cv[1]))[:4]
    context = "\n\n".join(f"[{c.source}] {c.text}" for c, _ in top)
    prompt = ("Answer only from the context. Cite sources in [brackets]. "
              "If the answer is not there, say you don't know.\n\n"
              f"Context:\n{context}\n\nQuestion: {question}")
    return chat(prompt)

Notes

  • Tell the model to say when the answer is not in the context; it reduces hallucinations.

Tips & tricks

  • Print the retrieved chunks for every question while developing.

Interview questions

How do you choose chunk size?

It's a trade-off: too big and retrieval is vague and wastes tokens; too small and chunks lose context. Start around a few hundred words with overlap, then tune using your test questions.

Why must the question and documents use the same embedding model?

Different models map text into different vector spaces, so similarity scores between them are meaningless.

The quick way to study this

Build it in a day with a small folder of documents you know well.

Practice project

A Q&A bot over your college notes or a product manual.

How to build a RAG app (full guide) →
3

RAG: Retrieval-Augmented Generation · Step 3 of 6 · 3 days

Better chunking and data preparation

Good retrieval starts with good chunks.

Most RAG failures happen because the right information was never retrieved. Split documents along their structure (sections, headings) and store metadata with each chunk.

Adding the document title and section heading to each chunk's text often improves retrieval noticeably.

What you'll learn

  • Structure-aware chunking (headings, sections)
  • Metadata: title, section, date, source
  • Cleaning PDFs and HTML
  • Tables and images
  • Deduplication

A chunk with metadata

  • Chunk #42
  • text: "Items can be returned within 30 days…"
  • source: Returns Policy.pdf
  • section: Refunds › Time limits
  • page: 2
  • updated: 2026-09-01
  • access: public

Key concepts

Metadata
Facts about a chunk (source, date, section) used for filtering and citations.
Structure-aware chunking
Splitting along a document's own headings so each chunk is one topic.
Contextual chunk header
Prefixing each chunk with its document title and section to improve retrieval.

Step by step

  1. 3.1

    Clean text

    Remove headers, footers, page numbers and broken line breaks.

  2. 3.2

    Chunk by structure

    Split on headings and sections instead of fixed sizes.

  3. 3.3

    Add metadata

    Store title, section, date, author and access level with each chunk.

  4. 3.4

    Handle tables

    Convert tables to Markdown or describe them in sentences.

Notes

  • Garbage in, garbage out: clean the text before chunking.

Tips & tricks

  • Look at 20 random chunks; if they do not make sense alone, change the chunking.

Interview questions

How would you handle PDFs with tables and images?

Use a document parser that keeps layout, convert tables to Markdown or row-wise sentences, add captions or descriptions for images (a vision model can help), and keep page numbers for citations.

Why add metadata to chunks?

Kwenda filter (only this product, only documents the user may see, only recent versions) and to cite sources accurately.

The quick way to study this

Switch from fixed-size to heading-based chunks and compare on your test questions.

4

RAG: Retrieval-Augmented Generation · Step 4 of 6 · 4 days

Vector databases and hybrid search

Find the right passages reliably.

Vector search finds passages with similar meaning; keyword search finds exact terms such as product codes or names. Hybrid search combines both and is usually better than either alone.

A re-ranker then reorders the top candidates with a more precise model before they reach the LLM.

What you'll learn

  • Vector databases and pgvector
  • Approximate nearest neighbour indexes
  • Keyword search (BM25)
  • Hybrid search
  • Metadata filters
  • Re-ranking

Hybrid retrieval

  1. Question
  2. Vector search + keyword search
  3. Merge results
  4. Re-rank
  5. Best 4–8 chunks

Key concepts

Hybrid search
Vector search (meaning) + keyword search (exact words). Usually beats either alone.
BM25
A classic keyword ranking method used by search engines.
Re-ranker
A slower, more precise model that reorders candidates by relevance to the question.
ANN index (HNSW)
A structure that finds nearest vectors fast without comparing every one.

Step by step

  1. 4.1

    Pick a store

    pgvector if you use PostgreSQL; a dedicated vector DB for very large scale.

  2. 4.2

    Add keyword search

    Run BM25/full-text search alongside vector search.

  3. 4.3

    Combine results

    Merge both lists (e.g. reciprocal rank fusion).

  4. 4.4

    Re-rank

    Score the top 20–50 with a re-ranker model; keep the best 4–8.

Notes

  • Filter by permissions so users only retrieve documents they may see.

Tips & tricks

  • Use PostgreSQL + pgvector if you already use PostgreSQL.

Interview questions

When does keyword search beat vector search?

For exact terms: product codes, names, error messages, IDs and rare words that embeddings may blur. That's why hybrid search is the default in production.

What does a re-ranker do and what does it cost?

It reads the question and each candidate together to score relevance more accurately than embeddings. It adds latency and cost, so apply it only to the top few dozen candidates.

The quick way to study this

Add keyword search and a re-ranker, then compare scores on your test set.

PostgreSQL & pgvector →
5

RAG: Retrieval-Augmented Generation · Step 5 of 6 · 3 days

Evaluate your RAG system

Measure retrieval and answer quality.

Evaluate the two halves separately. Retrieval: kwa each test question, was the right chunk in the top results? Generation: is the answer correct, complete and supported by the retrieved text?

Re-run the evaluation every time you change chunking, embeddings, prompts or models.

What you'll learn

  • Building a test set
  • Retrieval metrics: hit rate, recall@k
  • Answer faithfulness and relevance
  • LLM-as-judge
  • Regression testing

Evaluate the two halves separately

Retrieval

  • Hit rate / recall@k
  • Mean reciprocal rank
  • Fix: chunking, hybrid, re-rank

Generation

  • Correct & complete
  • Faithful to sources
  • Fix: prompt, model, context size

Key concepts

Recall@k
The share of questions where the right chunk appears in the top k results.
Faithfulness
Whether every claim in the answer is supported by the retrieved text.
Golden set
Your fixed list of test questions and expected sources/answers.

Step by step

  1. 5.1

    Collect questions

    Write 30 real questions; record which source should answer each.

  2. 5.2

    Measure retrieval

    Hit rate / recall@k: is the right chunk in the top k?

  3. 5.3

    Measure answers

    Correctness, completeness and faithfulness to sources (LLM judge + spot checks).

  4. 5.4

    Automate

    Run the suite on every change; keep a results table over time.

Notes

  • A wrong answer with a citation is worse than 'I don't know'.

Tips & tricks

  • Collect real user questions after launch and add them to the test set.

Interview questions

How do you know if a RAG problem is retrieval or generation?

Check whether the correct passage was retrieved. If not retrieved, it's retrieval. If retrieved but the answer is wrong, it's generation (prompt, model or too much noisy context).

How would you build a RAG evaluation set quickly?

Take real user questions or generate candidate questions from documents with an LLM, then have a person check them and record the correct source and answer.

The quick way to study this

Write 30 questions with expected sources and measure hit rate today.

6

RAG: Retrieval-Augmented Generation · Step 6 of 6 · 4 days

Advanced and agentic RAG

Handle complex questions and production needs.

Some questions need several searches or comparing documents. Agentic RAG gives retrieval to an agent as a tool, so it can search, read and search again.

In production, keep indexes fresh as documents change, enforce permissions and cache frequent questions.

What you'll learn

  • Query rewriting
  • Multi-step retrieval
  • Agentic RAG (retrieval as a tool)
  • GraphRAG basics
  • Caching
  • Access control and freshness

Agentic RAG

  1. Question
  2. Plan searches
  3. Search tool
  4. Read results
  5. Enough to answer?

Repeats as a loop

Key concepts

Query rewriting
Using the model to turn 'what about the blue one?' into a full search query.
Agentic RAG
An agent that searches as many times as needed before answering.
GraphRAG
Building a graph of entities and relationships to answer questions that span many documents.
Index freshness
Re-indexing changed documents so answers stay current.

Step by step

  1. 6.1

    Rewrite queries

    Turn vague or follow-up questions into clear search queries.

  2. 6.2

    Multi-step retrieval

    Break complex questions into sub-questions and search for each.

  3. 6.3

    Agentic RAG

    Give retrieval to an agent as a tool so it decides when and what to search.

  4. 6.4

    Production basics

    Keep indexes fresh, enforce permissions, cache frequent questions, monitor quality.

Notes

  • Add complexity only when your evaluation shows simple RAG falls short.

Tips & tricks

  • Expose your retrieval as an MCP tool so any AI app can use it.

Interview questions

How do you handle follow-up questions in a RAG chatbot?

Rewrite the latest message into a standalone query using the conversation history, then retrieve with that query.

How do you keep a RAG system up to date?

Track document versions, re-chunk and re-embed only changed documents on a schedule or via webhooks, delete removed ones, and store an 'updated' date in metadata.

The quick way to study this

Turn your retriever into a tool and let an agent decide when to search.

AI agents roadmap →MCP roadmap →

Shiriki ukurasa huu