An agent decides whether to retrieve, which source or tool to query and whether to search again after reading results. It helps multi-step and multi-source questions. It costs more calls, more latency and is harder to evaluate and debug. Add step limits and evaluate both final answers and retrieval trajectories.

Advanced RAG Interview Questions (2026): 15 Senior-Level Questions

A 5-step way to answer any advanced RAG question

Name the failure. Say which failure the technique targets, such as low recall, wrong ranking or missing context.
Explain the mechanism. Describe in one or two sentences how the technique works.
State the cost. Extra LLM calls, latency, index size or operational complexity.
Measure it. Retrieval recall@k, answer faithfulness and end-to-end accuracy on a labeled set.
Give a fallback. When you would not use it and what simpler option you would try first.

Contents: Query-side techniques (questions 1 to 4) · Index and chunking techniques (questions 5 to 8) · Ranking and fusion (questions 9 to 11) · Agentic, graph and long-context RAG (questions 12 to 15)

Query-side techniques (questions 1 to 4)

1. What is query rewriting and when does it help?

An LLM or rules rewrite the user query into a clearer, self-contained search query.
It helps with conversational follow-ups ("what about the second one?") and vague or misspelled queries.
It adds an LLM call, so it costs latency and can drift from user intent.
Evaluate by comparing retrieval recall with and without rewriting on real queries.

Common pitfall: Rewriting every query even when the original already retrieves well.

2. What is HyDE (Hypothetical Document Embeddings)?

The model writes a hypothetical answer passage, and that passage is embedded and used for retrieval instead of the short query.
It can improve recall when queries are short and documents are long, because the hypothetical passage looks more like the documents.
The hypothetical text may contain wrong facts, but it is used only for retrieval, not shown as the answer.
It adds a generation step before retrieval, increasing latency.

Common pitfall: Claiming HyDE puts the hypothetical answer in front of the user.

3. What is multi-query retrieval?

Generate several reformulations of the query, retrieve for each and merge the results.
It increases recall when one phrasing misses relevant documents.
Merging is often done with reciprocal rank fusion, followed by deduplication.
Cost grows with the number of queries, so keep the count small.

Common pitfall: Not deduplicating merged results, which wastes context.

4. How do you decompose a multi-hop question?

Break the question into sub-questions whose answers depend on each other.
Retrieve for each sub-question, possibly using earlier answers to form later queries.
Combine the evidence in a final generation step with citations per claim.
This is a step toward agentic RAG and adds latency and failure points.

Common pitfall: Retrieving once for a question that needs two separate facts.

Index and chunking techniques (questions 5 to 8)

5. What is parent-child (small-to-big) chunking?

Index small chunks for precise matching, but return their larger parent section to the model.
It balances retrieval precision with enough surrounding context to answer.
It requires storing the mapping from child to parent.
Watch context length, since parents can be large.

Common pitfall: Using only large chunks, which dilutes embedding similarity.

6. What is contextual retrieval or chunk enrichment?

Before embedding, add context to each chunk, such as the document title, section path or a short generated summary of where it sits.
It helps chunks that are ambiguous on their own, for example "the limit is 10 per day".
Generated context costs an LLM call per chunk at indexing time.
Measure the change in recall@k on the same evaluation set.

Common pitfall: Embedding chunks with no document context and then blaming the embedding model.

7. How do you handle tables and PDFs in RAG?

Use layout-aware parsing so tables, headings and reading order survive extraction.
Store tables as structured text (for example Markdown) or keep row-level chunks with headers repeated.
For scanned documents, add OCR and check its quality.
Test retrieval on questions that need table values, since they often fail silently.

Common pitfall: Flattening tables into plain text and losing which value belongs to which column.

8. When should you re-index, and how do you change embedding models safely?

Re-index when documents change, when chunking changes or when you switch embedding models.
Vectors from different embedding models are not comparable, so a model switch needs a full re-embed.
Build the new index in parallel, compare on your evaluation set, then switch traffic.
Keep document versions and deletion handling so stale content does not linger.

Common pitfall: Mixing vectors from two embedding models in the same index.

Ranking and fusion (questions 9 to 11)

9. What does a cross-encoder reranker add?

A cross-encoder reads the query and each candidate together and outputs a relevance score.
It is more accurate than bi-encoder similarity but too slow to run over the whole corpus.
Typical use: retrieve a few dozen candidates cheaply, rerank, keep the top few.
Measure precision at the top positions and added latency.

Common pitfall: Reranking only the top 3 results, which leaves no room to improve.

10. What is reciprocal rank fusion (RRF)?

RRF merges ranked lists by summing 1 / (k + rank) for each document across lists.
It uses ranks rather than raw scores, so it works when scores are on different scales, as with BM25 and cosine similarity.
The constant k (60 in the original paper) dampens the effect of top ranks.
It is simple and robust, but weighted score fusion can win when tuned on labeled data.

Common pitfall: Adding BM25 scores and cosine similarities directly without normalization.

11. How do you fit context into a limited window after retrieval?

Keep the most relevant chunks, remove duplicates and trim boilerplate.
Order matters: models can underuse information in the middle of long contexts, so place key evidence early or late.
Compress with extractive selection or summaries when needed.
Track faithfulness to see if compression removed needed facts.

Common pitfall: Concatenating every retrieved chunk regardless of relevance.

Agentic, graph and long-context RAG (questions 12 to 15)

12. What is agentic RAG?

An agent decides whether to retrieve, which source or tool to query and whether to search again after reading results.
It helps multi-step and multi-source questions.
It costs more calls, more latency and is harder to evaluate and debug.
Add step limits and evaluate both final answers and retrieval trajectories.

Common pitfall: Using an agent loop for simple single-lookup questions.

13. What is GraphRAG and when is it useful?

GraphRAG builds a knowledge graph of entities and relations from documents and retrieves over the graph, sometimes with community summaries.
It helps questions about relationships and global themes across many documents.
Graph construction is costly and needs updates as documents change.
Compare against a strong hybrid baseline before adopting it.

Common pitfall: Adopting a graph approach without showing the vector baseline fails.

14. Long context windows are large now. Do we still need RAG?

Long context helps for a few documents, but cost and latency grow with input tokens.
Retrieval scales to corpora far larger than any window and supports permissions and freshness.
Models can still miss facts buried in long inputs.
A common answer is both: retrieve, then use a generous context for the selected documents.

Common pitfall: Saying RAG is obsolete because context windows grew.

15. How do you decide which advanced technique to try first?

Diagnose with a labeled set: separate retrieval misses from generation errors.
If recall is low, try hybrid search, better chunking or query rewriting.
If recall is fine but top results are noisy, add a reranker.
Add agentic or graph methods last, since they cost the most.

Common pitfall: Stacking every technique at once so no single change can be evaluated.

Go deeper on related questions

FAQ

Which advanced RAG topics come up most?

Hybrid search with reranking, chunking strategy, query rewriting, evaluation and agentic RAG come up most often. GraphRAG tends to appear as a senior follow-up.

Should I memorize technique names?

Know the names, but focus on the failure each one fixes, its cost and how you would measure it. That is what interviewers score.

Is RRF always better than weighted fusion?

No. RRF is a robust default with no tuning; weighted fusion can be better when you have labeled data to tune weights.

How do I practice advanced RAG?

Build a small pipeline, create 30 to 50 labeled questions and measure recall@k before and after each technique.

The PlaybookWant the full method?

The 0 to 1 AI Engineer Interview Playbook covers answer frameworks, system-design walkthroughs and an 8-week plan. $19.99 on Gumroad.

Get the playbook

All interview topics · Prep guides · Question deep dives · Free 50 RAG and LLM questions