Retrieval

This week is the fourth core session of the course. It focuses on retrieval: how StatsChat finds the chunks most likely to help answer a user’s question.

The key message is:

Retrieval is evidence selection. It is the step where StatsChat decides which chunks the LLM should see.

Slide deck

Session overview

In this session, we focus on:

• where retrieval fits in the RAG pipeline
• why StatsChat retrieves chunks rather than whole PDFs
• why vector search helps with natural-language questions
• how L2 and cosine similarity relate when vectors are unit-normalised
• how StatsChat uses FAISS as first-stage vector search
• why “latest” questions are difficult for official statistics
• how retrieval can fail
• how retrieval could be improved through hybrid search, metadata, reranking and evaluation

Lab exercise

Week 7 exercise: choosing retrieved evidence

This short exercise is for the Thursday lab. It should take around 10-15 minutes.

The aim is to practise the main lesson from Tuesday’s session:

Retrieval is evidence selection: the system must decide which chunks are useful enough to show the LLM.

You do not need to write code.

Part 1: think about your own source

Use the publication or data source you had in mind in previous weeks, or choose a new one.

Answer briefly:

1. What source would you want to search in a StatsChat-like system?
2. What is one question a user might ask about it?
3. What would count as the right evidence for that question?
4. What metadata would help retrieval?

Part 2: compare retrieved chunks

Imagine a user asks:

What was the annual inflation rate in February 2025?

The retrieval system returns the following chunks.

Chunk A: February 2025 CPI release

The annual inflation rate was 5.1 per cent in February 2025.
The Consumer Price Index increased from the previous month.
Source: Kenya National Bureau of Statistics, CPI release, February 2025.

Chunk B: January 2025 CPI release

The annual inflation rate was 5.3 per cent in January 2025.
Food and non-alcoholic beverages contributed to the annual rate.
Source: Kenya National Bureau of Statistics, CPI release, January 2025.

Chunk C: CPI methodology note

The Consumer Price Index measures changes in the price of a basket of goods
and services purchased by households. The index is used to monitor inflation.

Chunk D: older economic survey

Inflation remained stable during 2019, supported by changes in food and fuel prices.
The annual average inflation rate was reported in the Economic Survey 2020.

Questions

1. Which chunk should definitely be passed to the LLM, and why?
2. Should Chunk B be passed to the LLM? Why or why not?
3. What is useful about Chunk C, and why is it not enough for this question?
4. What makes Chunk D risky?
5. Is there enough evidence here to answer the question safely?
6. What metadata would make these retrieval decisions easier?

Part 3: choose an improvement

For your own source, choose one improvement that would matter most:

• hybrid keyword + semantic search
• better metadata filters
• stronger reranking
• better latest/date handling
• better table-aware chunking
• query rewriting
• retrieval evaluation

Bring one short comment to the lab.

Lab Slides