Week 3 Lab: LLM systems, RAG, and modern tools

Lab focus

Review, discuss, extend

• revisit the exercise
• clarify what retrieval adds
• discuss why custom RAG still matters
• briefly look at modern LLM systems and tools

Quick recap of the exercise

What made the grounded answer better?

• more specific
• easier to verify
• tied to a source
• clearer about the measure being used

But what could still go wrong?

Worked example reminder

Question

According to the National Statistics Office’s Consumer Price Bulletin, what was the year-on-year inflation rate in March 2025, and which categories contributed most to the increase?

Plain LLM-style answer

Grounded / RAG-style answer

Why build custom RAG if modern LLMs already have tools?

• search our own trusted corpus
• get more control over output and references
• use a smaller / cheaper model on selected evidence
• improve even strong models on document-specific tasks
• support governance and deliberate corpus updates

Modern LLM systems are often model + tools

flowchart LR
    A[User request] --> B[LLM]
    B --> C[Web search]
    B --> D[File search / retrieval]
    B --> E[Calculator / Python]
    B --> F[Images / other tools]
    C --> G[Answer]
    D --> G
    E --> G
    F --> G

The key idea

Strong LLM vs custom RAG is often a false choice

• a RAG system can use a strong LLM as its generator
• retrieval adds grounding and control
• built-in tools are useful, but not the same as owning the retrieval layer

Optional deeper dive

Base model vs assistant model

flowchart LR
    A[Pretraining] --> B[Base model]
    B --> C[Post-training / alignment]
    C --> D[Assistant model]
    D --> E[Optional tools]

• Base model: broad language ability
• Assistant model: more helpful and instruction-following
• Tools: extend what the overall system can do

One simple example

Prompt

Explain inflation in simple terms.

Bridge to later sessions

Later we will look more closely at:

• retrieval quality
• generation quality
• evaluation
• how StatsChat fits together as a system

Final prompt

What documents in your organisation would you most want a system like this to search?

Be ready to discuss one or two examples.