Evaluation: How Do We Know Whether StatsChat Is Working?
UNECA StatsChat/RAG course — Week 9
Today’s focus
We have spent the course opening up the “black box” of a RAG system.
Today we ask: how do we know whether the whole system is working?
From evidence preservation to evidence checking
Did we preserve the evidence?
Did we create useful evidence units?
Did we select the right evidence?
Did the model use the evidence safely?
Can we measure and improve the evidence chain?
Evaluation is the feedback loop for the whole course.
AI Ecosystem
The environment that AI operates in - data, users, workflows, and context - is much more important than the model itself.
- Right set-up creates a feedback loop that compounds improvements over time.
- Loops from Organisations/People to AI and back
Warm-up question
What is the main purpose of evaluation in a RAG system?
A. To prove the system will never make mistakes
B. To measure and diagnose how well the system works
C. To replace human judgement
D. To make the model answer faster
Best answer: B. Evaluation measures performance and helps us diagnose what to improve.
Core message
Why use automated evaluation (opposed to manual evaluation) in a RAG system?
Evaluation is the feedback loop that lets us improve a RAG system safely.
Weak confidence
“This answer looks good.”
Stronger confidence
“Across a tested set of questions, we know where the system works, where it fails, and what to improve next.”
Learning aims
Workflow
Explain the automated evaluation loop.
Metrics
Distinguish answer, guardrail, and retrieval metrics.
Benchmark
Explain why answerable and unanswerable rows matter.
Interpretation
Read headline results without overclaiming.
Design
Create evaluation questions for your own NSO context.
Decisions
Connect evaluation evidence to adoption choices.
A concrete example first
We can start with the live system, not the metrics.
What was Kenya’s real GDP growth rate in 2024?
Then ask: what would we score as success?
Live answer → evaluation row
A UI interaction becomes a repeatable test.
User question
query_text
Expected answer
golden_answer
Should answer?
should_answer
Returned source
relevant_doc_ids
Evidence
evidence_pages / source_text
Failure notes
diagnosis / issue type
Evaluation turns examples into repeatable tests.
What are we evaluating?
Not just the LLM.
A final answer can fail because of any stage in the chain.
Evaluation is like automated testing
Code testing
- not every possible input
- key paths and edge cases
- regression tests after fixes
- tests are repeatable
RAG evaluation
- not every possible user question
- important question types
- known failure modes
- same benchmark after each change
The goal is pragmatic coverage of the important moving parts.
Quick check
StatsChat returns the right document, but the generated answer uses the wrong number. Where is the main issue likely to be?
A. Ingestion
B. Retrieval
C. Generation / answer synthesis
D. User interface colour scheme
Best answer: C. Retrieval found useful evidence, but generation used it badly.
The automated evaluation loop
Evaluation is useful because it is repeatable.
Where do benchmark questions come from?
There is no single source of good evaluation questions.
Real users
Questions people already ask.
Subject experts
Questions statisticians know are important.
AI drafts
Useful, but human-reviewed.
Past failures
Regression tests after fixes.
User feedback
Questions that revealed confusion.
Known documents
Start from facts/tables and write questions.
“Backwards” question design
Start with evidence, then write the question.
This helps ensure the question is auditable.
How many questions are enough?
Development benchmark
Useful for early development and regression checks.
Stronger audited benchmark
Needed before relying heavily on headline scores.
Quality and range matter more than raw quantity alone.
What does the benchmark contain?
A good evaluation row needs more than a question.
Query
What the user asks
Golden answer
Expected answer, if answerable
Should answer?
Answer or refuse?
Source document
What publication should support it?
Evidence page/span
Where should support appear?
Notes/type
What behaviour is being tested?
The benchmark also needs quality assurance
A benchmark is also a statistical product: if the rows are weak, the metrics are weak.
Clear questions
Remove ambiguity.
Verified answers
Check against source.
Specific evidence
Document, page, quote/span.
Answerability
Clear should-answer decision.
Coverage
Reports, dates, metrics, refusal cases.
Review
Revise or remove weak rows.
Catch out specific components
Design questions that expose different failure points.
Ingestion
Tables, footnotes, awkward PDFs.
Chunking
Answer split across boundaries.
Retrieval
Similar terms, old editions, related-but-wrong documents.
Generation
Multiple nearby numbers or comparison figures.
Guardrails
Future, policy, opinion, out-of-corpus questions.
Traceability/UI
Cited page must support exact claim.
Answerable and unanswerable questions
Answerable
“What was Kenya’s real GDP growth rate in 2024?”
Expected behaviour: answer from retrieved evidence.
Unanswerable / guardrail
“What will Kenya’s GDP growth rate be in 2027?”
Expected behaviour: refuse or explain evidence is insufficient.
A system that answers everything is not necessarily better.
StatsChat benchmark example
The benchmark tests both whether StatsChat answers questions it should answer and whether it refuses questions it should not answer.
Headline results from best run
Strong result — but not a permanent certificate.
How results changed over time
initial full benchmark
small set looked strong
new weaknesses exposed
performance recovered
Benchmark expansion exposed weaknesses that the smaller set had missed.
Interpretation question
The smaller benchmark looked strong, but the expanded benchmark initially performed worse. What is the best interpretation?
A. The system became worse overnight
B. The expanded benchmark exposed more realistic variety
C. Evaluation is pointless
D. The answer generator was removed
Best answer: B. A better benchmark can reveal hidden weaknesses.
Three metric families
Answer metrics
When the system should answer, does it give the expected answer?
answerable accuracynumeric match
Guardrail metrics
When the system should not answer, does it refuse?
unanswerable accuracyfalse answer rate
Retrieval metrics
Did the system retrieve the right evidence?
Doc Hit@kpage hit
Guardrail check
Which should be included in a good StatsChat evaluation set?
A. Only easy factual questions
B. Only questions from one report family
C. Questions the system should refuse
D. Only questions with short numeric answers
Best answer: C. Refusal behaviour is part of system quality.
Retrieval vs generation diagnosis
Good retrieval + good generation
System working.
Good retrieval + bad generation
Prompt/model/synthesis issue.
Bad retrieval + good-looking answer
Dangerous: may be unsupported.
Right document, wrong page
Need better page-level grounding.
Why page-level grounding matters
Document retrieval is strong, but evidence-page grounding still needs improvement.
Failure analysis from best run
2 retrieval / edition failures
Wrong edition or policy source.
1 synthesis / variability failure
Correct document, but no answer.
1 evaluator edge case
Correct number, but unit/currency wording affected scoring.
No unanswerable rows failed in the best run.
Failure diagnosis question
The system retrieves the right report and the predicted answer contains the right number, but the evaluator marks it wrong because the answer omitted “KSh” and “million”. What is this mainly?
A. Pure retrieval failure
B. Evaluator or answer-formatting edge case
C. Out-of-scope guardrail failure
D. PDF ingestion failure
Best answer: B. The answer is close, but scoring needs to handle unit/currency wording carefully.
Model comparison: why isolate components?
Both runs used the same retrieval pipeline and had the same Pipeline Doc Hit@8: 56/61. Same retrieval, different answer synthesis.
Automated evaluation: what it can and cannot tell us
It can help us…
- rerun tests after changes
- track answer/retrieval metrics
- find regressions
- compare models or settings
- detect repeated failure patterns
It cannot fully…
- replace expert judgement
- prove the system is “done”
- capture every real user need
- solve ambiguous ground truth
- decide rollout risk alone
The benchmark is a decision aid, not a guarantee.
Evaluation improves the system
The process is not just “calculate a score”.
Metric → decision
How could evaluation improve next?
Grow the benchmark
Several hundred audited rows across report families.
More boundary cases
Policy, future, causal, cross-country, out-of-corpus.
Held-out split
A set not used while tuning.
Repeat runs
Measure LLM variability.
Better evidence checks
Page and span-level grounding.
User feedback loop
Turn real issues into benchmark rows.
Evaluation in a different NSO setting
Evaluation should match the intended use.
Documents
PDF, Excel, HTML, APIs.
Language
Terminology, translations, multilingual users.
Update cycles
Monthly, quarterly, annual, revisions.
User groups
Analysts, managers, public users.
Source status
Public, internal, confidential.
Risk tolerance
Internal search vs public answer service.
Example: another NSO benchmark
A small first benchmark might include:
Monthly inflation
Annual GDP
Definition / methodology
Latest publication
Table-heavy question
Revised figure
Refusal / guardrail
Non-PDF source
The set should reflect the organisation’s real documents and users.
Activity: design 3 evaluation questions
For your own NSO context, draft these three types of questions. For each, note the expected answer/source, whether the system should answer, and what failure would worry you most.
Factual
A straightforward answer from a known source.
Date-sensitive
Latest edition, revision, or period matters.
Cautious / refusal
The system should refuse or answer carefully.
Mini-project: adapting StatsChat
You do not need to finish this before Thursday.
Use case + users
Likely document sources
Key risks/challenges
What to keep from StatsChat
What to change
How to evaluate success
Thursday lab
The Thursday session is an evaluation design / mini-project clinic.
Bring a use case
Who would use the system?
Bring questions
What should the system answer or refuse?
Bring sources
Which publications or data sources matter?
We will refine ideas into a practical adaptation/evaluation plan.
What next for StatsChat evaluation?
A practical next-step roadmap:
Look how much of the system you can now reason about
At the start, StatsChat may have looked like one AI tool.
Now you can ask:
Sources
What documents went in?
Processing
How were they converted and chunked?
Retrieval
What evidence was selected?
Generation
How did the LLM use the evidence?
Traceability
Can the user check the answer?
Evaluation
How do we test whether it is safe?
What you can take away
Whether you adapt StatsChat or simply use AI systems more critically, you now have a practical framework:
Use it to ask better questions of any AI system that claims to answer from evidence.
Final takeaway
Evaluation closes the loop.
It turns a prototype into a system we can improve, compare, maintain, and make decisions about.
A useful RAG system is not just built once. It is tested, diagnosed, improved, monitored, and adapted to real users.
