Week 9: Introduction to Large Language Models

Welcome to Week 9

Theme: Introduction to Large Language Models

Goals:

• Position LLMs within AI, ML, and NLP

• Build intuition for tokens, next-token prediction, and attention

• Understand knowledge bases, chunking, embeddings, and semantic retrieval

AI, ML & NLP Foundations

AI
└── ML
  └── NLP
    └── LLMs

How LLMs Generate Text

• Text is split into tokens (words, pieces of words, punctuation).

• For each step, the model predicts the next most likely token.

• It repeats this process token-by-token to build a full response.

Token demo:

"Business Services and Administration Managers"
→ ["Business", " Services", " and", " Administration", " Managers"]

Next-token example after “Business Services and”:

Transformer and Attention

• Transformers can process token relationships efficiently.

• Attention helps the model decide which earlier words matter most for the next token.

• This is why wording and context influence outputs.

Example sentence:

"Code 1211 covers finance managers, and it includes detailed unit-group descriptions."

When predicting “it”, attention weights may focus on:

• Code 1211 (high)
• finance managers (high)
• unit-group descriptions (medium)

Takeaway:

• Attention is a relevance mechanism inside the model.

Embeddings & Similarity

• Embeddings turn text into vectors that represent meaning.

• Texts with similar meaning are close in vector space.

• Cosine similarity measures closeness between vectors.

• This enables meaning-based retrieval beyond exact keyword matches.

Knowledge Base (KB)

• A knowledge base is a trusted collection of documents for your domain.

• Examples: ISCO-08 group titles, definitions, task descriptions, and unit groups.

• The quality of the KB strongly affects answer quality.

• In practice: better context usually beats clever prompting.

Chunking Documents

• Large documents are split into smaller pieces called chunks.

• Chunking makes retrieval faster and more precise.

• Good chunks are focused, coherent, and contain enough context.

• If chunks are too large, retrieval gets noisy; if too small, meaning is lost.

Example:

Practical tip:

• Keep each chunk tied to code + title + definition (and keep task text together).

Vector Search and Semantic Search

• Vector search retrieves chunks whose embeddings are closest to the query embedding.

• Semantic search finds related meaning, not just matching words.

• Keyword search asks: “Do terms match?”

• Semantic search asks: “Does the meaning match?”

• In practice, semantic retrieval helps when users phrase questions differently from source text.

Mini comparison: