03 / 17

Why Evaluation Matters

LLM Evaluations Notes (Part 3)

In this part, we'll cover why evaluating LLM applications is fundamentally different from testing traditional software. This is one of the most important concepts for AI Engineers and is frequently discussed in interviews.

Chapter 6: Why LLM Evaluation is Hard

English Translation

After explaining why evaluation is important, the speaker asks:

"If evaluation is so important, why doesn't everyone evaluate their LLM applications?"

The answer is simple:

Because evaluating LLM applications is not straightforward.

Compared to traditional software, LLM-based applications are much more difficult to test due to their probabilistic nature and multiple quality dimensions.

Traditional Software vs LLM Applications

Traditional Software

Imagine building a calculator.

Input:

2 + 2

Output:

No matter who uses it or when they use it:

2 + 2 = 4

always.

This property is called Determinism.

Deterministic System

Definition

A deterministic system always produces the same output for the same input.

Input
   │
   ▼
Program
   │
   ▼
Same Output

Example:

5 × 10

Output

Every single time.

Examples

Calculator

10 / 2 = 5

Sorting Algorithm

Input

[5,3,2]

Output

[2,3,5]

Binary Search

Same input

↓

Same output

Every time.

Characteristics of Deterministic Systems

✔ Predictable

✔ Repeatable

✔ Easy to Test

✔ Easy to Debug

✔ Easy to Measure

LLM Applications

Now consider ChatGPT.

Prompt:

Explain Machine Learning.

Today's answer:

Machine Learning is a branch of AI...

Tomorrow's answer:

Machine Learning enables computers to learn...

Next week:

ML is a field where algorithms improve using experience...

All answers are correct.

But they are different.

Why?

Because LLMs are Probabilistic Models.

Probabilistic System

Definition

A probabilistic system can generate multiple valid outputs for the same input.

Prompt
      │
      ▼
     LLM
      │
      ├─────────────► Output 1
      │
      ├─────────────► Output 2
      │
      └─────────────► Output 3

Same Prompt

↓

Different Answers

↓

All can be correct.

Why Does This Happen?

LLMs don't "look up" one fixed answer.

Instead, they predict the next most probable token repeatedly until the response is complete.

At every step, the model chooses from many possible words.

Example

Prompt

The capital of France is

Possible next tokens

Paris
99.7%

Lyon
0.2%

Marseille
0.1%

The model selects based on probability.

For creative prompts, probabilities are much closer, producing more varied outputs.

Interview Question

Why are LLMs called probabilistic models?

Because they generate text by predicting the probability distribution of the next token rather than executing fixed rules. Therefore, the same prompt can produce different valid responses.

Consequence for Testing

Suppose you ask

Explain Neural Networks.

Version 1

Uses 120 words.

Version 2

Uses 300 words.

Version 3

Uses examples.

Version 4

Uses equations.

Which one is correct?

Actually...

All of them.

So unlike software testing,

there isn't always a single expected output.

Biggest Challenge

Traditional Testing

Expected Output
        │
        ▼
Compare
        │
        ▼
Pass / Fail

LLM Evaluation

Prompt
      │
      ▼
Many Possible Correct Answers
      │
      ▼
Need Quality Measurement

Instead of asking

Is it correct?

we ask

How good is this answer?

Chapter 7: Single Metric vs Multi-Dimensional Evaluation

Traditional software usually has one important metric.

Example

Calculator

Question

10 × 5

Answer

Correct?

YES

Done.

Only One Metric

Correct

Incorrect

Nothing else matters.

LLM Evaluation

Imagine building a RAG chatbot.

Question

What is Transformer Architecture?

Suppose the answer is technically correct.

Can we deploy it immediately?

No.

We need to evaluate many more aspects.

Multiple Evaluation Dimensions

Prompt
      │
      ▼
LLM Response
      │
      ▼
┌────────────────────┐
│ Factuality         │
│ Completeness       │
│ Groundedness       │
│ Tone               │
│ Relevance          │
│ Safety             │
│ Latency            │
│ Cost               │
└────────────────────┘

1. Factuality

Question

Is the answer factually correct?

Example

Who discovered Gravity?

Correct

Isaac Newton

Wrong

Albert Einstein

2. Completeness

Question

Did the answer cover everything?

Example

Question

Explain RAG.

Poor

RAG combines search and LLM.

Good

Retrieval
Embedding
Vector Database
Retriever
Prompt Construction
LLM Generation

Much more complete.

3. Groundedness

Question

Did the answer come from the retrieved documents?

Suppose your PDF says

Maximum leave = 20 days.

Chatbot replies

Maximum leave = 40 days.

Even if it sounds convincing,

it is not grounded.

Groundedness checks whether the response is supported by the provided context.

4. Tone

Example

Healthcare chatbot.

User

I have cancer.

Bad response

That's unfortunate.

Better response

I'm sorry to hear that. Please consult a qualified medical professional for personalized advice.

Same information.

Different tone.

Tone matters.

5. Latency

How long does it take to answer?

User
 │
 ▼
Prompt
 │
 ▼
LLM
 │
 ▼
Response

If users wait 30 seconds,

they will likely abandon the application.

6. Cost

Every API call costs money.

Example

GPT-4

↓

Long Response

↓

More Tokens

↓

Higher Cost

Evaluation includes checking whether the system is financially sustainable.

Other Metrics

Production systems may also evaluate:

Fluency
Relevance
Helpfulness
Toxicity
Bias
Privacy
Robustness
Consistency
Hallucination Rate
Citation Quality
Faithfulness
Safety

Evaluation Pyramid

                Safety
           Hallucination
         Groundedness
      Completeness
      Relevance
      Correctness

Each layer contributes to overall quality.

Why Different Applications Need Different Metrics

Customer Support Bot

Important metrics:

Accuracy
Tone
Latency

Medical Assistant

Important metrics:

Factuality
Hallucination
Safety

Coding Assistant

Important metrics:

Correctness
Executability
Efficiency

RAG Chatbot

Important metrics:

Retrieval Quality
Groundedness
Faithfulness
Citation Quality

AI Agent

Important metrics:

Planning
Tool Usage
Task Completion
Recovery from Errors

Interview Questions

1. Why is LLM evaluation harder than software testing?

Because LLMs are probabilistic systems. The same input can produce multiple valid outputs, so evaluation must assess response quality across multiple dimensions rather than comparing against one fixed expected answer.

2. What is a deterministic system?

A deterministic system always produces the same output for the same input.

Example: Calculator, Binary Search, Sorting Algorithms.

3. What is a probabilistic system?

A probabilistic system generates outputs based on probability distributions, allowing multiple valid responses for the same input.

Example: ChatGPT, Claude, Gemini.

4. What are the major evaluation dimensions of an LLM response?

Common dimensions include:

Factuality
Completeness
Groundedness
Tone
Relevance
Safety
Latency
Cost
Faithfulness
Robustness

Summary (Part 3)

Traditional software is deterministic: the same input always produces the same output.
LLMs are probabilistic: the same prompt can generate multiple valid responses.
Traditional software is typically evaluated with a simple pass/fail criterion.
LLM applications require multi-dimensional evaluation, considering not only correctness but also factuality, completeness, groundedness, tone, latency, cost, safety, and more.
Different AI applications prioritize different evaluation metrics depending on their domain and purpose.

In Part 4, we'll dive into the learning roadmap introduced in the transcript:

What LLM Evals actually are.
The LLM Evaluation Landscape (frameworks, metrics, techniques, and tools).
LLM benchmarks such as MMLU, HellaSwag, HumanEval, GSM8K, and others.
Building a custom evaluation pipeline with golden datasets and rubrics.
Evaluation strategies for RAG systems, AI agents, safety, and operational monitoring.