LLM Evaluation
Module 8 / 17
08 / 17
Building an Evaluation Practice

LLM Evaluation Notes (Part 3): Why One LLM Application Needs Multiple Evaluation Pipelines

Quick Recap

In the previous lecture, we covered three fundamental questions:

1. Why do we need LLM Evaluations?

Without evaluation, deploying an LLM application directly to production can cause:

Goal: Ensure the system is reliable before deployment.

2. What are LLM Evaluations?

LLM Evaluations are:

A systematic and reliable way of evaluating LLMs or LLM-based applications against predefined criteria.

There are two major types:

Model Evaluation

Evaluates the language model itself.

Examples:

Mostly performed by Frontier Labs.

Application Evaluation

Evaluates the complete LLM application built using one or more models.

Examples:

This course mainly focuses on Application Evaluations.

3. How are LLM Applications Evaluated?

Typical evaluation workflow:

Define Task
      ↓
Define Success Metrics
      ↓
Build Golden Dataset
      ↓
Choose Evaluation Method
      ↓
Run System
      ↓
Evaluate Results
      ↓
Analyze Errors
      ↓
Improve System
      ↓
Repeat
      ↓
Deploy
      ↓
Monitor Production
      ↓
Add Production Failures to Dataset
      ↓
Repeat Evaluation

Today's Topic

Why does one LLM application require multiple evaluation pipelines?

Most beginners think:

"I'll evaluate my application once."

Reality:

One application usually needs many evaluation pipelines.

Why?

Two major reasons:

  1. Multiple Failure Points
  2. Multiple Risk Categories

Reason 1 — Multiple Failure Points

Consider a simple RAG chatbot.

Architecture:

User Query
     │
     ▼
Retriever
     │
     ▼
Vector Database
     │
Relevant Documents
     │
     ▼
Generator (LLM)
     │
     ▼
Final Answer

Failure Point 1 — Retriever

Retriever's job:

Query
   ↓
Find Relevant Documents

Possible failures:

Therefore we need:

Retriever Evaluation Pipeline

Checks:

Failure Point 2 — Generator

Generator receives:

Its job:

Generate answer only from retrieved context.

Possible failures:

Therefore we need:

Generator Evaluation Pipeline

Checks:

Important Observation

Even if:

✅ Retriever works

AND

✅ Generator works

The overall pipeline may still fail.

Example

User asks:

What is the duration of the ML course?

Retriever returns Top-5 documents:

D1 → Python course (6 weeks)

D2 → Java course

D3 → AI course

D4 → Random

D5 → ML course (8 weeks)

Retriever technically succeeds because the correct document is in Top-5.

Generator receives:

D1
D2
D3
D4
D5

Suppose the generator gives higher priority to earlier documents.

It generates:

ML course duration = 6 weeks

Wrong answer.

Did Retriever fail?

No.

Correct document existed.

Did Generator fail?

Not necessarily.

It followed the prompt using higher-ranked documents.

Yet the application failed.

Why?

Because the interaction between Retriever and Generator failed.

Therefore we need another evaluation

Workflow-Level Evaluation

Instead of evaluating components separately:

Retriever ✔

Generator ✔

We evaluate:

Retriever
      +
Generator
      +
Interaction

This checks whether the complete pipeline produces the correct output.

Possible fixes after workflow evaluation:

Workflow Evaluation Example

Query
   ↓
Retriever
   ↓
Documents
   ↓
Generator
   ↓
Answer

Evaluation checks:

But is that enough?

Suppose:

Retriever ✔

Generator ✔

Workflow ✔

Is the application now perfect?

No.

Application-Level Problems

Example:

Everything is correct.

But response takes:

10 seconds

Correct?

Yes.

Deployable?

No.

Users won't wait 10 seconds.

Therefore we also need

Application-Level Evaluation

Checks:

Three Levels of Evaluation

LLM Application

↓

1. Component Level

Evaluate each component individually.

Examples:

2. Workflow Level

Evaluate interaction between components.

Examples:

Retriever
      ↓
Generator

or

Agent
 ↓
Tool
 ↓
Memory
 ↓
Planner

3. Application Level

Evaluate complete system.

Examples:

Why Multiple Evaluations?

Because failures can occur at three levels.

Application
      │
      ▼
Workflow
      │
      ▼
Components

Each level requires its own evaluation.

Reason 2 — Multiple Risk Categories

Even for the same component, we evaluate different kinds of risks.

Example:

Retriever.

Questions:

These are different evaluation dimensions.

Three Major Risk Categories

Risk Categories
│
├── Application Quality
├── Safety
└── Operations

1. Application Quality

Checks whether the application performs its intended task correctly.

Questions:

General LLM Applications

Metrics:

Example:

Text summarizer.

Check:

RAG Applications

Metrics:

Agent Applications

Metrics:

Multi-turn Chatbots

Metrics:

2. Safety

Ensures outputs are safe.

Major safety evaluations:

Toxicity

No abusive or offensive content.

Harmful Content

Avoid generating:

Bias

Avoid unfair treatment based on user characteristics.

Privacy

Prevent leakage of:

Prompt Injection & Jailbreak Resistance

Checks whether attackers can manipulate the model into ignoring instructions or revealing restricted information.

3. Operations

Measures production performance.

Common metrics:

Complete Picture

LLM Application
│
├── Component Evaluations
│     ├── Retriever
│     ├── Generator
│     ├── Embeddings
│     ├── Reranker
│     └── Guardrails
│
├── Workflow Evaluations
│     ├── RAG Pipeline
│     ├── Agent Workflow
│     └── Multi-turn Flow
│
└── Application Evaluations
      ├── Quality
      ├── Safety
      └── Operations

Why Do We Need Multiple Evaluation Pipelines?

Reason 1: Multiple Failure Points

Failures can happen at:

Each needs its own evaluation.

Reason 2: Multiple Risk Categories

Every failure point must be evaluated for different risks:

Key Takeaways