LLM Evaluation
Module 16 / 17
16 / 17
Model Evaluation & Benchmarks

LLM Benchmarks — Complete Guide

What is an LLM Benchmark?

An LLM Benchmark is a standardized test used to measure a specific capability of a language model.

Instead of asking,

"Is GPT-5 better than Claude?"

we ask

"How well do they perform on the exact same standardized test under identical conditions?"

That standardized test is called a benchmark.

Definition

A benchmark is a standardized evaluation framework that measures one or more capabilities of an AI model using predefined datasets, evaluation rules, and scoring methods.

Every benchmark is designed to test a particular capability such as

Anatomy of Every Benchmark

Every benchmark contains 4 core components.

Benchmark
│
├── Dataset / Task
├── Run Configuration
├── Scoring Method
└── Aggregation Method

1. Dataset / Task

Every benchmark provides

Think of it like an exam paper.

Example

Question:
Natalia sold 48 clips in April.
She sold half as many in May.

How many clips did she sell?

Answer:
72

The benchmark already knows the correct answer.

The model simply has to generate it.

Example: GSM8K

GSM8K = Grade School Mathematics 8K

Purpose

Measures

Mathematical reasoning

Contains

~8,500 grade-school math problems

Each row contains

Question

Correct Answer

The task is simple

Input Question

↓

LLM

↓

Generate Correct Answer

2. Run Configuration

A benchmark doesn't only provide questions.

It also specifies how the evaluation should be run.

Otherwise comparisons become unfair.

Example

Bad comparison

Model A
Temperature = 0

Model B
Temperature = 1.2

Obviously results aren't comparable.

Therefore every benchmark defines a standard configuration.

Run Configuration Includes

Prompt Construction

How should the prompt be built?

Common choices

Zero-shot

Only give the question.

Question

↓

Answer

Few-shot

Provide solved examples before the real question.

Example 1

Example 2

Example 3

...

Actual Question

Few-shot generally improves performance.

Chain-of-Thought (CoT)

Allow the model to reason step by step.

Instead of

Answer = 72

Model produces

48

Half of 48 = 24

48 + 24 = 72

Final Answer = 72

This greatly improves reasoning benchmarks.

GSM8K Configuration

Historically GSM8K uses

8-shot prompting

+

Chain-of-Thought reasoning

Decoding & Sampling Configuration

The benchmark also specifies inference parameters.

Examples

Temperature

Usually ≈ 0

Reason

Higher temperature introduces randomness.

Benchmarks need deterministic outputs.

Maximum Tokens

Too few tokens

↓

Model gets cut off before reasoning finishes.

Too many tokens

↓

Unfair advantage.

Hence max tokens are fixed.

Scoring Strategy

Benchmarks define how predictions are counted.

Pass@1

Model gets

1 attempt

Correct

↓

Pass

Wrong

↓

Fail

Strict evaluation.

Pass@K

Example

Pass@5

Ask the same question five times.

If any one answer is correct

↓

Question counts as correct.

This is much more lenient.

Majority@K

Ask

K times

Suppose answers are

72

72

70

72

71

Most frequent answer

72

becomes the final prediction.

Tool Usage

Benchmarks also define whether external tools are allowed.

Examples

Allowed

Not Allowed

Pure LLM reasoning.

Example

GSM8K

No tools allowed

SWE-bench

Tools required

GitHub

Terminal

File system

3. Scoring Method

After inference,

the benchmark must determine

Correct

or

Incorrect

This happens in two stages.

Step 1

Extract Final Answer

LLMs generate free-form text.

Example

The correct answer is 72.

Need to extract

72

Methods

Step 2

Compare Against Ground Truth

Example

Ground truth

72

Prediction

72

↓

Correct

Prediction

73

↓

Wrong

Open-ended Tasks

Some benchmarks don't have exact answers.

Example

Write a business proposal.

Summarize an article.

Review code.

Now exact matching is impossible.

Instead

Evaluation uses

LLM-as-a-Judge

Another LLM evaluates

Sometimes

Human evaluators

are also used.

4. Aggregation Method

Each question produces

Correct = 1

Wrong = 0

Suppose

1000 questions

920 correct

Final score

920 / 1000

=

92%

Simple.

Weighted Aggregation

Some benchmarks contain multiple subjects.

Example

MMLU

57 subjects

Biology

Physics

Law

Economics

Medicine

...

Each subject has different numbers of questions.

You cannot simply average percentages.

Instead

Weighted Mean

is used.

Benchmark Evaluation Pipeline

The entire benchmark evaluation process follows a loop.

Load Question

↓

Build Prompt

↓

Apply Benchmark Settings

↓

Call Model

↓

Receive Response

↓

Extract Answer

↓

Compare with Ground Truth

↓

Store Score

↓

Repeat for Entire Dataset

↓

Aggregate Scores

↓

Final Benchmark Score

Step-by-Step Evaluation Workflow

Step 1

Load one benchmark example.

Step 2

Construct the prompt.

Include

Step 3

Run model using benchmark settings.

Example

Temperature

Max Tokens

Top-p

Stop Tokens

Step 4

Capture model output.

Step 5

Extract prediction.

Step 6

Compare with answer key.

Step 7

Store

1

or

0

Step 8

Repeat for every example.

Step 9

Aggregate scores.

Final benchmark score

Accuracy

Pass Rate

F1

etc.

Why Evaluation Isn't Just a Simple Loop

Conceptually

Evaluation is easy.

Practically

It's much harder.

Real systems must handle

This infrastructure is called an

Evaluation Harness

Evaluation Harness

An Evaluation Harness is software that automates the entire benchmark evaluation process.

Instead of writing thousands of lines of evaluation code,

you simply specify

Model

Benchmark

Configuration

and the harness does everything else.

Think of it as

Benchmark

=

Exam Paper

Evaluation Harness

=

Exam Administration System

Popular Evaluation Harnesses

LM Evaluation Harness

Most widely used benchmark framework.

Supports

Widely used by

Inspect

Another evaluation framework focused on systematic model evaluation.

DeepEval

Primarily focused on

Also supports some standard benchmarks.

Three Groups That Perform Model Evaluation

1. Frontier Labs (Model Builders)

Examples

Why they evaluate

Internal Development

Release Gating

Marketing

How to interpret their numbers

Treat them as the best-case (ceiling) performance because:

Do not rely solely on vendor-reported benchmark scores when selecting a model.

2. Third-Party Evaluators

Examples

Why they evaluate

Evaluation is their core product.

They independently benchmark models under the same conditions to provide neutral comparisons.

Why they are valuable

They standardize:

They also report practical metrics often omitted by labs:

Their results are generally more trustworthy for engineers comparing models.

3. Companies & AI Engineering Teams

General benchmarks answer questions like:

But they do not answer:

Therefore, engineering teams run their own evaluations using:

to measure performance under their own constraints.

Common Problems with Benchmarks

Benchmarks are useful, but not perfect. Four major issues can make benchmark scores misleading.

1. Benchmark Contamination

Public benchmarks are freely available online.

If benchmark questions appear in a model’s pretraining data, the model may simply memorize the answers instead of reasoning.

This leads to inflated benchmark scores.

Solutions

2. Benchmark Saturation

As models improve, many eventually score 95–99% on older benchmarks.

When almost every model gets similar scores, the benchmark no longer distinguishes between them.

Such benchmarks are considered saturated and are replaced by newer, harder benchmarks.

Examples of largely saturated benchmarks include GSM8K, MMLU, and SWE-bench (for recent frontier models).

3. Configuration Gaming

Benchmark scores can change dramatically depending on evaluation settings.

Examples of manipulation include:

Even small configuration changes can significantly affect results.

Always verify that competing models were evaluated under identical conditions.

4. Aggregation Bias

Many benchmarks evaluate multiple subjects or tasks.

A single overall score can hide weaknesses in specific domains.

Example:

A model may perform very well in:

but poorly in:

A high average score may conceal this weakness.

Always examine per-category scores in addition to the overall benchmark score.

Key Takeaways