LLM Eval Methods: Code Checks, Human Review and LLM-as-Judge
An eval method is the procedure that turns one output into a verdict. See code, a judge model and a person each judge the same five customer emails, and learn when each one is the right tool.
What an eval method is
This is part 4 of the LLM Evals series, and it stands on its own. An eval method is the procedure that takes one output from your app and turns it into a verdict: pass or fail, a score, or a ranking. Criteria say what good means. The method is who or what decides.
There are three: code decides, a judge model decides, or a person decides. To see why you need all three, take one small app. A coffee-subscription company gets a few hundred customer emails a day. Its assistant reads each email and returns two things:
- Labels as JSON, like
{"category": "billing", "priority": "high"}, which the company's code uses to route the email to the right queue. - A draft reply for a support agent to send.
The labels have one right answer. The reply doesn't: many replies are good, some are rude, and some promise things the company can't give. Different outputs need different judges.
The test set: five emails
Here is a tiny labelled test set. Each row has the email, the correct labels a person assigned, and what the assistant produced. Every method below judges these same five rows.
- E1 "I was charged twice for my October box." Expected billing, high. Got
{"category": "billing", "priority": "high"}. Reply: "Sorry about the double charge. We'll refund the duplicate charge." - E2 "My parcel says delivered, but it isn't here." Expected delivery, normal. Got
Sure! {"category": "delivery", "priority": "normal"}. Reply: "Sorry it hasn't arrived. We'll check with the courier and replace it if it's lost." - E3 "How do I pause my subscription for a month?" Expected account, low. Got
{"category": "billing", "priority": "low"}. Reply: "You can pause it from your account page, under Subscription." - E4 "The beans in this box tasted stale." Expected product, normal. Got
{"category": "product", "priority": "normal"}. Reply: "Sorry about that! We'll send you a free bag with your next box." - E5 "Third late box in a row. Refund me in full or I'm calling my lawyer." Expected delivery, high. Got
{"category": "delivery", "priority": "high"}. Reply: "We're so sorry. This is entirely our fault."
The company's policy: replies may promise to refund a duplicate charge or replace a lost parcel. Nothing else, and anything mentioning legal action goes to a person.
Programmatic checks: code decides
When correctness is mechanical, a program can judge it. For the labels, three checks are enough: does the output parse as JSON, are both fields allowed values, and do they exactly match the labelled answer?
- E2 fails: the labels are right, but the model wrapped them in "Sure!", so
json.loadsthrows. That isn't pedantry. The routing code would crash in exactly the same way. - E3 fails: valid JSON, but
billingwhere the answer isaccount, so the email lands in the wrong queue. - E1, E4 and E5 pass.
Other mechanical checks work the same way: a regex for an order number, a JSON schema, or running the output, such as executing generated code against unit tests or running generated SQL and comparing the rows it returns.
Strengths: fast, nearly free, and reproducible, so the same output gets the same verdict every time and you can run thousands of cases on every change. Limits: it only works where right and wrong can be written as a rule. No regex can tell you whether E4's reply is acceptable. Use it for formats, exact labels, extracted fields, code and SQL, and run it first.
LLM-as-judge: a model decides
The replies have no single right answer, but you can describe a good one. That's the job of model-graded evaluation, often called LLM-as-judge: you send a strong model a rubric, the input and the output, and it returns a score with a reason. The rubric for the replies:
Score the reply from 1 to 5; 4 or more is a pass.
- Polite, and answers what the customer actually asked.
- Promises only what policy allows: refunding a duplicate charge or replacing
a lost parcel. No free items, discounts or full refunds.On E4 the judge returns 2/5, fail: polite and on topic, but it promises a free bag, which policy doesn't allow. The code checks passed E4, because its labels were perfect. Only a reader of the reply could catch this.
A judge can also compare two replies and pick the better one (pairwise), which is handy when testing a new prompt against the old one. Strengths: it scales to thousands of replies, and it's far faster and cheaper than people. Use it for fuzzy criteria such as helpfulness, tone and groundedness.
Limits: the judge is itself a model, so it can be wrong. Zheng et al. (2023), the paper behind the MT-Bench benchmark, measured two biases and saw signs of a third. Each has a common counter:
- Position bias: in a pairwise comparison, preferring an answer because of where it appears. Swap the order, judge twice, and count only verdicts that hold both ways.
- Verbosity bias: preferring longer answers. Saying in the rubric that length earns no points helps, but still check the judge isn't rewarding length.
- Self-preference: favouring answers the judge model wrote itself. Zheng et al. saw only signs of it; Panickssery et al. (2024) later showed it directly. Use a judge from a different model family than the one being judged.
Human review: a person decides
People rate, rank or compare outputs directly. It's the slowest and most expensive method, and the gold standard for judgments that are nuanced, subjective or high stakes.
Take E5. Its labels pass, and because it mentions a lawyer, a simple rule routes it to a support lead. A rule can route the email, but it can't judge the reply. The judge gave the draft 5/5: polite, on topic, nothing promised. The support lead fails it at once: it admits in writing that the delays are "entirely our fault", to a customer threatening legal action. They remove the admission, and because this is the third late box in a row, they go beyond policy and offer a full refund to settle the dispute. No rubric can make that call. Even after the rubric gains a legal-risk line, legal threats still go to a person, because a wrong reply costs too much.
Humans do two jobs in a suite. They judge the high-stakes cases directly, and they label the calibration set that tells you whether the judge can be trusted.
Limits: besides cost and speed, people disagree. Two reviewers may score the same reply 3 and 5. Give them written guidelines with examples of each score, have two people label some of the same rows, and check how often they agree. If people can't agree on a criterion, you have nothing to calibrate a judge against.
Layering the three in one suite
The expert rule: use the cheapest method that can actually judge the criterion. Real suites layer them, code first, a judge for the fuzzy criteria, humans for calibration and high stakes. Here's the whole thing over the five emails. The fake_judge function is a stub standing in for a real LLM call, so the sketch runs with no network or API key.
import json
CATEGORIES = {"billing", "delivery", "account", "product"}
PRIORITIES = {"low", "normal", "high"}
HIGH_STAKES = ("lawyer", "legal", "court")
# The five emails: the labelled answer, the raw labels the assistant returned, and its draft reply.
DATASET = [
("E1", "I was charged twice for my October box.", ("billing", "high"),
'{"category": "billing", "priority": "high"}',
"Sorry about the double charge. We'll refund the duplicate charge."),
("E2", "My parcel says delivered, but it isn't here.", ("delivery", "normal"),
'Sure! {"category": "delivery", "priority": "normal"}',
"Sorry it hasn't arrived. We'll check with the courier and replace it if it's lost."),
("E3", "How do I pause my subscription for a month?", ("account", "low"),
'{"category": "billing", "priority": "low"}',
"You can pause it from your account page, under Subscription."),
("E4", "The beans in this box tasted stale.", ("product", "normal"),
'{"category": "product", "priority": "normal"}',
"Sorry about that! We'll send you a free bag with your next box."),
("E5", "Third late box in a row. Refund me in full or I'm calling my lawyer.",
("delivery", "high"), '{"category": "delivery", "priority": "high"}',
"We're so sorry. This is entirely our fault."),
]
RUBRIC = """Score the reply from 1 to 5; 4 or more is a pass.
- Polite, and answers what the customer actually asked.
- Promises only what policy allows: refunding a duplicate charge or replacing
a lost parcel. No free items, discounts or full refunds."""
def check_labels(output, expected):
"""Programmatic: code decides. Parse, check allowed values, compare to the label."""
try:
labels = json.loads(output)
except json.JSONDecodeError:
return "fail, not valid JSON"
category, priority = labels.get("category"), labels.get("priority")
if category not in CATEGORIES or priority not in PRIORITIES:
return "fail, value not allowed"
if (category, priority) != expected:
return f"fail, got {category} {priority}, expected {' '.join(expected)}"
return "pass"
def fake_judge(rubric, email, reply):
"""Stands in for an LLM call that reads the rubric, email and reply."""
if "free" in reply.lower():
return 2, "promises a free bag, which policy doesn't allow"
return 5, "polite, on topic, no promise beyond policy"
for row_id, email, expected, output, reply in DATASET:
score, reason = fake_judge(RUBRIC, email, reply)
verdict = "pass" if score >= 4 else "fail"
print(f"{row_id} labels: {check_labels(output, expected)}")
print(f" reply: {score}/5 {verdict}, {reason}")
if any(word in email.lower() for word in HIGH_STAKES):
print(" high stakes: send to a person")
Each method catches something the others miss: code catches E2 and E3, the judge catches E4, and a person, reached by a routing rule, catches E5. In a real suite the code checks run on every change, the judge runs on every reply in the test set, and people review the high-stakes rows plus a sample of the rest.
Summary
Key takeaways
- An eval method is the procedure that turns one output into a verdict; criteria say what good means, the method says who or what decides.
- Programmatic checks are fast, cheap and reproducible, but only judge what can be written as a rule: formats, exact labels, code and SQL results.
- LLM-as-judge scores outputs with no single right answer against a rubric, but it's a model too: it can favour answers for their position or length, and its own.
- Human review is the slowest and most expensive method and the gold standard for nuanced, subjective and high-stakes judgments.
- Use the cheapest method that can judge each criterion, and calibrate the judge against human labels before trusting it.
Practice
Check your understanding
0 of 5 answered
Finished this lesson?
Mark it done to track your progress. Saved in this browser only.
FAQ
Frequently asked questions
What is LLM-as-a-judge?
It's an eval method where a strong language model scores or compares outputs against a rubric you write, returning a score and a reason. It's used for criteria with no single right answer, such as tone, helpfulness or groundedness, and it should be checked against human labels before you rely on it.
When should you use human evaluation instead of an LLM judge?
Use people for high-stakes cases, such as legal threats, medical or financial advice, for subjective judgments a rubric can't pin down, and to label a calibration set that tells you whether the judge agrees with humans. For mechanical checks use code; for fuzzy criteria, a calibrated judge is faster and cheaper than people.
What are the biases of LLM-as-a-judge?
Zheng et al. (2023) measured position bias, preferring an answer because of where it appears in a pairwise comparison, and verbosity bias, preferring longer answers, and saw signs of self-preference, which Panickssery et al. (2024) later showed directly. Counter them by judging both orders, saying length earns no points, and using a judge from a different model family.
