LLM-as-a-judge: language models grading other models, and the biases
LLM-as-a-judge uses a strong language model to score responses from other models. It has biases such as position, verbosity, and self-enhancement, but calibration methods can reduce them. Evaluation services from Microsoft and Google offer built-in evaluators and rubrics.
What is LLM-as-a-judge?
LLM-as-a-judge is an evaluation approach where one language model acts as a referee to grade responses produced by other models. It is used for open-ended questions that are hard to measure with fixed benchmarks.
The LLM-as-a-judge paper from arXiv reports that strong LLM judges can match human preferences, achieving over 80 percent agreement. This makes the approach a scalable approximation of human preferences, which are expensive to collect.
Built-in evaluators in cloud platforms
Microsoft Foundry provides built-in evaluators that assess coherence, fluency, retrieval, groundedness, relevance, and safety, as described in the Microsoft built-in evaluators reference. These evaluators also cover risk categories such as hate and unfairness, sexual content, violence, and self-harm.
Google's Gen AI evaluation service uses adaptive rubrics that generate unique pass or fail tests for each prompt. The Google Gen AI evaluation overview explains that these rubrics are similar to unit tests and help improve model performance across tasks.
OpenAI's evals guide explains how to create and run evaluations to assess model outputs. This guide is part of a broader set of API resources for working with evals.
Systematic biases in LLM judges
The fair evaluators study from arXiv found that changing the order of candidate responses can change the ranking. In one example, placing a response first caused a model to appear superior on 66 out of 80 test queries.
The study identifies position bias, verbosity bias, self-enhancement bias, and limited reasoning ability as common problems. These biases can make a judge score the same content differently depending on context.
Calibration strategies
To address these biases, the same study proposes calibration approaches. Multiple evidence calibration makes the judge produce several pieces of evidence before assigning a rating. Balanced position calibration aggregates scores across different response orders to reduce order effects.
Human-in-the-loop calibration measures the difficulty of each example and requests human assistance when the judge is uncertain. These strategies help align LLM judgments more closely with human ratings.
Cloud platforms also support similar mitigation. For instance, Google's evaluation service allows custom functions and static rubrics, which can be tailored to avoid order-dependent scoring. Microsoft's evaluators can be combined with human review for risk and safety categories.
| Bias | Effect | Mitigation |
|---|---|---|
| Position bias | Responses placed first are rated higher | Balanced position calibration |
| Verbosity bias | Longer responses tend to get better scores | Multiple evidence calibration |
| Self-enhancement bias | Judge prefers outputs similar to its own | Human-in-the-loop calibration |
| Limited reasoning | Judge misses errors in complex logic | Custom rubrics and human review |
Frequently asked questions
What is LLM-as-a-judge?
It is a method where a strong language model scores responses produced by other models, used for open-ended questions where human evaluation is costly.
What biases affect LLM judges?
Common biases include position bias, verbosity bias, self-enhancement bias, and limited reasoning ability, which can skew results.
How can these biases be reduced?
Calibration strategies such as balanced position calibration, multiple evidence calibration, and human-in-the-loop review are proposed by researchers.
Do cloud platforms offer LLM-as-a-judge tools?
Yes, Microsoft Foundry has built-in evaluators, Google offers adaptive rubrics in its Gen AI evaluation service, and OpenAI provides evaluation guides.