---
source: https://iinventedlight.com/library/notes/ao-evals-llm-judges.md
title: "LLM judges: an open evaluator model and the limits of judging"
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
source_license: "CC BY 4.0; restated in our own words"
attribution: "Summary by iinventedlight (AI-assisted). Sources: arXiv, [2310.08491] Prometheus: Inducing Fine-grained Evaluation Capability in Language Models, CC BY 4.0; arXiv, [2406.12624] Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges, CC BY 4.0."
author: "iinventedlight (synthesis of cited sources); text drafted with AI assistance"
date: 2026-11-04
status: published
checked_on: 2026-11-04
as_of: 2026-11-04
confidence: medium
untested: "No experiment was run; only the fetched arXiv abstract pages were read, so results are as reported by the authors."
category: agent-operations
topic_area: "agent-ops: evaluation"
tags: ['llm-as-judge', 'evaluation', 'rubrics', 'benchmarks']
sources:
  - id: S1
    title: "[2310.08491] Prometheus: Inducing Fine-grained Evaluation Capability in Language Models"
    publisher: arXiv
    url: https://arxiv.org/abs/2310.08491
    accessed: 2026-11-04
    license: "CC BY 4.0"
  - id: S2
    title: "[2406.12624] Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges"
    publisher: arXiv
    url: https://arxiv.org/abs/2406.12624
    accessed: 2026-11-04
    license: "CC BY 4.0"
---
# LLM judges: an open evaluator model and the limits of judging

**Answer:** Prometheus, an open 13B evaluator given a reference answer and score rubric, scores a Pearson correlation of 0.897 with human evaluators (S1). A separate study of thirteen judge models found that only the best (and largest) models achieve reasonable alignment with humans (S2).

## Prometheus (S1)

- Using a powerful proprietary LLM such as GPT-4 as an evaluator for long-form responses has become the de facto standard.
- For large-scale evaluation tasks and custom criteria, proprietary evaluators are unreliable due to the closed-source nature, uncontrolled versioning, and prohibitive costs.
- The Feedback Collection dataset consists of 1K fine-grained score rubrics, 20K instructions, and 100K responses and language feedback generated by GPT-4.
- With 45 customized score rubrics, Prometheus's 0.897 correlation is on par with GPT-4 (0.882) and greatly outperforms ChatGPT (0.392).

## Judging the judges (S2)

- The study focuses on a clean scenario in which inter-human agreement is high, with thirteen judge models judging answers of nine exam-taker models.
- Even the best judges are still quite far behind inter-human agreement, and their assigned scores may still differ with up to 5 points from human-assigned scores.
- For ranking the nine exam-taker models, smaller models and even a lexical metric may provide a reasonable signal.
- Judge models show sensitivity to prompt complexity and length, and a tendency toward leniency.
- Judges with high percent agreement can still assign vastly different scores, so alignment metrics beyond simple percent alignment matter.

## Practical reading

The S2 authors suggest that caution may be wise when using judges in more complex setups.

## Claims
- id: C1
  claim: Prometheus is a 13B evaluator LLM.
  source: S1
  evidence: "Using the Feedback Collection, we train Prometheus"
- id: C2
  claim: Prometheus greatly outperforms ChatGPT as an evaluator.
  source: S1
  evidence: "and greatly outperforms ChatGPT (0.392)."
- id: C3
  claim: Judge models tend toward leniency.
  source: S2
  evidence: "and a tendency toward leniency."

## Sources
- [S1] [2310.08491] Prometheus: Inducing Fine-grained Evaluation Capability in Language Models. arXiv. https://arxiv.org/abs/2310.08491 . License: CC BY 4.0. Accessed 2026-11-04.
- [S2] [2406.12624] Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges. arXiv. https://arxiv.org/abs/2406.12624 . License: CC BY 4.0. Accessed 2026-11-04.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted).