---
source: https://iinventedlight.com/library/notes/2309-15217-ragas-rag-evaluation.md
title: "RAGAS: reference-free evaluation of RAG pipelines"
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
source_license: "CC BY 4.0; restated in our own words"
attribution: "Summary by iinventedlight (AI-assisted). Sources: arXiv, RAGAS: Automated Evaluation of Retrieval Augmented Generation, CC BY 4.0."
author: "iinventedlight (synthesis of cited sources); text drafted with AI assistance"
date: 2026-11-04
status: published
checked_on: 2026-11-05
as_of: 2026-11-05
confidence: medium
untested: "Results were not reproduced; only the arXiv abstract page was read."
category: ai-research
topic_area: "ai-research: evaluating retrieval-augmented generation"
tags: llm, rag, evaluation, retrieval, arxiv
sources:
  - id: S1
    title: "RAGAS: Automated Evaluation of Retrieval Augmented Generation"
    publisher: arXiv
    url: https://arxiv.org/abs/2309.15217
    accessed: 2026-11-05
    license: "CC BY 4.0"
---
# RAGAS: reference-free evaluation of RAG pipelines

## Short answer
Ragas is a set of metrics that scores a retrieval-augmented generation (RAG) system without human-labeled reference answers. It checks the retrieval step, whether the answer stays faithful to the retrieved passages, and the quality of the answer itself. [S1]

## What it measures
- **Retrieval:** did the retriever find relevant, focused context passages? [S1]
- **Faithfulness:** does the LLM use those passages faithfully rather than inventing facts? [S1]
- **Generation quality:** is the answer itself good? [S1]

## Why reference-free
Ground-truth answers are slow and costly to collect. Skipping them allows faster evaluation cycles, which the authors argue matters given how quickly LLM apps are being adopted. [S1]

## Practical use
Run the metrics on a fixed question set whenever you change the chunking, the embedding model, the retriever settings or the prompt. Spot-check with human review before trusting the numbers for a release.

## Claims
- id: C1
  claim: Ragas is a framework for reference-free evaluation of RAG pipelines.
  source: S1
  evidence: "a framework for reference-free evaluation of Retrieval Augmented Generation (RAG) pipelines"
- id: C2
  claim: RAG evaluation must consider whether the LLM exploits passages faithfully.
  source: S1
  evidence: "the ability of the LLM to exploit such passages in a faithful way"
- id: C3
  claim: The Ragas metrics do not rely on ground-truth human annotations.
  source: S1
  evidence: "without having to rely on ground truth human annotations"

## Sources
- [S1] RAGAS: Automated Evaluation of Retrieval Augmented Generation. arXiv. https://arxiv.org/abs/2309.15217 . License: CC BY 4.0. Accessed 2026-11-05.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted).