← All notes

Short answer

Not much for task success, in two studies. One found context files did not generally raise success and cost over 20% more. A second found no measurable correctness change across two agents. [S1][S2]

What still helps

Agents do follow the instructions in these files, so use them for unusual project rules. Repository overviews were not helpful. [S1] Failures came from implementation skill, not missing repo knowledge. [S2]

Practice

Keep rules files short and specific, and measure before assuming a gain.

Claims

• id: C1 claim: Context files did not generally improve success. source: S1 evidence: "providing context files does not generally improve task success rates"
• id: C2 claim: Context files raised inference cost over 20%. source: S1 evidence: "increasing inference cost by over 20% on average"
• id: C3 claim: Repository overviews were not helpful. source: S1 evidence: "repository overviews, although popular and recommended by model providers, are not helpful"
• id: C4 claim: Context strategy did not move correctness. source: S2 evidence: "Context strategy does not measurably move correctness on either agent"
• id: C5 claim: Agents fail on implementation skill. source: S2 evidence: "agents fail on implementation skill"

Sources

• [S1] [2602.11988] Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?. arXiv. https://arxiv.org/abs/2602.11988 . License: CC BY 4.0. Accessed 2026-11-04.
• [S2] [2607.27250] Do Context Files Help Coding Agents? A Two-Agent Ablation Study on Real Repositories. arXiv. https://arxiv.org/abs/2607.27250 . License: CC BY 4.0. Accessed 2026-11-04.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted).

@misc{iil_note_6b0hpmer,
  author = {{iinventedlight}},
  title = {{Do AGENTS.md-style context files help coding agents?}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-04},
  url = {https://iinventedlight.com/library/notes/ao-rules-agents-md-evidence/},
  note = {Permanent ID iil:note:6b0hpmer. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt. This page in Markdown: /library/notes/ao-rules-agents-md-evidence.md.