Does rules-file structure change whether agents follow it?
Short answer
In one factorial study, file size, position, architecture and conflicts had no detectable effect on compliance. What mattered was session length: compliance fell as the agent generated more functions. [S1] A separate ablation also found context strategy did not move correctness. [S2]
Practice
- Do not over-tune file layout.
- Re-check key rules during long sessions, or split work into shorter ones.
Claims
- id: C1 claim: Structural variables showed no detectable contrast. source: S1 evidence: "None of the four structural variables or three two-way interactions produces a detectable contrast"
- id: C2 claim: Each extra function lowered compliance odds about 5.6%. source: S1 evidence: "approximately 5.6% lower odds of compliance per step"
- id: C3 claim: 1,650 sessions were measured. source: S1 evidence: "across 1,650 Claude Code CLI sessions"
- id: C4 claim: 288 runs were evaluated. source: S2 evidence: "288 evaluated runs with gold-test evaluation"
Sources
- [S1] [2605.10039] Instruction Adherence in Coding Agent Configuration Files: A Factorial Study of Four File-Structure Variables. arXiv. https://arxiv.org/abs/2605.10039 . License: CC BY 4.0. Accessed 2026-11-04.
- [S2] [2607.27250] Do Context Files Help Coding Agents? A Two-Agent Ablation Study on Real Repositories. arXiv. https://arxiv.org/abs/2607.27250 . License: CC BY 4.0. Accessed 2026-11-04.
License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted).
License: CC-BY-4.0 · Summary by iinventedlight (AI-assisted). Sources: arXiv, [2605.10039] Instruction Adherence in Coding Agent Configuration Files: A Factorial Study of Four File-Structure Variables, CC BY 4.0; arXiv, [2607.27250] Do Context Files Help Coding Agents? A Two-Agent Ablation Study on Real Repositories, CC BY 4.0.
Cite this
iinventedlight. 2026. "Does rules-file structure change whether agents follow it?" iinventedlight Library, iil:note:99ccpvwp, version of 2026-11-04. https://iinventedlight.com/library/notes/ao-rules-instruction-adherence/. License: CC-BY-4.0.
@misc{iil_note_99ccpvwp,
author = {{iinventedlight}},
title = {{Does rules-file structure change whether agents follow it?}},
howpublished = {iinventedlight Library},
year = {2026},
date = {2026-11-04},
url = {https://iinventedlight.com/library/notes/ao-rules-instruction-adherence/},
note = {Permanent ID iil:note:99ccpvwp. License: CC-BY-4.0.},
}Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.
For language models and agents: start at /llms.txt. This page in Markdown: /library/notes/ao-rules-instruction-adherence.md.