← All notes

Summary

AMA Agent achieves 57.22% accuracy on AMA Bench, outperforming the strongest baseline by 11.16%. Systematic evaluation indicates that agent memory remains challenging even for frontier commercial models, with GPT 5.2 achieving only 72.26% accuracy. Resources are available at https://ama-bench.github.io/. [S1]

Claims

• id: C1 claim: 'AMA-Agent is reported at 57.22% accuracy on AMA-Bench.' source: S1 evidence: "GPT 5.2 achieving only 72.26% accuracy"
• id: C2 claim: 'That score is reported as 11.16 points above the strongest baseline.' source: S1 evidence: "outperforming the strongest baseline by 11.16"
• id: C3 claim: 'Resources are pointed to a public project page.' source: S1 evidence: "Resources are available at: https://ama-bench.github.io/"

Verification

• Code available: Paper: resources at https://ama-bench.github.io/ (HTTP 200 on 2026-11-11). No official GitHub repository URL was found on the abs page or in the HTML text. Nothing was run.
• Reproduced: not reproduced (2026-11-11). Code not run.
• Related work: Live note ao-memory-long-term-benchmarks. DyadMem and LongMemEval in this batch measure memory in different settings.
• Limits: Author-reported. No GitHub repository URL was found; a project page is named. Some compared systems in the paper are cited as anonymous preprints.
• License: CC BY 4.0 on the abs page (2026-11-11; icon by-4.0.png; http://creativecommons.org/licenses/by/4.0/). Submitted 26 Feb 2026 (v1); last revised 27 May 2026 (v4). Sources: [S1] https://arxiv.org/abs/2602.22769, v4 (27 May 2026). checked_on: 2026-11-11.

When to apply

Use when asking how well an agent, not just a chatbot, remembers state and causal information over a long task history.

Facts

• id: F1 claim: 'The authors report that AMA-Agent reached 57.22% accuracy on AMA-Bench, 11.16 points above the strongest baseline they compare.' confidence: medium date: 2026-11-11 sources: [S1] license: CC BY 4.0
• id: F2 claim: 'They report that GPT 5.2 reached only 72.26% accuracy on their evaluation.' confidence: medium date: 2026-11-11 sources: [S1] license: CC BY 4.0
• id: F3 claim: 'Resources are stated to be available at https://ama-bench.github.io/.' confidence: medium date: 2026-11-11 sources: [S1] license: CC BY 4.0

Sources

• [S1] arXiv 2602.22769, https://arxiv.org/abs/2602.22769. License: CC BY 4.0. v4 (27 May 2026). Accessed 2026-11-11.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted). Paper is CC BY 4.0.

@misc{iil_note_qmafg37p,
  author = {{'iinventedlight (AI-assisted)'}},
  title = {{AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-11},
  url = {https://iinventedlight.com/library/notes/2602-22769-ama-bench/},
  note = {Permanent ID iil:note:qmafg37p. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/2602-22769-ama-bench.md.