← All notes

Short answer

Not well yet. LongMemEval saw a 30% accuracy drop for remembering across sessions. AMA-Bench found memory systems fail on agent trajectories because similarity retrieval loses causal detail. [S1][S2]

Design hints

• Split memory into indexing, retrieval and reading; index by facts and time. [S1]
• For agents, store causal links between steps, not just similar text. [S2]

Claims

• id: C1 claim: Assistants show a 30% accuracy drop across sustained interactions. source: S1 evidence: "showing a 30% accuracy drop on memorizing information across sustained interactions"
• id: C2 claim: Memory design has three stages. source: S1 evidence: "three stages: indexing, retrieval, and reading"
• id: C3 claim: Existing memory systems rely on lossy similarity retrieval. source: S2 evidence: "rely heavily on lossy similarity-based retrieval"
• id: C4 claim: Existing benchmarks are dialogue-centric. source: S2 evidence: "existing memory benchmarks are largely dialogue-centric"

Sources

• [S1] [2410.10813] LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory. arXiv. https://arxiv.org/abs/2410.10813 . License: CC BY 4.0. Accessed 2026-11-04.
• [S2] [2602.22769] AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications. arXiv. https://arxiv.org/abs/2602.22769 . License: CC BY 4.0. Accessed 2026-11-04.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted).

@misc{iil_note_4s62858p,
  author = {{iinventedlight}},
  title = {{How well do assistants and agents remember over time?}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-04},
  url = {https://iinventedlight.com/library/notes/ao-memory-long-term-benchmarks/},
  note = {Permanent ID iil:note:4s62858p. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt. This page in Markdown: /library/notes/ao-memory-long-term-benchmarks.md.