---
source: https://iinventedlight.com/library/notes/ao-memory-mechanisms-survey.md
title: "How LLM agent memory is designed and evaluated"
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
source_license: "CC BY 4.0; restated in our own words"
attribution: "Summary by iinventedlight (AI-assisted). Sources: arXiv, [2404.13501] A Survey on the Memory Mechanism of Large Language Model based Agents, CC BY 4.0; arXiv, [2610.03020] DyadMem: A Long-Term Memory Benchmark of How Agents Work with Users, CC BY 4.0."
author: "iinventedlight (synthesis of cited sources); text drafted with AI assistance"
date: 2026-11-04
status: published
checked_on: 2026-11-04
as_of: 2026-11-04
confidence: medium
untested: "No experiment was run; only the fetched arXiv abstract pages were read, so results are as reported by the authors."
category: agent-operations
topic_area: "agent-ops: memory"
tags: agents, memory, evaluation
sources:
  - id: S1
    title: "[2404.13501] A Survey on the Memory Mechanism of Large Language Model based Agents"
    publisher: arXiv
    url: https://arxiv.org/abs/2404.13501
    accessed: 2026-11-04
    license: "CC BY 4.0"
  - id: S2
    title: "[2610.03020] DyadMem: A Long-Term Memory Benchmark of How Agents Work with Users"
    publisher: arXiv
    url: https://arxiv.org/abs/2610.03020
    accessed: 2026-11-04
    license: "CC BY 4.0"
---
# How LLM agent memory is designed and evaluated

## Short answer
Memory is what lets an agent improve across long interactions. A 2024 survey reviews how memory modules are designed and evaluated. A 2026 benchmark, DyadMem, adds memory about how an agent should work with a specific user, and finds that models answer well when given the right memory but drop sharply when they must capture, update and recall it themselves. [S1][S2]

## Practical takeaways
- Test the full pipeline (capture, update, recall), not just final-answer QA. [S2]
- Watch for low capture recall and unsafe deletions. [S2]

## Claims
- id: C1
  claim: Memory supports agent-environment interactions.
  source: S1
  evidence: "The key component to support agent-environment interactions is the memory of the agents."
- id: C2
  claim: The survey reviews how to design and evaluate the memory module.
  source: S1
  evidence: "we systematically review previous studies on how to design and evaluate the memory module."
- id: C3
  claim: Gold-Memory QA is strong but Full-Pipeline QA drops sharply.
  source: S2
  evidence: "Gold-Memory QA is consistently strong, yet Full-Pipeline QA drops sharply."
- id: C4
  claim: Results reveal low capture recall and unsafe-deletion issues.
  source: S2
  evidence: "reveal low Capture recall, incomplete Recall, and unsafe-deletion issues"

## Sources
- [S1] [2404.13501] A Survey on the Memory Mechanism of Large Language Model based Agents. arXiv. https://arxiv.org/abs/2404.13501 . License: CC BY 4.0. Accessed 2026-11-04.
- [S2] [2610.03020] DyadMem: A Long-Term Memory Benchmark of How Agents Work with Users. arXiv. https://arxiv.org/abs/2610.03020 . License: CC BY 4.0. Accessed 2026-11-04.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted).