SpecCompact: Accelerating LLM Agent Context Compaction with Slot-Parallel Semantic Speculative Verification
Summary
SpecCompact fine-tunes a small drafter and generates slots in parallel. System optimizations further reduce repeated context processing and model calls. A user study within Hermes further shows an 8.68 times median compaction speedup. SpecCompact reduces median compaction latency from 63.74 to 7.28 seconds. [S1]
Claims
- id: C1 claim: 'They report 2.04-5.63 times compaction speedups over Hermes-style summarization.' source: S1 evidence: "compaction speedups over Hermes-style summarization"
- id: C2 claim: 'A user study is reported as an 8.68 times median speedup.' source: S1 evidence: "SpecCompact reduces median compaction latency from 63.74 to 7.28 seconds"
- id: C3 claim: 'The method drafts structured slots in parallel with a small drafter.' source: S1 evidence: "SpecCompact fine-tunes a small drafter and generates slots in parallel"
Verification
- Code available: https://github.com/shawnyin128/Speculative-Compaction is linked from an author affiliation on the HTML page (HTTP 200 on 2026-11-11). The paper body has no separate code-availability sentence. Not run.
- Reproduced: not reproduced (2026-11-11). Code not run.
- Related work: Parallel Context Compaction in this batch also changes compaction structure, but for volume control rather than latency.
- Limits: Speedups vary by model (larger on Gemma4 than on Qwen3.5) and by workload. Downstream accuracy trade-offs are described as model- and task-dependent. Latency tests used extra drafter GPU capacity.
- License: CC BY 4.0 on the abs page (2026-11-11; icon by-4.0.png; http://creativecommons.org/licenses/by/4.0/). Submitted 4 Nov 2026 (v1). Sources: [S1] https://arxiv.org/abs/2611.06876, v1 (4 Nov 2026). checked_on: 2026-11-11.
When to apply
Use when asking whether compaction itself is too slow, and whether speculative, slot-parallel summaries can cut that latency.
Facts
- id: F1 claim: 'SpecCompact fine-tunes a small drafter and generates structured summary slots in parallel, with target-model verification.' confidence: medium date: 2026-11-11 sources: [S1] license: CC BY 4.0
- id: F2 claim: 'The authors report 2.04-5.63 times compaction speedups over Hermes-style summarization.' confidence: medium date: 2026-11-11 sources: [S1] license: CC BY 4.0
- id: F3 claim: 'A Hermes user study is reported as an 8.68 times median compaction speedup, from 63.74 seconds to 7.28 seconds.' confidence: medium date: 2026-11-11 sources: [S1] license: CC BY 4.0
Sources
- [S1] arXiv 2611.06876, https://arxiv.org/abs/2611.06876. License: CC BY 4.0. v1 (4 Nov 2026). Accessed 2026-11-11.
License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted). Paper is CC BY 4.0.
License: CC-BY-4.0 · Summary by iinventedlight (AI-assisted). Source: Yulu An, Yifei Feng, Xiangyang Yin, SpecCompact: Accelerating LLM Agent Context Compaction with Slot-Parallel Semantic Speculative Verification (CC BY 4.0).
Cite this
'iinventedlight (AI-assisted)'. 2026. "SpecCompact: Accelerating LLM Agent Context Compaction with Slot-Parallel Semantic Speculative Verification." iinventedlight Library, iil:note:232p6my2, version of 2026-11-11. https://iinventedlight.com/library/notes/2611-06876-speccompact/. License: CC-BY-4.0.
@misc{iil_note_232p6my2,
author = {{'iinventedlight (AI-assisted)'}},
title = {{SpecCompact: Accelerating LLM Agent Context Compaction with Slot-Parallel Semantic Speculative Verification}},
howpublished = {iinventedlight Library},
year = {2026},
date = {2026-11-11},
url = {https://iinventedlight.com/library/notes/2611-06876-speccompact/},
note = {Permanent ID iil:note:232p6my2. License: CC-BY-4.0.},
}Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.
For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/2611-06876-speccompact.md.