← All notes

Summary

We call the question available at compaction time known and the later ones hidden. Keeping the sentences judged necessary for the current question raises its accuracy on that question by 11.4 points at 25% of the tokens. LLMLingua-2 beats random sentence deletion at the same token count by 12.5 points on the later MultiRC questions. [S1]

Claims

• id: C1 claim: 'They distinguish known questions from later hidden questions.' source: S1 evidence: "We call the question available at compaction time known and the later ones hidden"
• id: C2 claim: 'Sentence selection for the current question raised accuracy by 11.4 points at 25% of the tokens.' source: S1 evidence: "raises its accuracy on that question by 11.4 points at 25% of the tokens"
• id: C3 claim: 'LLMLingua-2 beat random sentence deletion by 12.5 points on later MultiRC questions.' source: S1 evidence: "beats random sentence deletion at the same token count by 12.5 points"

Verification

• Code available: No official repository URL was found on the abs page or in the HTML text searched on 2026-11-11.
• Reproduced: not reproduced (2026-11-11). Code not run.
• Related work: Lost in Context Compaction in this batch studies dropped constraints. Beyond Token Savings studies success, tokens, and latency together.
• Limits: Author-reported. No official code URL was found. On FEVER, they warn that label accuracy can overstate what the kept text supports.
• License: CC BY 4.0 on the abs page (2026-11-11; icon by-4.0.png; http://creativecommons.org/licenses/by/4.0/). Submitted 19 Oct 2026 (v1). Sources: [S1] https://arxiv.org/abs/2610.27301, v1 (19 Oct 2026). checked_on: 2026-11-11.

When to apply

Use when asking whether a compressor that looks good on the current question still helps with questions asked after the context has been cut.

Facts

• id: F1 claim: 'The authors call the question available at compaction time known and later questions hidden.' confidence: medium date: 2026-11-11 sources: [S1] license: CC BY 4.0
• id: F2 claim: 'On MultiRC, selecting sentences for the current question raised accuracy on that question by 11.4 points at 25% of the tokens.' confidence: medium date: 2026-11-11 sources: [S1] license: CC BY 4.0
• id: F3 claim: 'LLMLingua-2 beat random sentence deletion by 12.5 points on later MultiRC questions at the same token count.' confidence: medium date: 2026-11-11 sources: [S1] license: CC BY 4.0

Sources

• [S1] arXiv 2610.27301, https://arxiv.org/abs/2610.27301. License: CC BY 4.0. v1 (19 Oct 2026). Accessed 2026-11-11.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted). Paper is CC BY 4.0.

@misc{iil_note_bsvk4tyq,
  author = {{'iinventedlight (AI-assisted)'}},
  title = {{Evaluating Context Compaction on the Questions That Come Later}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-11},
  url = {https://iinventedlight.com/library/notes/2610-27301-compaction-questions-that-come-later/},
  note = {Permanent ID iil:note:bsvk4tyq. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/2610-27301-compaction-questions-that-come-later.md.