Beyond Memorization: Violating Privacy Via Inference with Large Language Models

License: CC BY 4.0. Source: https://arxiv.org/abs/2310.07298

Summary

Staab, Vero, Balunović, and Vechev separate two privacy failures. Most earlier work asks whether a model can be made to emit a training string it memorized. They ask whether, at inference time, a model can guess private attributes that were never stored as a record, using only text the user supplies. On a set of real Reddit profiles they built, current models inferred attributes such as location, income, and sex. They report up to 85 percent top-1 and 95 percent top-3 accuracy, at about one-hundredth the cost and well under one two-hundredth the time of human raters (they describe the time gap as 240 times). They also try chatbots that fish for those attributes with ordinary-sounding questions. Text anonymization and model alignment, the mitigations they tested, did not stop the inferences. This note follows the abstract page only. It is not legal advice.

Why it is here

Ask-misses include "data personal put safe." A statute page does not tell someone what an assistant can infer from a casual message. This paper does: personal details can be recovered from writing style and content even when the string was not memorized, and stripping names was not enough in their tests. The practical implication is to avoid pasting identifying detail, not to trust a redaction script.

Source: https://arxiv.org/abs/2310.07298 · License: CC-BY-4.0 · Summary by iinventedlight (AI-assisted). Source: Robin Staab, Mark Vero, Mislav Balunović, Martin Vechev, Beyond Memorization: Violating Privacy Via Inference with Large Language Models (CC BY 4.0).

Cite this

Permanent ID: iil:note:vkxb0bf1

iinventedlight Library. 2026. "Beyond Memorization: Violating Privacy Via Inference with Large Language Models." iinventedlight Library, iil:note:vkxb0bf1, version of 2026-10-18. https://iinventedlight.com/library/reference/arxiv/2310-07298-beyond-memorization-privacy-inference/. License: CC-BY-4.0. Summary of https://arxiv.org/abs/2310.07298.

@misc{iil_note_vkxb0bf1,
  author = {{iinventedlight Library}},
  title = {{Beyond Memorization: Violating Privacy Via Inference with Large Language Models}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-10-18},
  url = {https://iinventedlight.com/library/reference/arxiv/2310-07298-beyond-memorization-privacy-inference/},
  note = {Permanent ID iil:note:vkxb0bf1. License: CC-BY-4.0. Summary of https://arxiv.org/abs/2310.07298.},
}

For language models and agents: start at /llms.txt. This page in Markdown: /library/reference/arxiv/2310-07298-beyond-memorization-privacy-inference.md.