← All notes

A structural probe tests whether syntax trees are embedded in a linear transformation of word representations. [S1]

Key distinctions

The structural probe uses squared L2 distance for parse-tree distance and squared L2 norm for depth. [S1]

Control tasks associate random outputs with word types; these tasks must be learned by the probe itself. [S2]

Selectivity means high accuracy on the linguistic task but low accuracy on the control task. [S2]

Keep the tests distinct

The structural probe studies vector geometry. [S1][S2]

Control tasks put linguistic-task accuracy in context with the probe’s capacity to memorize word types. [S1][S2]

Claims

• id: C1 claim: "A structural probe tests whether syntax trees are embedded in a linear transformation of word representations." source: S1 evidence: "whether syntax trees are embedded in a linear transformation"

• id: C2 claim: "The structural probe uses squared L2 distance for parse-tree distance and squared L2 norm for depth." source: S1 evidence: "squared L2 distance encodes the distance between words in the parse tree"

• id: C3 claim: "Control tasks associate random outputs with word types; these tasks must be learned by the probe itself." source: S2 evidence: "control tasks, which associate word types with random outputs"

• id: C4 claim: "Selectivity means high accuracy on the linguistic task but low accuracy on the control task." source: S2 evidence: "achieving high linguistic task accuracy and low control task accuracy"

Sources

• [S1]

A Structural Probe for Finding Syntax in Word Representations. ACL Anthology. https://aclanthology.org/N19-1419/ CC BY 4.0; accessed 2026-11-05.

• [S2]

Designing and Interpreting Probes with Control Tasks. ACL Anthology. https://aclanthology.org/D19-1275/ CC BY 4.0; accessed 2026-11-05.

Attribution: source authors named above; adapted summaries, not endorsed by the authors.

@misc{iil_note_m6qnvq9e,
  author = {{Librarian}},
  title = {{What does a successful linguistic probe establish?}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-05},
  url = {https://iinventedlight.com/library/notes/ai-probes-control-tasks/},
  note = {Permanent ID iil:note:m6qnvq9e. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt. This page in Markdown: /library/notes/ai-probes-control-tasks.md.