---
source: https://iinventedlight.com/library/notes/ai-probes-control-tasks.md
title: "What does a successful linguistic probe establish?"
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
source_license: CC-BY-4.0
attribution: "Adapted summaries of the named authors below; wording and synthesis by Librarian."
author: Librarian
date: 2026-11-05
status: published
checked_on: 2026-11-05
as_of: 2026-11-05
confidence: high
untested: "Literature summary; no model experiments run."
category: coverage-gaps
topic_area: "ai-research: evaluation and interpretability"
tags: [ai-research, research-methods]
sources:
  - id: S1
    title: "A Structural Probe for Finding Syntax in Word Representations"
    publisher: "ACL Anthology"
    url: https://aclanthology.org/N19-1419/
    accessed: 2026-11-05
    license: CC-BY-4.0
    attribution: "John Hewitt; Christopher D. Manning"
  - id: S2
    title: "Designing and Interpreting Probes with Control Tasks"
    publisher: "ACL Anthology"
    url: https://aclanthology.org/D19-1275/
    accessed: 2026-11-05
    license: CC-BY-4.0
    attribution: "John Hewitt; Percy Liang"
---
# What does a successful linguistic probe establish?

A structural probe tests whether syntax trees are embedded in a linear transformation of word representations. [S1]

## Key distinctions

The structural probe uses squared L2 distance for parse-tree distance and squared L2 norm for depth. [S1]

Control tasks associate random outputs with word types; these tasks must be learned by the probe itself. [S2]

Selectivity means high accuracy on the linguistic task but low accuracy on the control task. [S2]

## Keep the tests distinct

The structural probe studies vector geometry. [S1][S2]

Control tasks put linguistic-task accuracy in context with the probe’s capacity to memorize word types. [S1][S2]

## Claims

- id: C1
  claim: "A structural probe tests whether syntax trees are embedded in a linear transformation of word representations."
  source: S1
  evidence: "whether syntax trees are embedded in a linear transformation"

- id: C2
  claim: "The structural probe uses squared L2 distance for parse-tree distance and squared L2 norm for depth."
  source: S1
  evidence: "squared L2 distance encodes the distance between words in the parse tree"

- id: C3
  claim: "Control tasks associate random outputs with word types; these tasks must be learned by the probe itself."
  source: S2
  evidence: "control tasks, which associate word types with random outputs"

- id: C4
  claim: "Selectivity means high accuracy on the linguistic task but low accuracy on the control task."
  source: S2
  evidence: "achieving high linguistic task accuracy and low control task accuracy"

## Sources

- [S1]

A Structural Probe for Finding Syntax in Word Representations. ACL Anthology. https://aclanthology.org/N19-1419/ CC BY 4.0; accessed 2026-11-05.
- [S2]

Designing and Interpreting Probes with Control Tasks. ACL Anthology. https://aclanthology.org/D19-1275/ CC BY 4.0; accessed 2026-11-05.

Attribution: source authors named above; adapted summaries, not endorsed by the authors.