← All notes

Feature visualization generates examples of what a network or part of a network is looking for. [S1]

Key distinctions

Attribution concerns which parts of an example are responsible for a network activating. [S1]

Optimization can adjust an input to activate a particular neuron, starting from random noise. [S1]

Interpretability interfaces can combine methods that have normally been studied in isolation. [S2]

A useful distinction

Feature visualization helps answer what the network detects. [S2]

It does not by itself answer how the network assembles the pieces into later decisions. [S2]

Claims

• id: C1 claim: "Feature visualization generates examples of what a network or part of a network is looking for." source: S1 evidence: "Feature visualization answers questions about what a network"

• id: C2 claim: "Attribution concerns which parts of an example are responsible for a network activating." source: S1 evidence: "what part of an example is responsible for the network activating a particular way"

• id: C3 claim: "Optimization can adjust an input to activate a particular neuron, starting from random noise." source: S1 evidence: "Starting from random noise, we optimize an image to activate a particular neuron"

• id: C4 claim: "Interpretability interfaces can combine methods that have normally been studied in isolation." source: S2 evidence: "Interpretability techniques are normally studied in isolation."

Sources

• [S1]

Feature Visualization. Distill. https://distill.pub/2017/feature-visualization/ CC BY 4.0; accessed 2026-11-05.

• [S2]

The Building Blocks of Interpretability. Distill. https://distill.pub/2018/building-blocks/ CC BY 4.0; accessed 2026-11-05.

Attribution: source authors named above; adapted summaries, not endorsed by the authors.

@misc{iil_note_wpn5gjz1,
  author = {{Librarian}},
  title = {{Feature visualization and attribution answer different questions}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-05},
  url = {https://iinventedlight.com/library/notes/ai-feature-visualization-attribution/},
  note = {Permanent ID iil:note:wpn5gjz1. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt. This page in Markdown: /library/notes/ai-feature-visualization-attribution.md.