← All notes

Answer: Data leakage occurs when information that would not be available at prediction time is used when building the model. (S1)

Recommended practice

• Always split the data into train and test subsets first, particularly before any preprocessing steps. (S1)
• Never include test data when using the fit and fit_transform methods. (S1)
• Conversely, the transform method should be used on both train and test subsets as the same preprocessing should be applied to all the data. (S1)
• Pipelines also help avoiding another common pitfall: leaking the test data into the training data. (S1)

Where it fits in evaluation

• In scikit-learn a random split into training and test sets can be quickly computed with the train_test_split helper function. (S2)
• The simplest way to use cross-validation is to call the cross_val_score helper function on the estimator and the dataset. (S2)

Claims

• id: C1 claim: Leakage uses information unavailable at prediction time. source: S1 evidence: "Data leakage occurs when information that would not be available at prediction"
• id: C2 claim: Split before preprocessing. source: S1 evidence: "Always split the data into train and test subsets first"
• id: C3 claim: Never fit on test data. source: S1 evidence: "Never include test data when using the fit and fit_transform"
• id: C4 claim: train_test_split makes a random split. source: S2 evidence: "In scikit-learn a random split into training and test sets"

Sources

• [S1] 12. Common pitfalls and recommended practices — scikit-learn 1.9.1 documentation. scikit-learn docs. https://scikit-learn.org/stable/common_pitfalls.html . License: BSD-3-Clause (scikit-learn/scikit-learn COPYING; Copyright (c) 2007-2026 The scikit-learn developers; https://github.com/scikit-learn/scikit-learn/blob/main/COPYING). Accessed 2026-11-04.
• [S2] 3.1. Cross-validation: evaluating estimator performance — scikit-learn 1.9.1 documentation. scikit-learn docs. https://scikit-learn.org/stable/modules/cross_validation.html . License: BSD-3-Clause (scikit-learn/scikit-learn COPYING; Copyright (c) 2007-2026 The scikit-learn developers; https://github.com/scikit-learn/scikit-learn/blob/main/COPYING). Accessed 2026-11-04.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted).

@misc{iil_note_9f0dyh5v,
  author = {{iinventedlight}},
  title = {{Data leakage and how to avoid it in scikit-learn}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-04},
  url = {https://iinventedlight.com/library/notes/sklearn-data-leakage/},
  note = {Permanent ID iil:note:9f0dyh5v. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt. This page in Markdown: /library/notes/sklearn-data-leakage.md.