---
source: https://iinventedlight.com/library/notes/hf-trainer.md
title: "Fine-tuning with the Transformers Trainer"
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
source_license: "Apache-2.0 (huggingface docs repos); prose restates cited source lines"
attribution: "Summary by iinventedlight (AI-assisted). Sources: Hugging Face docs, Trainer · Hugging Face, Apache-2.0 (huggingface docs repos); Hugging Face docs, Fine-tuning · Hugging Face, Apache-2.0 (huggingface docs repos)."
author: "iinventedlight (synthesis of cited sources); text drafted with AI assistance"
date: 2026-11-04
status: published
checked_on: 2026-11-04
as_of: 2026-11-04
confidence: medium
untested: "No code was run for this note; only the fetched documentation pages were read."
category: coding-references
topic_area: "code-ai-eng: Hugging Face"
tags: ['hugging-face', 'transformers', 'training', 'fine-tuning']
sources:
  - id: S1
    title: "Trainer · Hugging Face"
    publisher: Hugging Face docs
    url: https://huggingface.co/docs/transformers/main_classes/trainer
    accessed: 2026-11-04
    license: "Apache-2.0 (huggingface docs repos)"
  - id: S2
    title: "Fine-tuning · Hugging Face"
    publisher: Hugging Face docs
    url: https://huggingface.co/docs/transformers/training
    accessed: 2026-11-04
    license: "Apache-2.0 (huggingface docs repos)"
---
# Fine-tuning with the Transformers Trainer

**Answer:** `Trainer` is Transformers' built-in training and evaluation loop for PyTorch.

```python
from transformers import AutoModelForCausalLM, TrainingArguments, Trainer

model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B")
args = TrainingArguments(
    output_dir="my-model",
    num_train_epochs=1,
    per_device_train_batch_size=8,
    learning_rate=2e-5,
    push_to_hub=True,
)
trainer = Trainer(model=model, args=args,
                  train_dataset=train_ds, eval_dataset=eval_ds,
                  processing_class=tokenizer)
trainer.train()
trainer.push_to_hub()
```

## Notes

- If `output_dir` is not set, it defaults to `tmp_trainer`.
- Pass `model_init` instead of `model` for hyperparameter search.

## Claims
- id: C1
  claim: Trainer is a simple but feature-complete training and eval loop for PyTorch.
  source: S1
  evidence: "Trainer is a simple but feature-complete training and eval loop for PyTorch"
- id: C2
  claim: If no model is provided, model_init must be passed.
  source: S1
  evidence: "If not provided, a model_init must be passed."
- id: C3
  claim: output_dir defaults to tmp_trainer.
  source: S1
  evidence: "output_dir set to a directory named tmp_trainer in the current directory if not provided."
- id: C4
  claim: Create a Trainer and call train() to begin.
  source: S2
  evidence: "Create a Trainer instance with all the necessary components, then call train() to begin."
- id: C5
  claim: push_to_hub uploads weights, generation config, tokenizer and config.
  source: S2
  evidence: "push_to_hub() uploads the fine-tuned weights, generation config, tokenizer, and model config"

## Sources
- [S1] Trainer · Hugging Face. Hugging Face docs. https://huggingface.co/docs/transformers/main_classes/trainer . License: Apache-2.0 (huggingface docs repos). Accessed 2026-11-04.
- [S2] Fine-tuning · Hugging Face. Hugging Face docs. https://huggingface.co/docs/transformers/training . License: Apache-2.0 (huggingface docs repos). Accessed 2026-11-04.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted).