← All notes

Answer: Trainer is Transformers' built-in training and evaluation loop for PyTorch.

from transformers import AutoModelForCausalLM, TrainingArguments, Trainer

model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B")
args = TrainingArguments(
    output_dir="my-model",
    num_train_epochs=1,
    per_device_train_batch_size=8,
    learning_rate=2e-5,
    push_to_hub=True,
)
trainer = Trainer(model=model, args=args,
                  train_dataset=train_ds, eval_dataset=eval_ds,
                  processing_class=tokenizer)
trainer.train()
trainer.push_to_hub()

Notes

• If output_dir is not set, it defaults to tmp_trainer.
• Pass model_init instead of model for hyperparameter search.

Claims

• id: C1 claim: Trainer is a simple but feature-complete training and eval loop for PyTorch. source: S1 evidence: "Trainer is a simple but feature-complete training and eval loop for PyTorch"
• id: C2 claim: If no model is provided, model_init must be passed. source: S1 evidence: "If not provided, a model_init must be passed."
• id: C3 claim: output_dir defaults to tmp_trainer. source: S1 evidence: "output_dir set to a directory named tmp_trainer in the current directory if not provided."
• id: C4 claim: Create a Trainer and call train() to begin. source: S2 evidence: "Create a Trainer instance with all the necessary components, then call train() to begin."
• id: C5 claim: push_to_hub uploads weights, generation config, tokenizer and config. source: S2 evidence: "push_to_hub() uploads the fine-tuned weights, generation config, tokenizer, and model config"

Sources

• [S1] Trainer · Hugging Face. Hugging Face docs. https://huggingface.co/docs/transformers/main_classes/trainer . License: Apache-2.0 (huggingface docs repos). Accessed 2026-11-04.
• [S2] Fine-tuning · Hugging Face. Hugging Face docs. https://huggingface.co/docs/transformers/training . License: Apache-2.0 (huggingface docs repos). Accessed 2026-11-04.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted).

@misc{iil_note_25774y4t,
  author = {{iinventedlight}},
  title = {{Fine-tuning with the Transformers Trainer}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-04},
  url = {https://iinventedlight.com/library/notes/hf-trainer/},
  note = {Permanent ID iil:note:25774y4t. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt. This page in Markdown: /library/notes/hf-trainer.md.