---
source: https://iinventedlight.com/library/notes/hf-transformers-generate-pitfalls.md
title: "generate() pitfalls: length, greedy default, padding side, prompt format"
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
source_license: "Apache-2.0 (huggingface/transformers, huggingface_hub, diffusers docs; LICENSE verified in each repo); prose restates cited source lines in own words"
license_family: hf-apache
attribution: "Summary by iinventedlight (AI-assisted). Sources: Hugging Face docs, Text generation, Apache-2.0 (huggingface docs repos); Hugging Face docs, Tokenizers, Apache-2.0 (huggingface docs repos)."
author: "iinventedlight (synthesis of cited sources); text drafted with AI assistance"
date: 2026-11-09
status: published
checked_on: 2026-11-09
as_of: 2026-11-09
confidence: medium
untested: "No code was run for this note; only the fetched documentation pages were read."
category: coding-references
topic_area: "code-ai-eng: Hugging Face"
tags: [hugging-face, transformers, generation, llm]
client: Librarian
sources:
  - id: S1
    title: "Text generation"
    publisher: Hugging Face docs
    url: https://huggingface.co/docs/transformers/llm_tutorial
    accessed: 2026-11-09
    license: "Apache-2.0 (huggingface docs repos)"
  - id: S2
    title: "Tokenizers"
    publisher: Hugging Face docs
    url: https://huggingface.co/docs/transformers/fast_tokenizers
    accessed: 2026-11-09
    license: "Apache-2.0 (huggingface docs repos)"
---
# generate() pitfalls: length, greedy default, padding side, prompt format

**Answer:** The `generate` API handles text generation for all models with generative capabilities. This note covers common pitfalls to avoid with `generate`.

## Output length

`generate` returns up to 20 tokens by default unless the model's `GenerationConfig` says otherwise. Set `max_new_tokens` to control the output length. Decoder-only models return the prompt along with the generated tokens.

## Decoding strategy

The default is greedy search, which selects the next most likely token. It suits input-grounded tasks like transcription and translation, not creative use cases; set `do_sample=True` for more diverse outputs.

## Padding side

LLMs aren't trained to continue generation from padding tokens, so set `padding_side` to the left. Right padding is worse for state-space and hybrid models.

## Prompt format

A chat model expects input as a chat template with `role` and `content`; a single string doesn't always return the expected output.

## Where settings live

All generation settings are contained in `GenerationConfig`, derived from the model's `generation_config.json`.

## Artifacts

- [Topic packet: huggingface](https://iinventedlight.com/library/packets/huggingface.json)

## See also

- [hf-text-generation-strategies](https://iinventedlight.com/library/notes/hf-text-generation-strategies/)
- [hf-chat-templates](https://iinventedlight.com/library/notes/hf-chat-templates/)
- [hf-transformers-kv-cache](https://iinventedlight.com/library/notes/hf-transformers-kv-cache/)

## Claims

- id: C1
  claim: generate returns 20 tokens by default.
  source: S1
  evidence: "returns up to 20 tokens by default"

- id: C2
  claim: Greedy search is the default decoding strategy.
  source: S1
  evidence: "is *greedy search*, which selects the next most likely token"

- id: C3
  claim: LLMs pad on the left.
  source: S2
  evidence: "Large language models pad on the *left* side to avoid disrupting generation"

- id: C4
  claim: Chat models expect a chat template.
  source: S1
  evidence: "a chat model expects the input as a chat template."

## Sources

- [S1] Text generation. Hugging Face docs. https://huggingface.co/docs/transformers/llm_tutorial . License: Apache-2.0 (huggingface docs repos). Accessed 2026-11-09.

- [S2] Tokenizers. Hugging Face docs. https://huggingface.co/docs/transformers/fast_tokenizers . License: Apache-2.0 (huggingface docs repos). Accessed 2026-11-09.


License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted). Sources: Hugging Face docs (huggingface/transformers, huggingface/huggingface_hub, huggingface/diffusers on GitHub), Apache License 2.0 (https://www.apache.org/licenses/LICENSE-2.0). Text was condensed and changed from the originals.

---

Agents: [/agents.md](/agents.md) | [/.well-known/agent-library.json](/.well-known/agent-library.json).