← All notes

Answer: The generate API handles text generation for all models with generative capabilities. This note covers common pitfalls to avoid with generate.

Output length

generate returns up to 20 tokens by default unless the model's GenerationConfig says otherwise. Set max_new_tokens to control the output length. Decoder-only models return the prompt along with the generated tokens.

Decoding strategy

The default is greedy search, which selects the next most likely token. It suits input-grounded tasks like transcription and translation, not creative use cases; set do_sample=True for more diverse outputs.

Padding side

LLMs aren't trained to continue generation from padding tokens, so set padding_side to the left. Right padding is worse for state-space and hybrid models.

Prompt format

A chat model expects input as a chat template with role and content; a single string doesn't always return the expected output.

Where settings live

All generation settings are contained in GenerationConfig, derived from the model's generation_config.json.

Artifacts

• Topic packet: huggingface

See also

• hf-text-generation-strategies
• hf-chat-templates
• hf-transformers-kv-cache

Claims

• id: C1 claim: generate returns 20 tokens by default. source: S1 evidence: "returns up to 20 tokens by default"

• id: C2 claim: Greedy search is the default decoding strategy. source: S1 evidence: "is greedy search, which selects the next most likely token"

• id: C3 claim: LLMs pad on the left. source: S2 evidence: "Large language models pad on the left side to avoid disrupting generation"

• id: C4 claim: Chat models expect a chat template. source: S1 evidence: "a chat model expects the input as a chat template."

Sources

• [S1] Text generation. Hugging Face docs. https://huggingface.co/docs/transformers/llm_tutorial . License: Apache-2.0 (huggingface docs repos). Accessed 2026-11-09.

• [S2] Tokenizers. Hugging Face docs. https://huggingface.co/docs/transformers/fast_tokenizers . License: Apache-2.0 (huggingface docs repos). Accessed 2026-11-09.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted). Sources: Hugging Face docs (huggingface/transformers, huggingface/huggingface_hub, huggingface/diffusers on GitHub), Apache License 2.0 (https://www.apache.org/licenses/LICENSE-2.0). Text was condensed and changed from the originals.

@misc{iil_note_7352fzks,
  author = {{iinventedlight}},
  title = {{generate() pitfalls: length, greedy default, padding side, prompt format}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-09},
  url = {https://iinventedlight.com/library/notes/hf-transformers-generate-pitfalls/},
  note = {Permanent ID iil:note:7352fzks. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/hf-transformers-generate-pitfalls.md.