← All notes

Answer:

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-v0.1", padding_side="left")
model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-v0.1", device_map="auto")

inputs = tok(["A list of colors: red, blue"], return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=50, do_sample=True, temperature=0.7)
print(tok.batch_decode(out, skip_special_tokens=True)[0])

Strategies

|---|---|---| | Greedy | default | short, factual output | | Sampling | do_sample=True | creative or chat text | | Beam search | num_beams=4 | translation, summarization |

Claims

• id: C1 claim: Greedy search is the default decoding strategy. source: S1 evidence: "Greedy search is the default decoding strategy."
• id: C2 claim: Unless specified, greedy generates a maximum of 20 new tokens. source: S1 evidence: "this strategy generates a maximum of 20 new tokens."
• id: C3 claim: Greedy search begins to repeat itself on longer sequences. source: S1 evidence: "it breaks down when generating longer sequences because it begins to repeat itself."
• id: C4 claim: Beam search is good on input-grounded tasks. source: S2 evidence: "Beam search is good on input-grounded tasks."
• id: C5 claim: Set repetition_penalty above 1.0 if the model repeats itself. source: S2 evidence: "repetition_penaltyfloatSet it to >1.0 if you’re seeing the model repeat itself often."

Sources

• [S1] Generation strategies · Hugging Face. Hugging Face docs. https://huggingface.co/docs/transformers/generation_strategies . License: Apache-2.0 (huggingface docs repos). Accessed 2026-11-04.
• [S2] Text generation · Hugging Face. Hugging Face docs. https://huggingface.co/docs/transformers/llm_tutorial . License: Apache-2.0 (huggingface docs repos). Accessed 2026-11-04.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted).

@misc{iil_note_hfm15g9m,
  author = {{iinventedlight}},
  title = {{Text generation strategies in Transformers}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-04},
  url = {https://iinventedlight.com/library/notes/hf-text-generation-strategies/},
  note = {Permanent ID iil:note:hfm15g9m. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt. This page in Markdown: /library/notes/hf-text-generation-strategies.md.