← All notes

Summary

ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs (2023), by Yujia Qin, Shihao Liang, Yining Ye, and coauthors, is at https://arxiv.org/abs/2307.16789. The authors present a general framework covering data construction, training, and evaluation. ToolBench, their instruction-tuning dataset, is built automatically with ChatGPT in three stages: collecting 16,464 real-world RESTful APIs spanning 49 categories from RapidAPI Hub, generating single-tool and multi-tool instructions, and annotating solution paths. They also describe a depth-first search-based decision tree for exploring reasoning traces, and ToolEval, an automatic evaluator. They fine-tune LLaMA to obtain ToolLLaMA, and equip it with a neural API retriever. The authors report that this model can carry out complex instructions, generalize to unseen APIs, perform comparably to ChatGPT, and show strong zero-shot generalization on APIBench. Experiments show that ToolLLaMA demonstrates an ability to execute complex instructions and generalize to unseen APIs, and exhibits comparable performance to ChatGPT. Title, authors, abstract, and comments were taken from the arXiv abs page. The version is v2 (3 Oct 2023), submitted 31 Jul 2023.

Claims

Pointer notes do not quote the paper. See Verification and Sources for the link and license.

Verification

• Format: pointer note only. The paper is not reproduced here.
• Why it matters: It matters because it scaled tool-use data and evaluation to many real APIs and became a standard resource for tool-calling work.
• Reproduced: not reproduced (2026-11-11).
• Related work: Related notes already in the library include Gorilla and CodeAct.
• Limits: This is a pointer note only. The API counts and ChatGPT comparisons are the authors' reported results from the v2 paper (3 Oct 2023).
• License: arXiv default (non-exclusive distribution license) on the abs page (2026-11-11; http://arxiv.org/licenses/nonexclusive-distrib/1.0/). Submitted 31 Jul 2023 (v1); last revised 3 Oct 2023 (v2). Sources: [S1] https://arxiv.org/abs/2307.16789, v2 (3 Oct 2023). checked_on: 2026-11-11. Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it. The note itself is CC BY 4.0.

When to apply

Use when you need the original ToolLLM and ToolBench paper on scaling tool use to many real APIs.

Facts

• id: F1 claim: 'ToolBench is described as an instruction-tuning dataset constructed automatically using ChatGPT, including 16,464 real-world RESTful APIs spanning 49 categories from RapidAPI Hub.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0
• id: F2 claim: 'The authors develop ToolEval, an automatic evaluator of tool use, and a depth-first search-based decision tree algorithm for reasoning traces.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0
• id: F3 claim: 'They fine-tune LLaMA to obtain ToolLLaMA, equipped with a neural API retriever, and report comparable performance to ChatGPT plus strong zero-shot generalization on APIBench.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0

Sources

• [S1] arXiv 2307.16789, https://arxiv.org/abs/2307.16789. Paper license: arXiv default (non-exclusive distribution license). v2 (3 Oct 2023). Accessed 2026-11-11. This note links to the paper and does not reproduce it.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Pointer by iinventedlight (AI-assisted). Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it.

@misc{iil_note_k1tgaj5r,
  author = {{'iinventedlight (AI-assisted)'}},
  title = {{ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-11},
  url = {https://iinventedlight.com/library/notes/2307-16789-toollm/},
  note = {Permanent ID iil:note:k1tgaj5r. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/2307-16789-toollm.md.