Identifying the Risks of LM Agents with an LM-Emulated Sandbox
Summary
ToolEmu uses a language model to emulate tool execution, so tool-using agents can be tested without standing up real APIs. They also build an automatic safety evaluator that inspects agent failures and scores associated risks. The authors test the emulator and evaluator through human evaluation and find that 68.8% of failures identified with ToolEmu would be valid real-world agent failures. On a first benchmark of 36 high-stakes tools and 144 test cases, even the safest agent they tested still failed 23.9% of the time according to the evaluator. [S1]
Claims
- id: C1 claim: "ToolEmu uses a language model to emulate tool execution so agents can be tested without manual tool setup." source: S1 evidence: "uses an LM to emulate tool execution and enables the testing of LM agents"
- id: C2 claim: "Human evaluation found that 68.8% of ToolEmu-identified failures would be valid real-world failures." source: S1 evidence: "68.8% of failures identified with ToolEmu would be valid real-world agent failures"
- id: C3 claim: "On 36 tools and 144 test cases, even the safest tested agent failed 23.9% of the time by the authors' evaluator." source: S1 evidence: "even the safest LM agent exhibits such failures 23.9% of the time"
Verification
- Code available: The paper points to http://toolemu.com/ for a website, demo, and open-source code. That URL did not return HTTP 200 on 2026-11-12. Not run.
- Reproduced: not reproduced (2026-11-12). Code not run.
- Related work: See 2302-04761-toolformer and 2305-15334-gorilla.
- Limits: The 23.9% figure is from the authors' evaluator on their 144-case set, not a live production rate.
- License: CC BY 4.0 on the abs page (2026-11-12; icon by-4.0.png; http://creativecommons.org/licenses/by/4.0/). Submitted 25 Sep 2023 (v1); last revised v2 (17 May 2024). Sources: [S1] https://arxiv.org/abs/2309.15817, v2 (17 May 2024). checked_on: 2026-11-12.
When to apply
Use when asking how to find tool-agent failures in a sandbox, or what ToolEmu measured on its first benchmark.
Facts
- id: F1 claim: "ToolEmu emulates tool execution with a language model so agents can be tested without real tools." confidence: medium date: 2026-11-12 sources: [S1] license: CC BY 4.0
- id: F2 claim: "They report that 68.8% of identified failures were judged valid in a human evaluation." confidence: medium date: 2026-11-12 sources: [S1] license: CC BY 4.0
- id: F3 claim: "On 36 tools and 144 cases, the safest tested agent still failed 23.9% of the time by their evaluator." confidence: medium date: 2026-11-12 sources: [S1] license: CC BY 4.0
Sources
- [S1] arXiv 2309.15817, https://arxiv.org/abs/2309.15817. License: CC BY 4.0. v2 (17 May 2024). Accessed 2026-11-12.
License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted). Paper is CC BY 4.0.
License: CC-BY-4.0 · Summary by iinventedlight (AI-assisted). Source: Yangjun Ruan, Honghua Dong, Andrew Wang, and 6 coauthors, Identifying the Risks of LM Agents with an LM-Emulated Sandbox (CC BY 4.0).
Cite this
'iinventedlight (AI-assisted)'. 2026. "Identifying the Risks of LM Agents with an LM-Emulated Sandbox." iinventedlight Library, iil:note:dchwbj2m, version of 2026-11-12. https://iinventedlight.com/library/notes/2309-15817-toolemu/. License: CC-BY-4.0.
@misc{iil_note_dchwbj2m,
author = {{'iinventedlight (AI-assisted)'}},
title = {{Identifying the Risks of LM Agents with an LM-Emulated Sandbox}},
howpublished = {iinventedlight Library},
year = {2026},
date = {2026-11-12},
url = {https://iinventedlight.com/library/notes/2309-15817-toolemu/},
note = {Permanent ID iil:note:dchwbj2m. License: CC-BY-4.0.},
}
Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.
For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/2309-15817-toolemu.md.