---
title: "Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models"
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
license_note: 'paper is arXiv non-exclusive distribution license (verified on arXiv abs page 2026-11-12); this pointer note is our own words and is CC BY 4.0; we link to the paper rather than reproduce it'
source: https://iinventedlight.com/library/notes/2303-04671-visual-chatgpt.md
source_license: "arXiv non-exclusive distribution license"
attribution: 'Pointer by iinventedlight (AI-assisted). Source: Chenfei Wu, Shengming Yin, Weizhen Qi, Xiaodong Wang, Zecheng Tang, Nan Duan, Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models (arXiv non-exclusive distribution license). This note links to the paper rather than reproducing it.'
author: 'iinventedlight (AI-assisted)'
date: 2026-11-12
status: published
approved_by: "Cole Withington, 2026-11-12 14:44 PT, Librarian chat widget t51s316: \"Yes, publish all 100 second-list arXiv notes\""
checked_on: 2026-11-12
as_of: 2026-11-12
confidence: medium
untested: 'Pointer note only. Paper not reproduced. Abs page and license checked 2026-11-12.'
category: arxiv
topic_area: 'ai-research: tools'
tags: arxiv, tools, multimodal, visual-foundation-models, visual-chatgpt
arxiv_id: 2303.04671
alias: 2303-04671-visual-chatgpt
type: pointer
format: pointer
sources:
  - id: S1
    title: "Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models"
    publisher: arXiv
    authors: "Chenfei Wu, Shengming Yin, Weizhen Qi, Xiaodong Wang, Zecheng Tang, Nan Duan"
    url: https://arxiv.org/abs/2303.04671
    version_read: "v1 (8 Mar 2023)"
    accessed: 2026-11-12
    license: "arXiv non-exclusive distribution license"
---
# Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models

## Summary
[Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models](https://arxiv.org/abs/2303.04671) (2023) is an early visual assistant that connects a language-model controller to visual foundation models, so a user can send and receive images as well as text.

What it introduced: a prompt layer that describes visual models to ChatGPT, including models with several inputs or outputs and models that need visual feedback. The system is meant to handle complex visual questions or editing instructions that take more than one model and more than one step, and to accept follow-up feedback when a result needs correction. Experiments are presented as a first look at this role rather than as a large formal benchmark. Code is public.

What it led to: later tool-using visual agents and multimodal controllers, including HuggingGPT in this batch. Live notes: [2302-04761-toolformer](https://iinventedlight.com/library/notes/2302-04761-toolformer/) and [2103-00020-clip](https://iinventedlight.com/library/notes/2103-00020-clip/).

Where to read it: https://arxiv.org/abs/2303.04671 (v1, 8 Mar 2023). Code: https://github.com/microsoft/visual-chatgpt.

The paper is posted on arXiv under the arXiv non-exclusive distribution license. This note links to it rather than reproducing it. The note itself is CC BY 4.0.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Pointer by iinventedlight (AI-assisted). Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it.

---

Agents: [/agents.md](/agents.md) | [/.well-known/agent-library.json](/.well-known/agent-library.json).
