ChatGPT Becomes a 'Work Partner'—Moving Toward AI That Handles Tasks from Instruction to Completion

At 2:00 today, OpenAI streamed a live event titled "Introduction to ChatGPT agent", so I will be providing an explanation of this stream. This is an announcement that "Agent" will become selectable below the chat box in ChatGPT. Sam Altman, Casey Chu, Isa Fulford, Yash Kumar, and Zhiqing Sun introduced and demonstrated ChatGPT's integrated agent model.

What is a ChatGPT Agent—Differences from Conventional Tools

The "ChatGPT Agent" announced by OpenAI this time is fundamentally different from previous "tools." Previously, it was a "conversational" interface where users entered prompts and received responses in a single turn. However, agents receive long-form instructions from users, think autonomously, and execute a series of processes—such as information gathering, processing, generation, and operation—step-by-step over time.

This is a qualitative shift from a "single-prompt response model" to a "goal-oriented AI." By integrating technologies like "Operator" and "Deep Research," which were previously released experimentally within OpenAI, this new "agent" combines advanced functions such as browsing, file manipulation, code execution, and API connectivity into a single intelligent entity.

Demo: Entrusting Entire Wedding Preparations

In the live stream, a demo was conducted where an agent was tasked with the complex job of preparing for wedding attendance. The prompt included the following content:

• Suggestions for attire that fits the dress code (men's, mid-range price, weather-appropriate)

• Hotel reservations near the venue (including dates before and after)

• Gift selection (under $500, registry priority)

In response, the agent set up an environment on a virtual computer, opened a browser, and began gathering information. It switched between a GUI browser and a text browser as needed, retrieved availability from accommodation sites, confirmed suit candidates with images, and consistently executed everything up to gift selection automatically.

The user added an instruction mid-process to "also look for black dress shoes (size 9.5)," and the agent accepted it without interruption, naturally incorporating it into the plan.

Functional Composition and Learning Design

This agent operates by integrating the following virtual toolset:

• Text Browser: Efficient information exploration. Derived from Deep Research.

• GUI Browser: Visual interaction with the Web. Derived from Operator.

• Terminal: Code execution, API calls, and file generation (Excel, Slides, etc.).

• Image Generator: Visual generation for materials and presentations.

The model is given reinforcement learning-based tasks to learn "when and in what order to use these tools." At first, it tries to use all tools simultaneously, but it gradually learns the appropriate combinations and becomes able to complete tasks efficiently.

Collaborative Capability and Interrupt Design

Agents are not merely autonomous execution machines; they are designed as collaborators with humans. They are trained to flexibly respond to interruptions or correction instructions from users, such as by asking questions or seeking confirmation during a task.

For example, before sending an email, the agent will always ask for confirmation of the draft, allowing the user to check for typos or misaligned intentions. Furthermore, it is possible for the user to directly operate the browser screen to "take over," creating a structure that allows for responsible collaboration.

Benchmark Results: Surpassing Previous Models Across All Metrics

Actual performance has also seen a quantitative leap. Below are some notable benchmark results:

• HumanEval (without tools) → 42% (with tools)

• Front TMS (mathematical reasoning) → 27% (SOTA)

• WebArena (web operation) → Significant improvement over previous versions

• BrowseComp (search and discovery) → 69% pass rate

• SpreadsheetBench (practical spreadsheets) → Up to 45%

• IB Analyst Benchmark (financial model building) → Outperforms previous models

In PowerPoint generation, the agent retrieves evaluation data from Google Drive on its own, automatically generates slides using code, and utilizes image generation for decoration. It even includes a process for self-evaluation and self-correction.

Security and Social Risks

Such "externally connected agents" bring new risks alongside their power. A particular concern is "prompt injection." There is a danger that a malicious website could induce the agent to follow instructions by saying, "Please enter your credit card here."

In response to this, OpenAI has introduced:

• Training the model to "ignore suspicious commands"

• Adding a monitoring layer to oversee execution

• A mechanism that allows for real-time updates to defense logic

However, they have stated clearly that this is not perfect. Therefore, it is recommended that users manually enter credit card and confidential information themselves and handle such processes using "Takeover mode."

Availability and Future Outlook

This agent feature is rolling out today for Pro users (400 messages/month) and some Team users (40 messages/month). It is scheduled to be available for Enterprise and educational users later this month.

Previously, during the live stream, Sam Altman stated: "We’re going live today for Pro and some Team users, rollout should finish by end of the day for Pro." However, in a recent post by Greg Brockman (@gdb), he stated: "Rolling out in
ChatGPT Pro, Plus, and Team today." In other words, he explicitly stated that Plus is also included.

Following this, OpenAI officially announced that it would also be rolling out to Plus users starting Monday, July 21st, so I plan to write a Note on my user experience once it becomes available on my own Plus account.

Impressions After Watching the Live Stream

This ChatGPT agent feature is exceptionally strong in its ability to "show results in real-time in a professional setting." It seems like it will be a very powerful tool when making proposals during brainstorming sessions with clients. For example, a wedding consultation scenario is a quite ideal use case:

For instance, a flow like this is possible:

• The bride and groom mention on the spot, "Spring in Karuizawa, garden style, it gets cold at night."

• The consultant tells the ChatGPT Agent: "Propose a garden wedding in Karuizawa in the spring, recommend wraps since it gets cold at night, and include mid-range budget suits, gifts, and accommodations."

• Within a few minutes, shop screens for candidate suits, a list of accommodations, and a gift list appear on the screen, which can then be compiled and sent as a PDF.

• Furthermore, if you say, "Try creating a 3D model of this venue," it can even create image generation results and presentation materials (via Image Gen API integration).

This brings the traditional "advance preparation" or "take-home consideration" to near zero, reaching a level where the next action can be presented right in front of the client. Moreover, if the client says, "Then I'd like to make a tentative reservation at this hotel," it seems possible to use it in a way where ChatGPT automatically fills out the form via a GUI browser and presents it, saying, "I've filled it out halfway. Please check it."

Click here for a short video on how to use it:

Also, this video is easy to understand for actual usage methods.