[Stanford] Agentic AI

In modern AI research, language models are receiving significant attention as the foundation for natural language processing. In particular, methods that go beyond traditional text input/output to enable interaction with the environment as agentic AI are developing rapidly. This article explains the basic concepts of language models, training methods, best practices for prompt design, and how agentic AI can solve complex tasks through concrete examples and citations.

1. Basics of Language Models and Training Methods

1-1. Overview of Language Models

A language model is a machine learning model that predicts the next word to appear given an input text. For example, if the input sentence "The students are..." is provided, the probability of words such as "books" or "laptops" appearing is calculated, and the most likely words are generated one after another. Because the model is pre-trained on a large-scale text corpus (such as books or publicly available text on the internet), it is possible for it to possess vast world knowledge.

1-2. Pre-training and Post-training

Through pre-training, the model acquires basic language generation capabilities based on the task of "next word prediction." However, since it is difficult for a pre-trained model to generate the specific output a user desires as-is, post-training is necessary.

• Instruction Following
  To generate responses according to specified commands or questions, the model is trained with pairs of pre-prepared instructions and their expected outputs.

• Reinforcement Learning from Human Feedback (RLHF)
  By applying a reward scheme based on human preferences and accuracy, more refined responses can be obtained.

Through these adjustment processes, for example, conversational AI like ChatGPT has become capable of generating appropriate and natural responses to user input.

2. Evolution to Agentic AI

2-1. Definition of Agentic AI

Traditional language models simply predicted the next word for input text. However, agentic AI enables interaction with the environment. In other words, the following extensions are made:

• Tool Use
  The model can query external APIs or search engines; for example, if asked "What is the weather in San Francisco?", it retrieves actual weather information to generate a response.

• Integration of Thought and Action (ReAct)
  By incorporating a "think before acting" process into the model, step-by-step processing for complex tasks is realized. For example, in customer support, a flow can be envisioned where the model sequentially references three information sources—refund policy, customer information, and product information—to make a final decision.

2-2. Design Patterns for Agentic AI

There are several design patterns for agentic AI. Representative ones are as follows:

• Planning
  The model is made to break down tasks and plan the necessary actions for each step. For example, in code refactoring, a flow is created to first analyze the current code, identify areas for improvement, and then present improvement proposals.

• Reflection (Self-Reflection)
  The model reflects on its own output and generates feedback to achieve further improvements. An approach of "first think of your own answer, then compare it with the provided answer" is effective.

• Multi-Agent Collaboration
  By having multiple specialized agents (e.g., climate control agent, lighting control agent, etc.) work together, complex systems like smart homes can be managed efficiently.

3. Best Practices for Prompt Design and Evaluation

3-1. Key Points for Prompt Design

To maximize the performance of language models, clear and specific prompt design is essential. Below are some concrete examples.

• Detailed Instructions
  Instead of simply saying "please answer," write detailed instructions such as "analyze the following text and list three main points."

• Few-shot Learning
  Provide concrete examples (input and output pairs) so that the model can learn the required output format.

• Providing Context
  By providing relevant information or reference materials, you can help the model avoid incorrect information (so-called "hallucinations").

• Promoting Thought Processes
  It is recommended to use chain-of-thought to improve accuracy by giving instructions such as "first, break down the task and think about it."

3-2. Refinement of Evaluation Methods

For evaluating generated output, in addition to the conventional method of "using an LLM as a judge," agentic evaluation methods have been proposed. For example, by having one LLM perform an initial evaluation and then re-evaluating it from a different perspective (such as that of a senior engineer), more reliable evaluation becomes possible. Such iterative evaluation processes are extremely useful for application improvement.

4. Real-world Application Examples of Agentic AI

4-1. Software Development Support

Agentic AI is also being utilized as a software development support tool for tasks such as code generation and bug fixing. Specifically, the following flow can be considered.

1. Code Analysis and Feedback
   The model evaluates existing code and suggests improvements.

2. Automatic Code Generation or Refactoring
   Based on the feedback provided, the model generates refactored code as an improvement proposal.

3. Execution and Re-evaluation
   The generated code is executed in a secure sandbox environment, and the results are fed back to the model again.

4-2. Customer Support

In customer support, Agentic AI automates the process of making final decisions by coordinating tasks such as checking refund policies, retrieving customer information, and referencing product information according to the content of the inquiry. A mechanism where "the agent executes the necessary API calls at each step to support the final decision" has been introduced, contributing to improved operational efficiency.

Agentic AI goes beyond conventional simple text generation models and will be an important direction for future AI applications in that it enables interaction with the environment, tool use, and complex task decomposition.

• Technological Evolution and Rapid Model Improvement
  As can be seen from the latest research and practical examples, designs that can flexibly adapt to technology that evolves daily are required for Agentic AI.

• Ethical Considerations and Safety Measures
  At the same time, efforts to address ethical issues, such as preventing hallucinations and protecting data privacy, are also essential.

• Continuous Improvement through Experimentation and Feedback
  In the initial stages, it is recommended to build a simple prototype and continue to improve it by collecting feedback in practice.

The approach of 'starting simple and gradually scaling up to complex tasks' is an effective strategy for the implementation and application of agentic AI. Experts also recommend trial and error in playground environments, and obtaining information from the latest research and the community will be the key to future success.

Related Articles