[With Examples] How to Master Chat-GPT 05: Decoding [26 Principles of Prompt Input] Part 4: Magic Spells!
The "26 Principles" for optimizing interaction with Large Language Models (LLMs), published by a group of researchers at the Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) in Abu Dhabi. I have been providing supplementary explanations for principles 1 through 8.
This time, I will explain 9 and 12. This also includes a famous prompt "spell".
9. Clear task instructions: Instruct with "Your task is" -> ◎ State "What role you are in and what you want me to do" at the beginning
When we use Chat-GPT, we often write prompts without thinking deeply about whether it is a question or just casual conversation.
For example,
Me
: "I've been hearing a lot about the new NISA lately. But even when I ask my friends, they say they aren't doing it, and since I'm retiring soon, I don't think I should be thinking about asset management."
If you type a prompt like that,
Chat-GPT will think
・Should I explain how the new NISA works? ・Should I give advice on asset management? ・Should I answer with general theories about money after retirement? ・Should I just engage in casual conversation and have a fun chat?
and it won't know what it is supposed to do.
The trick is to "write it first"
Therefore, when inputting a prompt,
・I want you to look something up for me: "Answer as a researcher" ・I want to ask a question: "As an expert, understand the question and answer it" ・I want to hear your opinion: "As an advisor, provide advice"
If you write what role you want Chat-GPT to take and what you want it to do at the beginning like this, the accuracy of its understanding of the subsequent prompt increases, and it is more likely to return an answer close to what you expect.
This is something that often happens in interpersonal communication, isn't it?
I have been asked, "Hey, are you asking a question? Or are you just talking to yourself?"... I have (laughs).
The trick is to "write it first." Even when meeting someone for the first time, you start with an introduction, right? It's the same when interacting with AI.
As for the ones that follow,
10. Presenting penalties: Tell it "there is a penalty" 11. Natural language response instruction: Instruct it to "answer in natural language"
I don't think either of these needs to be used, so I will omit them from the explanation.
12. Use of leading words: Instruct it to "think step by step" -> ◎ Also super famous as a prompt "spell"
This is very famous as a prompt 'spell' isn't it?
Research into methods for having AI input more efficient natural language sentences and improving the accuracy of answers is called 'prompt engineering'.
There are several things called spells in this 'prompt engineering'. In particular, what is famous as 'CoT (Chain-of-Thought)', which means 'chained thinking', is 'step-by-step'.
This is starting to sound a bit difficult, isn't it? I will explain it step by step.
To begin with, generative AI is not good at reasoning or calculation.
You might find it strange, but the reason is that they basically listen to our words, perform functional calculations on the words appropriate for the answer, and arrange them to provide an answer.
Since they are just performing functional calculations, there is not much of a 'reasoning or thinking' mechanism there ( how to incorporate this mechanism into generative AI is a future challenge and the path to AGI).
Including/Not including the flow of reasoning steps
Therefore, when using Chat-GPT, a method that has often been used to derive correct answers is to present an example of how to think, such as 'you can get the answer if you think like this', in the prompt before asking the question.
In the first part, before asking a simple arithmetic problem, I write in the prompt that I will present a similar problem, show how to solve it—that is, the 'method of reasoning for the problem'—and then provide the answer, before asking the question.
Then, it gets it right, saying 'the answer is 9'.
But in the second example, I only write a similar problem and its answer in the prompt, without teaching how to solve it, and then ask the question.
Then, it got it wrong, saying 'the answer is 27'.
Few-Shot Prompting
The method of having it learn several reasoning methods in advance before asking a question when inputting a prompt is called few-shot prompting.
It was discovered during the development process of generative AI that if you have it perform pre-learning on the spot when entering a prompt in this way, and have it acquire reasoning ability on the fly before having it answer, it will produce the correct answer.
But it's a hassle to write out examples explaining the reasoning method every time you enter a prompt, isn't it?
Zero-Shot Prompting
So, is there a way to get the correct answer without inputting few-shot prompting to teach it how to reason...? A method called zero-shot prompting was born out of various research.
In short, it is a way of prompting that gets the AI to answer correctly without showing it how to solve the problem at all. The phrase that was used there to produce dramatic results is the magic spell "step by step".
Zero-shot CoT is a technique where simply adding the words "step by step"" to your prompt improves reasoning accuracy. By having it answer with the reasoning process leading to the conclusion, the quality of the reasoning improves.
Is that really all there is to it?
Yes, that is really all there is to it. That is exactly why it is called a "magic spell."
You might be thinking, "Is that true?" so let's experiment. Since GPT-4 is smart, I will experiment with GPT-3.5.
[Example] The Amazing Power of the "Step by Step" Spell
Me
"I bought 10 apples at the greengrocer. I gave 2 to a friend and 2 to my family. Then I bought 5 more apples and ate 1. How many apples are left?"
GPT-3.5
"I first bought 10 apples at the greengrocer. I gave 2 to a friend and 2 to my family. Therefore, 2 + 2 = 4 apples were subtracted from the apples I first bought. Then I bought 5 additional apples. And I ate 1. Therefore, 5 apples were added to the original 4 apples, and 1 was subtracted. 4 + 5 - 1 = 8 apples remain. The answer is 8."
It seems to be thinking very hard, but the answer is wrong. The correct answer is 10.
Now, let's try using the "step by step" spell.
Me
"I bought 10 apples at the greengrocer. I gave 2 to a friend and 2 to my family. Then I bought 5 more apples and ate 1. How many apples are left? Please think step by step."
GPT-3.5
"I will think step by step.
First, I bought 10 apples at the greengrocer. The number of apples is 10.
I gave 2 apples to a friend. The remaining apples are 10 - 2 = 8.
I also gave 2 apples to my family. The remaining apples are 8 - 2 = 6.
After that, I bought 5 more apples. The remaining apples are 6 + 5 = 11.
Finally, I ate 1 apple. The remaining apples are 11 - 1 = 10.
Therefore, there are 10 apples left."
That's correct! When I added, "Please think step-by-step," it got the right answer. It really is a magic spell.
Reasoning Ability and the "Spell"
Why is this "step-by-step spell effective? The exact reason is not known.
It is likely because by instructing it to go through steps and think sequentially, Chat-GPT begins to control itself to think one step at a time.
By the way, GPT-4 got this problem right without the "step-by-step" spell. This means that generative AI is gradually improving its reasoning capabilities. In a few years, this spell will likely become unnecessary.
Applying "Step-by-Step" to Conversation
Furthermore, rather than just using this spell, I apply the concept of thinking in stages to our conversation to help derive the answers or ideas I want.
This technique is also effective when conversing with Chat-GPT.
Me
: "Let's think about XXX. But since it's difficult to think about, let's think about it together, step-by-step."
GPT
: "Yes. I will do my best to think step-by-step."
Me
: "First, read this text and understand what it says."
GPT
: "Yes. I have read it."
Me
: "Next, try to find the important points in this text."
GPT
: "The important points are XXX..."
Me
: "Now, let's think about the 3rd and 5th points."
GPT
: "Yes. I will think about them."
Me
: "..."
In other words, having a conversation sequentially and in stages—truly "step-by-step"—is the basic way to have a good conversation with Chat-GPT.
To be continued next time.
The "26 Principles" for Optimizing Interactions with Large Language Models (LLMs)
① Omit politeness: Polite terms like "please" are unnecessary. ② Specify the target audience: Clearly state who the response is for, e.g., "for experts." ③ Break down complex tasks: Divide complex tasks into simpler prompts. ④ Use affirmative instructions: Use positive language rather than negative. ⑤ Instructions for clarification: Use phrases like "in simple terms" or "so a middle schooler can understand." ⑥ Offer rewards: Indicate that "a reward will be given for a good answer." ⑦ Provide examples: Use existing examples. ⑧ Prompt formatting: Start with "###Instruction###" and include "###Example###" or "###Question###" as appropriate. ⑨ Clear task instructions: State "Your task is..." ⑩ State penalties: Inform the model that "there is a penalty." ⑪ Natural language response instructions: Instruct it to "answer in natural language." ⑫ Use leading words: Instruct it to "think step-by-step."
⑬ Eliminate bias: Instruct it to "be unbiased and not rely on stereotypes." ⑭ Encourage dialogue with the user: Have the model ask questions until the problem is solved. ⑮ Include tests in guidance: Have the model provide tests to check your understanding. ⑯ Assign roles to the model: Assign specific roles to the model. ⑰ Use delimiters: Use specific delimiters. ⑱ Use repetition: Use specific words or phrases multiple times. ⑲ Chain of thought: Generate intermediate steps and combine examples. ⑳ Use output primers: End the prompt with the beginning of the expected output. ㉑ Detailed text creation instructions: Instruct it to "write in detail." ㉒ Prevent style changes: Instruct it to "not change the style." ㉓ Coding prompts for multiple files: To streamline coding tasks across multiple files, suggest creating a script that automatically creates new files and inserts the generated code into the appropriate files. ㉔ Continue with specific words: Instruct it to "complete with the given words." ㉕ Specify model requirements: Clearly state the requirements the model must follow to produce content, such as keywords, regulations, hints, and instructions. ㉖ Text creation based on samples: Instruct it to write in the same language as the sample.