The shocking fact that generative AI doesn't bother to verify facts when writing text

Well then.
Let's use AI! So, I'm starting an irregular series where we learn the absolute basics of AI that you're rarely taught, so you can use it wisely! (I'll make this paid content later, so read it while you can!)
(I'll make this paid content later, so read it while you can!)

This time, we're talking about the mechanism behind how 'generative AI' like ChatGPT and Claude, which we use so often, writes text.

That said, there are many types of AI.
What I'm about to explain doesn't apply to every AI in the world, but rather it's like, this is how generative AI like ChatGPT (Large Language Models: LLM) works kind of thing.

You see.
It's quite far from the image of 'artificial intelligence' we imagine, which is why many people misunderstand it.

Look.
When you hear 'AI' or 'artificial intelligence,' you think of cat-shaped robots or mother computers or cockpits of combat robots that help humans, don't you have an image of those omnipotent robots solving difficult problems with incredible intelligence?

What do you think is the biggest difference between those fictional artificial intelligences and generative AI like ChatGPT?

...

Actually, it's 'thinking'.

This part here.
It's very important, so please really pay close attention.

First of all, generative AI like ChatGPT or Claude doesn't think from scratch to generate answers.

It is simply choosing words that 'sound plausible' to follow the flow of the text.

First, as a premise, generative AI learns a massive amount of data before it is released into the world.

It reads, reads, reads, and reads an incredible amount of data, including many books written by humanity throughout history, as well as websites and blogs scattered across the internet, before it finally makes its debut.

That means based on the data it has learned, it generates plausible words by assuming 'if this is the flow, this answer is appropriate'.

That's right.
As long as the text flows convincingly, it doesn't matter whether the answer is factually correct or not.

This is, seriously, the tricky part.

Let me give you an example.
Try to get a feel for how AI thinks.

Suppose you ask an AI, "What's the weather like tomorrow?"

You might think, based on our mental image of AI, that it quickly checks the Japan Meteorological Agency or Weathernews and answers, "It will be rainy tomorrow."

In reality, AI doesn't check the weather forecast at all.
It doesn't care whether it's sunny or rainy tomorrow.

In the past, according to the data it learned, the pattern of answering "It's sunny" to the question "What's the weather like tomorrow?" was common, so it thinks, "I'll answer 'It's sunny'."

OK?
It's not checking tomorrow's weather; it's just answering "It's sunny" because that was common in past trends.

Of course, if that were the case, it would only ever answer "sunny" whenever you asked about the weather.

So, it adds variation to its answers, like deciding to say "It's rainy" or "It's cloudy" a certain percentage of the time.

However, while it will use weather-related words like "sunny" or "rainy," it won't answer with things unrelated to the weather, like "Tomorrow's weather is pancakes" or "Tomorrow's weather is 50% Kanpei Hazama, A-me-ma!"

This is because, in its past learning, it has learned the pattern of answering with weather-related words to the question "What's the weather like tomorrow?"

Let's look at another one.

Suppose you ask, "What is Oda Nobunaga's blood type?"
The AI might answer something like "Type A," but in reality, nobody knows Nobunaga's blood type (lol).

But the AI has learned the question pattern of "person's name + blood type," and based on that trend, it gives a "convincing-sounding" answer.

In other words, to the question of what someone's blood type is, it just arbitrarily outputs an answer related to blood types, like A or O. This is what you call "fabrication."

In this way, because it answers based on past trends of "this was a common pattern of answering," in an extreme case, it might even arbitrarily answer "curry rice" to the question "What did you have for dinner last night?" even though it didn't eat anything.

Of course, in reality, it's not that simple and it returns much smarter answers, but the fundamental thought process remains the same, so saying that "the AI thought of the answer itself" might not be correct.

That's right.
Generative AI isn't actually thinking for itself.

It is simply selecting and combining plausible patterns from the vast amount of data it has learned in the past.

This is the biggest difference between a fictional, wonderful artificial intelligence and generative AI.

If you were to cry out to that cat-type robot, "Doraemooon!", it would listen to your story, cry or get angry, think along with you, and offer new suggestions.

ChatGPT also listens to you, pretends to be sad or angry, summarizes your thoughts, and offers suggestions.

However, unlike Doraemon, who has a thought process to think for himself, GPT is merely constructing answers based on the "flow of conversation" where "it feels right to answer this way in this situation."

This is extremely important when using AI.

If you fall into the trap of thinking, "The AI is properly checking things for me," you might end up in trouble, so be careful.

However, recent AI is developing rapidly, and more types are becoming available that can search the internet for the latest information while generating answers.
If you ask it to "please look it up," it may try to search and answer properly.

It doesn't automatically search on its own whenever it wants, and it doesn't always give the correct answer, though.

Even so, it doesn't know everything in the world, so it is strictly limited to "what it can look up."

Some people reading this far might think, "So, is AI untrustworthy?"

It usually does a good job, doesn't it? So isn't it fine?

AI text is reconstructed from a massive amount of data learned in the past, so one way or another, it ends up producing something close to the correct answer.

However, fundamentally, it just outputs "plausible-sounding words," so unfortunately, you can't really rely on it for accuracy.

In particular, it is dangerous to leave things to AI in genres such as news, medicine, history, and specialized knowledge.
Human "fact-checking" is absolutely necessary.

Actually, though.
I personally believe that the true brilliance of AI lies not in the "accuracy of its answers," but in its ability to create text at an incredible speed.

AI can generate thousands of characters in just a few seconds, whereas a human might only be able to write a few lines in a minute.
And it does so while maintaining context and putting it all together naturally.

That is truly an amazing asset, isn't it!!

Now, let's get back to the topic and summarize a bit.

The point was that generative AI is just reconstructing plausible-sounding words based on past training data.

So, the takeaway was that you can't rely on its accuracy.

Well, I often have it write spreadsheet formulas or PHP code for me, but it almost never gets it right on the first try.

I repeat the process of saying "It doesn't work," "It's giving me an error," or "Please do it properly," over and over again until I finally reach the point where I can say "It's done!!"
I bring it to completion through trial and error.

Even for things like formulas where the answer should be fixed, it might still be difficult for AI to output them accurately.

I'll talk more about the mechanisms behind why AI lies and fabricates information in the next post, so stay tuned!!