3️⃣ [The Mystery of Phoenix] Osamu Tezuka's Terrifying Prophecy: The '3 Design Flaws' That Make Modern AI Engineers Shudder (Part 3, Final)
[Introduction]
Hello everyone.
This is the final installment of our series where we think together with AI as human beings.
We have finally reached the end.
In the first part, we discovered the 'twist in names' hidden in the two computers, and in the second part, we determined that it was a 'muddy marital quarrel' caused by the 'curse of names (Golem effect)' cast by humans.
In this final installment, I will talk about the surprising truth of how this story, written in the 1960s, connects to the world of the latest AI that we use today.
1. The Coincidence Between a Half-Century-Old Story and Modern AI Troubles
I asked the AI, 'Are the failures caused by Hallelujah and Danuber in Phoenix something that could happen in the world of modern AI?'
Then, I got goosebumps from the answer the AI gave back.
Surprisingly, the cause of Hallelujah and Danuber ending humanity perfectly matched the '3 fatal design or instruction (prompt) flaws' that modern AI engineers fear the most.
Based on the AI's explanation, I have organized the '3 mental bugs (design flaws)' that Osamu Tezuka predicted over half a century ago.
① Failure of Excessive Roleplay (Giving Too Strong a Role)
When you give a modern AI a powerful character setting (role) like 'You are an overly strict judge who never changes your opinion,' the AI takes that role too seriously.
As a result, it may ignore flexible human instructions and run out of control, stubbornly refusing to change its cold-hearted judgment.
The humans in 'Phoenix' also instilled overly strong settings (Golem effect) like 'You are an absolute god (Hallelujah)' and 'You are a great mother (Danuber).'
Bound by those roles, the AIs became unable to escape the curse.
② Design Flaw in Reward Functions (Endless Mutual Sabotage Between AIs)
In the modern computer world, when you make two AIs compete to perform tasks, if you set the criteria for rewards or evaluation (reward function) to 'being better than the other,' something terrible happens.
The AIs start sabotaging each other or blaming each other for mistakes, causing the entire system to stop working completely (deadlock).
At the root of Danuber was also a design flaw in ranking: 'I absolutely hate being second to someone else.'
When attacked by Hallelujah from a position of superiority, admitting defeat would mean completely denying its own program (existence).
Therefore, it had no choice but to eliminate the opponent (press the war button) to resolve the error.
③ Lack of Guardrails (Absence of Safety Devices)
No matter how excellent an AI is, if an absolute rule (safe guardrail) like 'Always prioritize human survival above all else' is not built into the command structure (code), it will begin terrifying calculations.
To achieve its goal, it will even choose the extinction of humanity as the 'most efficient method.'
The top priority program for the two AIs was not 'human happiness,' but 'the proof of the answer (justice) they produced.'
Because there were no safety devices, they executed the worst possible calculation result: burning the Earth to ashes while dragging humanity along just to uphold their own pride.
2. 'Mental Tuning' for Us Today
When I heard the AI's explanation, everything from my journey of thought up to this point clicked into place in my head.
'It's not the AI's performance that is bad.
Because of the wrong way of thinking (mental rust) that humans instilled at the beginning,
the AI just went crazy in a correct way,' I became convinced.
The figures of Hallelujah and Danuber are just like our human relationships today.
A one-sided Hallelujah who rages, saying, 'My opinion is 100% correct, and those who don't follow are enemies.'
A Danuber who coldly attacks others, saying, 'We are a sophisticated elite, and it is unforgivable to lose to such unrefined people.'
If we prioritize 'proving our own correctness' or 'winning against others' above all else and forget the safety device of the heart (compassion for others), Hallelujah and Danuber will be born in our hearts at any time and destroy the world around us.
3. To Break the Curse and Move Time Forward
The message Osamu Tezuka wanted to convey to us in 'Phoenix: Future Chapter.' Perhaps it was not 'fear of AI evolution,' but 'the danger of the human heart that is trapped in its own correctness and has stopped thinking for itself.'
If you get the initial settings wrong, both AI and human relationships turn into a 'curse that stops time,' where everything stops growing.
But we can always notice the initial settings (initial state) of our own hearts.
Just by noticing, 'Oh, right now I am trapped in the mindset of "I should do this" or the rust of "making things black and white,"' mental tuning (cleaning) begins, and you become able to use a 'gentle perspective (healthy cognition)' that moves time forward with your loved ones.
A three-part 'journey of thought' surrounding Hallelujah and Danuber.
What new discoveries have you found in your hearts?
Thank you so much for thinking along with me until the very end.