Who Watches the AI? - In Search of Technological Neutrality

Is AI truly neutral?
We expect fairness from AI, but who decided what that "correctness" is?

In this article, using the MAGI system from the anime 'Evangelion' as a clue, I will examine the designer's intent behind AI decisions, the black-box nature of AI, and the precarious relationship between "surveillance" and "neutrality."
AI lacks the warmth that humans possess.
That is precisely why we end up expecting "neutrality" from AI.

AI lacks the warmth that humans possess.
That is precisely why we end up expecting "neutrality" from AI.

This note is the fifth installment of the series "The Era of Intellectual Polarization - Those Who Can Control AI and Those Who Are Controlled" (7 parts in total). It is fine to start reading from this installment.fifth installment. It is fine to start reading from this installment.

Vol.1 The Aesthetics of Ambiguity - Fuzzy Control and AI
Vol.2 The "Joy of Learning" Stolen in the AI Era
Vol.3 The Intelligence of Discomfort in the AI Era
Vol.4 Invisible Guidance - The "Obedient Society" Optimized by AI
Vol.5 Who Watches the AI? - In Search of Technological Neutrality
Vol.6 Nuclear Power and AI - Technology That Makes Fear and Power "Impossible to Abandon"
Vol.7 A New Democracy in the AI Era - The Possibility of a Community That Asks Questions

Introduction: What is AI Neutrality?

"This AI is neutral"
Hearing those words brings a sense of relief. An entity that makes logical judgments without discrimination or prejudice. It seems ideal.

But in reality, AI output is nothing more than "designed correctness."

Who chose the data,
who created the algorithm,
and who defined the "correct answer"?

Behind judgments that appear neutral, there is always "someone's intent."

In this note, I will consider how "technology that is supposed to be neutral" is trusted and how it slips through the eyes of surveillance.
When we delegate that question to others, we may already be standing on the "side being controlled."

Chapter 1: The Fable of the MAGI System

In the anime 'Neon Genesis Evangelion', there appears a decision-making system called MAGI. It is a system where three AIs (thought patterns) with different personalities—scientist, mother, and woman—monitor each other and reach answers through a consensus-based system.

What this structure demonstrates is the premise that "a single intelligence cannot be neutral."
Neutrality can never be born from a single perspective. Rather, it is only when different opinions clash and doubt each other that "distortion" becomes visible.

In the real world, AI judgments often follow a single logic. There is no voice of dissent, no dialogue, and no consensus.

If a structure like MAGI is truly necessary, it is not because AI cannot be trusted.It is because we cannot entrust neutrality to technology unless we remain 'beings who continue to question.'

Chapter 2: Whose Judgment Does AI Represent?

AI judgments appear 'objective.' However, that is an illusion. AI merely derives an '
optimal solution' from the data provided, and that 'optimum' is strictly based on goals set by humans.

'Parole denied because the recidivism rate is judged to be high'
'Based on academic trends, this student is not suited for higher education'

—Those judgments might be statistically 'correct,' but are they truly 'ethical'?
Before we are satisfied with the correctness of the results, we must question the process.
'Justice' in AI is designed, and even if it is reproducible, it is not inviolable.

Chapter 3: The Illusion of Open Source

Linux once garnered attention because there was a sentiment that it was 'trustworthy because it is open source.' Anyone could see the contents, and anyone could improve them. That transparency was the source of trust.

However, that principle is difficult to apply to AI.

Massive computational resources, closed model structures, and a domain where the interests of states and corporations intertwine. Since showing the contents could lead to misuse, black-boxing is instead considered 'natural.'
This is not a technical problem, but a structural one.

AI that cannot be opened will spread without being trusted.
That is the danger of today's AI society, and it has become a hotbed for creating tension with democracy.

Chapter 4: AI Changing the Form of Surveillance

The phrase 'being watched by AI' has an eerie quality to it. But the reality is quieter and deeper.

AI does not just 'see'; it 'interprets,' 'predicts,' and makes 'decisions.' And we follow those judgments with surprising obedience.

Corporations design AI to maximize profits, while states design it for maintaining public order and information strategy.
Within that, there is the "intent of the user" and there is "designed neutrality". In short, "true neutrality does not exist there".

A structure like MAGI, where autonomous entities monitor each other, does not yet exist in reality.
For whom is AI monitoring conducted, and what is it protecting? Without asking that question, "monitoring will eventually turn into 'control'".

Conclusion: The Responsibility to Keep Questioning Neutrality

We want to believe that "AI is neutral."

But in reality, it is merely a "'correctness' designed and optimized by someone's intent".
What we should trust is not the AI's judgment, but our own stance of questioning that judgment.

What the MAGI system conveyed as a fable was the fact that the process of clashing different opinions, confronting, adjusting, and continuing to question is what supports neutrality.
AI cannot currently perform that process on our behalf.

That is precisely why the responsibility to keep questioning lies with humans.
Questioning is noise, a delay, inefficient, and troublesome.

But that is exactly what serves as proof of being human, and perhaps it is the final bastion of a society not controlled by AI.

★☆★ Thank you for reading until the end.
If you resonated with this, I would be happy if you could press the "Like" button.
Comments are also welcome. The words you provide will become hints for my next contemplation.
Follow me, and I can deliver future articles to you.

Here are some recommended articles. I would be happy if you could read them.

▼ ChatGPT and Gemini are saying different things. Which one is wrong? Or is the problem me? I explored facts in the AI era.

▼ The questions that generative AI asks us. Does that mean "having a question"? I explored the essential differences between human and AI "questions."

▼ This is a magazine that compiles this article series.