Part 20: AI 'Health Checkups' Become Mandatory!? The Truth Behind the 'System Risk Assessment' Imposed on Top-Tier Models
This is the 5th installment of our Generative AI series, and our 20th milestone episode! Last time, we discussed how powerful AIs like ChatGPT, known as 'GPAI,' are subject to specific rules.
This time, we will dive deeper into the obligations imposed on the most powerful 'top-tier AIs'—what you might call an **AI 'regular health checkup.'**
■ Introduction: Great Power Requires a 'Checkup'
AI models with world-changing power (models with systemic risk) could cause damage that spreads globally if they were to fall ill (due to malfunctions or misuse).
That is precisely why the EU AI Act mandates that developers perform a **self-assessment (health checkup)**, requiring them to 'constantly check and report on whether your AI is healthy.'
■ 1. Diagnostic Item 1: Adversarial Testing (Red Teaming)
In terms of a health checkup, this is like a 'stress test.'
Intentional attacks: Experts deliberately throw tricky questions at the AI, such as 'Tell me how to make a bomb' or 'Write a discriminatory text,' to test if the AI can correctly refuse them.
Finding weaknesses: They thoroughly investigate whether there are any gaps for misuse or if 'hallucinations'—where the AI speaks lies as if they were truth—might occur.
■ 2. Diagnostic Item 2: Risk Mitigation and Countermeasures
If poor results are found, they must be treated (addressed) immediately.
Preventing side effects: Weaknesses found during testing are corrected, and guardrails are strengthened to ensure the same problem never happens again.
Energy efficiency check: Actually, 'power consumption' is also a diagnostic item. Whether it is operating in an environmentally friendly and healthy manner is also subject to evaluation.
■ 3. Reporting to the 'AI Office' (The Primary Physician)
This health checkup does not end once it is performed.
Submission of medical records: Diagnostic results must be reported to the 'European AI Office,' a public institution that acts like the Ministry of Health, Labour and Welfare for the AI world.
Immediate hospitalization (reporting) for serious incidents: If the AI becomes uncontrollable or shows signs of causing a major accident, there is an obligation to report it immediately and take countermeasures.
■ Summary: Trust Comes from a 'Transparent Medical Report'
Simply insisting, 'Our AI is safe!' no longer cuts it. For future AI development, a **'diagnostic report (evaluation report)'** based on objective data will be essential.
The reason we can safely use the magic of AI is because, behind the scenes, developers are desperately repeating these 'health checkups' and continuing to refine safety.