ISO 31000 / EDPS 'Guidance for Risk Management of AI systems' / Random thoughts on breaking down 'principles' into 'risks'

0 Introduction

The discussion surrounding AI systems, including generative AI, has moved past the stage of performance competition and now feels as though it has entered the phase of 'how it breaks' and 'how to accept responsibility.' From a practical standpoint, the speed at which governance documents are proliferating is faster than the speed at which models are becoming smarter. Furthermore, while many of these thick volumes of documents proclaim lofty principles, what the field truly craves is not slogans like 'uphold the principles,' but concrete breakdown diagrams that explain 'what the risks are, where things break, and what needs to be implemented.'

In November 2025, the European Data Protection Supervisor (EDPS) published the 'Guidance for Risk Management of Artificial Intelligence systems,' a document that presents such a breakdown diagram quite explicitly. Specifically, it takes the general principles of data protection (fairness, accuracy, data minimization, security, etc.) and 'unravels' them into concrete risks that can occur in each phase of AI development, procurement, and operation, while mapping technical measures (controls/measures) to each. It is a legal document that talks about implementation. That is what makes it interesting. In this article, using this guidance as a guide, I will examine the translation work of 'from principles to risks' in AI governance.

1 The location of the problem

What this guidance emphasizes first is the cold, hard fact that the very act of EU institutions (EUIs) developing, procuring, and deploying AI systems that process personal data entails risks to fundamental rights (including privacy and data protection). Furthermore, under the accountability principle of the EUDPR (the data protection regulation applicable to EU institutions), EUIs are considered to bear the responsibility to identify, mitigate, and 'demonstrate that they have mitigated' these risks.

What is important here is that AI systems are the product of complex supply chains, and multiple entities handle personal data from different positions. In modern AI development, it is rare for a single organization to complete everything from data collection to model training, fine-tuning, and deployment. In many cases, the foundation model provider, the application developer, and the deploying organization (controller) are separated. The more dispersed the responsibility, the more accountability naturally faces pressure to disappear. That is precisely why accountability must not be something one 'prays for,' but must be 'designed' as part of the system.

It should be noted that this guidance is explicitly stated as being provided by the EDPS not as a market surveillance authority under the AI Act, but as a data protection supervisory authority. While it connects to the context of AI regulation, its footing remains firmly in data protection legislation. Therefore, it is necessary to reaffirm that the risk management discussed here is not for protecting the organization's interests, but is a procedure for protecting the fundamental rights of data subjects.

2 Reading data protection through ISO 31000

This guidance relies entirely on the framework of ISO 31000:2018, the de facto standard for risk management, and views risk management as a continuous, iterative process of risk identification, analysis, evaluation, and treatment.

What is noteworthy is that this guidance focuses primarily on 'risk identification' and 'treatment,' clearly taking the position that the evaluation of probability and impact should be left to the context of each organization. Here, one can see resistance to simple checklist-making. The weight of a risk is not determined solely by the attributes of the product (the number of model parameters or architecture). It only takes shape within a context that includes the usage scenario, the nature of the input data, the organization's authority structure, and the scope of outsourcing.

Also, the definition of the risk concept is interesting. The EDPS adopts a breakdown of risk source, event, consequence, and control. And it grasps an 'event' as 'non-compliance with specific legal provisions.' In other words, it models the abstract risk of fundamental rights infringement as 'legal compliance risk.' Of course, compliance does not equal harmlessness. Compliance is the floor of ethics, not the ceiling. However, for the practical problem of where to place the reference points for evaluation and explanation, the decision to 'use explicit provisions as proxy variables first' is an extremely effective pragmatism for creating a common language between lawyers and engineers.

3 Distinguishing between 'AI systems' and 'AI models'

As a pitfall that lawyers often stumble into, this guidance clearly distinguishes between 'AI systems' and 'AI models.' While referring to the definition in the AI Act, it asserts that a model is merely a mathematical representation as a collection of parameters, and by itself cannot interact with users or the environment, and therefore is not a system.

Even if you are using a foundation model (such as an LLM) provided via API, the moment authority design, log collection, UI, workflows, additional rule-based logic, and RAG (Retrieval-Augmented Generation) are combined with it, it takes on behavior as a 'system,' and risks also arise as a system. The performance evaluation (benchmark score) of a model alone and the risk assessment of the entire system are different things.

This distinction hits home particularly in the procurement phase. It is dangerous to feel secure just by receiving a 'model performance sheet' or an 'overview of training data' from a vendor. What is called into question is the connection point of the system: what inputs the model receives in your organization's processing activities, what judgments it assists with, and who can intervene (Human-in-the-loop) and where. The fact that this guidance details risk management in the 'procurement' process, not just development, will be a godsend for those in charge at user companies that utilize AI.

4 Interpretability and explainability are different from transparency

One of the cores of this guidance is that it places interpretability and explainability as a sine qua non for compliance with all data protection principles. It also warns not to easily confuse them with 'transparency.'

According to the EDPS's organization, interpretability and explainability are concepts for the controller (manager) to understand the operation of their own system. In contrast, transparency is the obligation to provide meaningful information to the data subject (EUDPR Article 14, etc.). The former is the technical and cognitive foundation that supports the latter, but just because the former exists does not mean the latter is immediately satisfied. Even if the model's behavior can be analyzed internally, there is no transparency if it cannot be conveyed to the user in words they understand.

What is interesting here is that while it mentions technical explainability tools (XAI) such as LIME and SHAP, it places the unglamorous task of 'documentation' as the top priority measure. It says to leave behind the model type, the origin of training data, accuracy differences between groups, potential biases, limitations, etc., in an understandable form. The implication is that explainability is not about introducing expensive visualization tools, but first about the work of leaving a history of responsibility regarding 'why that model was chosen and how it was nurtured.'

5 Risk breakdown of the four principles and the face of 'technical measures'

The true value of this guidance lies in Chapter 5, which breaks down the four principles of fairness, accuracy, data minimization, and security into specific risk events and maps them to technical measures.

Regarding Fairness, it is meticulously categorized by the type of bias. Examples include insufficient training data quality, training data bias, overfitting, algorithmic bias, and interpretation bias. Here, the normative depth of 'fairness' is inevitably drawn toward 'measurement and correction.' This embodies a certain kind of technological violence—the idea that 'what cannot be measured cannot be managed.' However, it is only by measuring the measurable parts to the greatest extent possible and documenting the limitations and residual risks that the normative concept of fairness can be translated into organizational procedures.

Regarding Accuracy, it distinguishes between 'personal data being accurate and up-to-date (factual accuracy)' in a legal sense and 'model performance (statistical accuracy)' in a statistical sense, referring to the latter as 'statistical accuracy.' This serves as a useful note for legal professionals. The causes of errors are a mix of input data, models, operational environments (data drift), and LLM hallucinations. In particular, operational measures such as 'continuous monitoring and retraining' for data drift suggest AI-specific governance costs that differ from traditional IT systems, which are considered 'finished' once implemented.

Regarding Data Minimization, the conflict between the machine learning intuition that 'the more data, the better the learning' and the principle of 'necessity and minimization' is addressed head-on. Here, the use of 'data sampling' and 'synthetic data' to counter indiscriminate collection is presented. The technical observation that models can be trained on statistically representative subsets rather than full datasets may be the key to resolving the trade-off between compliance and performance.

Regarding Security, AI-specific threats such as model inversion, membership inference, training data 'regurgitation,' and data poisoning are brought to the forefront. These cannot be prevented by traditional firewalls. The application of Privacy Enhancing Technologies (PETs) such as differential privacy and perturbation is recommended.

Furthermore, the feasibility of implementing data subject rights (access, rectification, erasure) is discussed. It is extremely difficult to 'identify' and 'erase' personal data once it has been dissolved into the weights of a neural network. While mentioning nascent technologies such as metadata retention, search tools, and machine unlearning, this guidance also considers output filtering for cases where erasure is impossible. The technical cost of realizing these rights is higher than ever before.

6 Random thoughts

This guidance clarifies that the listed risks and measures are not exhaustive and do not replace compliance obligations under the EUDPR. Nevertheless, practitioners will likely want to turn this into a checklist. This is because humans cannot endure complexity.

Therefore, the way this document is used should be the opposite. Before forcing your company's AI into a pre-existing checklist, first map out the connection points of your organization's AI system (inputs, outputs, permissions, monitoring, and vendors). Then, select which risks from this guidance apply to your specific connection points. Finally, document the 'reasons for your selection.' This should become the minimum unit of accountability that can withstand future disputes or audits.

In short, governance documents are not tools for gaining peace of mind, but rather a history of responsibility. The EDPS document seems to be saying that this history should not be written using the beautiful words of 'principles,' but rather from the raw events of 'risks.' The social implementation of AI systems means that the magic is gone, and they become subjects of messy management. This guidance can be called an extremely practical manual for accepting that messiness.

(Reference) International Standardization and Certification ISO/IEC

(Magazine) 'AI and Law - Random Thoughts'

※ Please refer to the following for the table of contents

note General Terms of Service Article 3, Paragraph 2, First Sentence
3.2 The copyright of digital content created by the creator belongs to the creator.