AI and the Intelligence Explosion / Preparing for the Intelligence Explosion and the 'Speed of Law'—Reflections on reading MacAskill and Moorhouse's 'Preparing for the Intelligence Explosion'

0 Introduction

Japan has finally announced its Basic Plan for Artificial Intelligence and guidelines for ensuring the appropriateness of research, development, and utilization of AI-related technologies. It is precisely because of this timing that I want to record these reflections.

The law is slow. Being slow is not necessarily a bad thing in itself. Deliberation, justification, predictability, and remedy are all supported by this slowness. On the other hand, when the world starts running at ten times the speed, the slowness of the law can become not a virtue, but a void. MacAskill and Moorhouse's 'Preparing for the Intelligence Explosion' is a paper that confronts that void head-on.

1 Challenging the 'Alignment-Only' Thesis

The paper's starting point is a challenge to the common 'alignment-only' thesis. This is the idea that if superintelligence is coming, there are ultimately only two choices (catastrophe if alignment fails, or the superintelligence solves other problems if it succeeds), and therefore, all we should do now is focus on alignment. The authors reject this as an 'all-or-nothing' view, stating that if an AI capable of accelerating research emerges, a series of 'hard-to-reverse' major decisions will arrive within a few years.

They call these 'grand challenges,' which include issues such as weapons of mass destruction, AI-driven tyranny, the scramble for space resources, and the moral status of digital entities. What is important here is the temporal structure in which these grand challenges can accumulate (not just being diverse, but) 'in a short period' and 'in a nearly irreversible manner'.

2 A Thought Experiment: 'A World Where 100 Years Are Compressed into 10'

To ground this temporal structure in a physical sense, the authors propose a thought experiment: 'compressing the 100 years from 1925 to 2025 into the 10 years starting from 1925.' It is a world where the distance from the moon landing or nuclear fission to atomic bomb testing becomes a few years, or even a few hundred days. Before institutions can catch up, the pace of human judgment that institutions assumed as a premise collapses. This is likely one of the core legal implications of an intelligence explosion. It is not that the law is inherently slow; it is that the speed of social change assumed by the law no longer matches the actual speed of society.

The authors state that there is already a 'pacing problem' (the imbalance where digital technology develops faster than regulation and social norms), and that a technological explosion will multiply this by ten. In the vocabulary of legal studies, regulatory lag becomes chronic, and the number of situations where this lag triggers a chain of decision-making failures increases. So, what specific 'legally troublesome situations' will come first?

3 Organizing Legal Issues

(1) Concentration of Power and Lock-in of Values

First is the concentration of power and the lock-in of values. If AI enables a loyal automated military and police force, the cost of maintaining a dictatorship decreases. Coups and 'self-coups' also become realistic through the seizure of AI systems or backdoors. Furthermore, if economic growth explodes and the labor share of income shrinks, wealth can concentrate through rents on capital and land. While this may seem like a topic for political science or economics, the work of the law suddenly increases here. This is because institutional packages such as competition law (regulation of monopolies), national security (placement of supply chains and data centers), and regulations on elections, party funding, and lobbying (responsibility and accountability of those in power) are all called into question simultaneously in a way that must keep pace with the speed.

(2) Technological Lock-in

Second, value lock-in is not just a 'side effect of power concentration,' but the point that technology itself can become a locking mechanism. Examples given by the authors include surveillance, lie detection, 'permanent loyalty' of AI, and even commitment technologies with strong binding power (third-party enforcement like treaty bots). Translated into legal systems, (i) control of surveillance authority, (ii) legitimacy and procedures for automated enforcement, and (iii) constitutional and international legal implications of allowing long-term binding, all become urgent issues. This is less about simply 'regulating AI' and more a design problem of 'how much should the law permit technologies that create constraints?'

(3) AI Agents and Digital Entities

Third are AI agents and digital entities. The authors point out that while protocols, laws, and norms (such as TCP/IP or Section 230) determined the shape of the internet, similar 'initial decisions' will be needed for the influx of AI agents. Who bears the responsibility for agents that cause harm? How do we prove human identity? How do we handle attribution of acts or cross-border issues? While these can be addressed to some extent by a 'combination' of existing tort, contract, criminal, and private international law, in a situation where agents are active in vast numbers, individual litigation-centered remedies may collapse as a system. In other words, before the 'correctness' of liability jurisprudence, the throughput of enforcement becomes the issue.

To go a step further, there is the issue of rights for digital people. The authors note that digital entities currently have no rights by default and that little consideration has been given to the matter, while suggesting the possibility of granting rights (primarily as negative liberties), including property rights, the ability to enter into contracts, and even claims based on torts. At the same time, they are cautious, noting that granting rights could facilitate a takeover, and thus they do not recommend unlimited expansion of rights. Can this tension between 'moral consideration' and 'security' be handled solely by the standard tools of law (the principle of proportionality in restricting rights, categories of exceptions, and procedural guarantees)? One cannot be optimistic here. This is because the very foundations of the concept of personhood—'who is a rights-bearing subject?' and 'how do we treat copies and identity?'—are being shaken. The 'right not to be reloaded (restored)' suggested by the authors is one aspect of this instability.

(4) Space Governance

Fourth is space governance. While resource competition is a classic theme of international law, the authors extend the possibility that 'the first mover gains a permanent advantage by securing resources' to off-world resources. What is at stake for the law here is not the elegance of treaty interpretation, but the early formation of norms. The authors cite examples such as the prohibition of ownership and use of space resources, UN approval requirements, or 'if-then agreements' that trigger once an intelligence explosion begins, and discuss the value of forming agreements in advance. In short, it is the obvious yet difficult point that space law must be designed 'before space becomes crowded'.

4 Why 'Now'?

Based on the above, why is it insufficient to 'wait until aligned superintelligence arrives'? The authors' framework is practical. Specifically, they point to (i) challenges that arise early, (ii) windows of opportunity that close early, (iii) the formation of precedents, (iv) time lags in institutions, human resources, and negotiations, and (v) the veil of ignorance (the possibility of power-sharing agreements because it is not yet determined who the winners will be). Here lies the 'time of law.' Treaties, institutions, and human resources all assume the speed of today's political processes, and there are phases where a delay of a few months or a year can turn into a state of 'indeterminacy'.

5 Model Specifications and Law

In the authors' proposal (AGI preparedness), what legal scholars cannot overlook is the discussion of 'model specifications.' Even if alignment is technically possible, the question of 'what to align with' remains. The authors raise questions such as, for example, when 'obey the Constitution' and 'obey the President's orders' conflict, with what degree of certainty should the AI refuse? This is not only an AI problem but also a problem of constitutional and administrative law theory. This is because 'making AI follow the law' is a distribution problem regarding who assumes the uncertainty of legal interpretation and application, and through what procedures. Moreover, that distribution changes depending on whether the AI remains an advisor or approaches the role of an executor.

6 'It Is Too Late Once You See the Smoke'

Finally, the metaphor in the authors' conclusion is suggestive. They argue that waiting for certainty is an 'evidence dilemma.' One cannot buy fire insurance after smoke starts coming in from under the kitchen door. Whether an intelligence explosion will occur is uncertain, but much of the preparation is 'cheap if it doesn't happen,' while 'the difference is enormous if it does.' Law and governance should be well-suited to this kind of expected value calculation. Nevertheless, the discussion is often drawn into the dichotomy of 'alignment or catastrophe.' I understood the authors' point to be that we must resist that gravitational pull and redesign the reach of the law to match 'speed' and 'irreversibility'.

7 Reflections

In the world of jurisprudence, there is a saying by Hegel: 'The owl of Minerva spreads its wings only with the falling of the dusk.' This means that summaries by law and philosophy always come after events have concluded. We legal professionals have held a certain pride and resignation in this role as the 'wise ones who arrive late.' However, in a world where '100 years are compressed into 10' as depicted by MacAskill and others, the owl waiting for dusk is powerless. By the time it takes flight, the forest itself may have disappeared, or it may have been designated as a no-fly zone for owls.

What personally struck me deeply is that the technical discussion of 'Model Spec' is directly linked to the extremely legal problem of 'constitutional interpretation.' It is easy to input 'obey the Constitution' into an AI, but whether its output aligns with the constitutionalism we know is an entirely different matter. How do we embed the 'rule of law' into the weight parameters of an AI while leaving the ambiguity of natural language intact? This is a programming problem, but it is also a task for legal scholars to redefine 'what is law'.

What comes to mind here is H.L.A. Hart's discussion of the 'rule of recognition' in 'The Concept of Law.' Hart posited the recognition by legal practitioners and citizens as a social fact as the ultimate basis for the validity of a legal system. However, in phases where AI 'executes' the law, by whom and how is this rule of recognition confirmed? When an AI system determines 'this is valid law,' is it mere pattern matching, or does it involve some form of normative judgment? This question touches the core of legal philosophy.

Another point of concern is the reach of the 'veil of ignorance' mentioned by the authors. The 'veil of ignorance' assumed by Rawls presupposed human rational agents. However, in a post-intelligence explosion world, the parties to negotiations may include AI systems or, in some cases, digital entities. Can they also stand 'behind the veil'? If they can, how should the theory of justice be reconstructed? The authors do not directly address this point, but the implications are clearly readable.

The warning that 'you cannot buy insurance after you see the smoke' is a matter of risk management, but it is also a call for 'early intervention in practice' for us legal scholars. Shifting the role of jurisprudence from 'ex-post dispute resolution' to 'ex-ante architectural design.' This involves a 'bet on uncertainty' that is uncomfortable for traditional law students, but the risk of not participating in that bet may actually be the greatest one.

What I felt again after reading this paper is that the discussion of AI governance has moved beyond the stage where it can be confined to a special domain called 'AI Law.' Constitutional law, administrative law, international law, property law, tort law, criminal law, and private international law—all of these are being forced to be re-examined simultaneously under the hypothetical scenario of an intelligence explosion. For legal scholars, this is a threat, but also an opportunity for intellectual excitement. The question is how much time is left to be immersed in that excitement.

AI is moving too fast. The paper assumes a 'world where 100 years are compressed into 10' as a premise for its thought experiment, but personally (though this is a matter of intuition), I think the compression and acceleration of speed by AI are even faster. The speed of AI developments since this summer is too abnormal. Technical progress and the trends in global AI governance have been evolving with such intensity that it is frightening (as you can see if you take a quick look at the 'AI and Law Reflections' I started writing in August). Only a limited number of people are keeping up with this speed, and globally, it is highly conceivable that various things will be decided among a very limited group of people while most others know nothing.

It is precisely because of this that I am writing these 'AI and Law Reflections' with the hope that it might trigger someone to think about something. If I were a young researcher in an era without AI, I would certainly not write things like this. Given my personality, I would highly value tradition, wait patiently to deepen my studies, accumulate knowledge, and build a track record before publishing anything. At the very least, I would not publish in media like this. But in an era with AI, the story changes. Regarding AI, if I wait until I have built a full track record, it is highly likely that everything will be over by then.

In my generation, I think there will be no legal domain that AI does not touch. Yet, it has been a very short time since AI was recognized as a serious legal issue in reality. It is no exaggeration to say that it is around the same time I came to academia. In any case, it is a flat field where everyone is chasing from zero.

It is foreseen that AI will bring about changes greater than those of the Industrial Revolution or the post-war period. For this reason, it is highly likely that relying solely on the accumulation of academic knowledge from the past or on fragmented, specialized expertise will not suffice. The speed and dizzying pace of technological development are unprecedented in human history, and it is clear that technology and practical application will outpace legal responses. Furthermore, current legal systems were not designed with the existence of AI—the first entity in human history to surpass human intelligence—in mind. We can see distortions and vortices emerging in various places. In the future, a transformation of social structures and legal systems will be required. That is precisely why, as a young researcher, I have a strong desire to keep a close watch on AI in order to prepare for this.

(Reference)

First, as a researcher, I never want to turn a blind eye to the intuitions I believe to be true. Continuing to learn is a given, and beyond that, I believe that nothing will remain confined to a single domain. On top of that, I am certain that AI will be at the center of it all. How one perceives this is a matter of personal perspective. I am glad to be a researcher in the age of AI. I can learn a wide and deep range of knowledge both domestically and internationally, and I can meet people from various fields. I consider myself lucky to have been born in this era of great transformation.

As a side note, we are now in an era where the existence of AI allows everyone in the world to instantly acquire intelligence on par with a PhD. The leveling of all things is progressing (*though there is a potential for disparities in the artificial intelligence one can use depending on the model and subscription level). Along with this, I believe that what was previously considered 'amazing' will no longer be so, and what was previously considered 'not amazing,' 'a bit strange,' or 'eccentric' will flip to become 'amazing.'
There will certainly be a rupture and division between the time before and after the full-scale AI era.

How to live in such an era? Finding the act of continuous thinking to be supremely enjoyable is the nature of a researcher. I believe that the required research, the distance to social implementation, and the time and methods involved will all change.

While continuing to prioritize speed and freshness, I would like to record these 'Reflections on AI and Law' with the hope that they might serve as a 'catalyst for thought' for those who notice, those who are quick, those who can see ahead, and those who share the same sensibilities and perceptions.

(Magazine) 'AI and Law - Reflections'

*Please refer to the following for the table of contents

note General Terms of Service Article 3, Paragraph 2, First Sentence
3.2 The copyright of digital content created by the creator belongs to the creator.