Quantum AI as 'Wigner's Friend': Is the Measurement Problem an Unsolved Mystery of 100 Years of Quantum Mechanics?

“It is safe to say that no one understands quantum mechanics.” The famous physicist Feynman once said this. During the pioneering era of quantum mechanics in the last century, the so-called “measurement problem” troubled many physicists. Even in this year, which marks the 100th anniversary of quantum mechanics, is the measurement problem truly an unsolved mystery?

Current understanding makes it clear that the measurement problem does not exist within the theory of quantum mechanics itself. All experimental results to date support the correctness of quantum mechanics, and as shown in the article below, there are no issues with its consistency as a theory.

Therefore, if asked, “Is the measurement problem an unsolved problem in physics?”, the answer is “No.” However, if asked more broadly, “Is the measurement problem unsolved?”, one could say “Yes.”

Quantum mechanics denies the concept of “reality” and the deterministic time evolution of physical phenomena. These facts are perfectly consistent with the results of experiments and observations to date, including the violation of Bell's inequalities. Nevertheless, even in the 21st century, there are those who insist on positions of reality and determinism. In other words, for them, the “measurement problem” still remains. Why is it so difficult to accept, on an emotional level, the world view of quantum mechanics that nature continues to demonstrate? The mystery of human psychology itself cannot be a subject of physics. In that sense, the measurement problem can be called an “unsolved problem for 100 years,” not as a problem of physics, but as a problem of the humanities, such as psychology.

Currently, quantum mechanics is considered a type of information theory, and the wave function is simply a mathematical representation that summarizes the probability distribution of experimental data for physical quantities. Therefore, if you obtain measurement results for the position of a particle in an experiment, the wave function, which is “information” in the form of a probability distribution, is updated to a function localized around that position based on those results. This is what is called the collapse of the wave function, or the collapse of the wave packet. Since the wave function is not a real wave but merely a probability distribution for predicting the future, it was a concept that had to collapse firmly upon observation. Once the essence of “wave function = probability distribution” as an information theory was clarified in this way, it became clear that the measurement problem in physics did not exist at all.

Therefore, now that it is the 100th anniversary of quantum mechanics, not only is there no need to take Feynman's words literally, but we have entered an era where many “quantum natives” who deeply understand quantum mechanics are being born.

In the last century, the understanding of quantum mechanics remained chaotic. It was like, “If the wave function is a de Broglie wave as a real wave, the Copenhagen interpretation, which involves a collapse that violates relativistic causality upon observation, has a serious flaw. Let's call this the measurement problem,” and various competing interpretation theories continued to be proposed. I have summarized this situation in Figure 1.

However, a major change occurred when the “quantum state tomography method” was established both theoretically and experimentally in the field of quantum information. In other words, it was understood that wave functions and state vectors are defined operationally, without ambiguity, solely from the probability distributions obtained by measuring several types of physical quantities. Since this revolution, the way of viewing the theory of quantum mechanics has changed as shown in Figure 2.

In other words, it became clear that “there are no self-contradictions or flaws called the measurement problem in the information theory of quantum mechanics.”

However, quantum mechanics is just one physical theory that should continue to be verified in future experiments in uncharted territories. Whether the theory of quantum mechanics is correct in high-energy regions where quantum gravity takes effect or in the evaporation process of black holes has not yet been verified by experiments or observations. Therefore, there is a possibility that a breakdown of the theory of quantum mechanics will be found in the future. In fact, many physicists, including myself, are eagerly waiting, hoping that “a breakdown of the theory of quantum mechanics will be found in an experiment soon.” That is more fun for a physicist.

In the midst of the confusion, the various proposals from the last century that emerged to resolve the measurement problem should now be accepted as separate theories that challenge the theory of quantum mechanics. Continuing to speak of “resolving the measurement problem, which is a flaw in quantum mechanics,” or expressing that “the measurement problem is an unsolved problem in physics,” only serves to induce misunderstanding among people.

Those interpretation theories of the last century should only be evaluated as modern physical theories if they can predict that they will cause phenomena different from quantum mechanics in future experiments. For example, the many-worlds interpretation is often claimed to make exactly the same predictions as the Copenhagen interpretation of quantum mechanics. And since the wave function in that interpretation is not updated or collapsed by observation, it has become useless in modern times.

There is also a “spontaneous collapse theory” which posits that the wave function is real and its collapse is a physical process, but this has the potential to produce results different from quantum mechanics in future experiments, and it is recognized as having significance as a “non-quantum mechanical theory” that challenges quantum mechanics. There are various other “non-quantum mechanical theories” that make verifiable predictions for the future.

However, what is important is the fact that everything from experiments in the micro-domain to data from cosmic observations has remained consistent with quantum mechanics. At present, quantum mechanics remains correct. There is absolutely no necessity in the data to consider those non-standard theories.

A recent issue of Scientific American featured an article titled “Can AI Save Schrödinger’s Cat?”. The author is science writer Anil Ananthaswamy. A translated version of the article has also appeared in Nikkei Science magazine.

However, this article states that a serious measurement problem exists in the theory of quantum mechanics as a matter of physics. And it reports that an idea to solve this unsolved problem using quantum AI has been put forward by a certain research group. The paper written by that group is below.

This paper deals with the paradox known as “Wigner's Friend,” proposed by Eugene Wigner. Wigner and von Neumann, who were talking about this story at the time, were thinking about various roles of “consciousness” in quantum mechanics.

The story Wigner started in that process is a thought experiment now called “Wigner's Friend.” For example, consider the experimental setup of Schrödinger's cat. It is a story where a machine operated by gamma rays from an unstable atom creates poison gas inside an isolated room, and the cat in that room enters a superposition of being alive and dead. If quantum mechanics were correct, such a superposition of “life” and “death” would be realized, but Schrödinger did not believe it. This is because he stubbornly thought, “The cat must be either alive or dead.”

So, what if we prepare an observer named Charlie in this room to observe the cat? Because Charlie is taking precautions, he will not die from the poison gas and can safely continue to observe the cat indefinitely. In many scenarios, Charlie is set up to be Wigner's friend. That is why he is called 'Wigner's Friend.' And Wigner himself is outside the room.

For Charlie, the cat is alive during the time the gas is not released, and the moment the gas is released, the cat dies. There is no time period where both life and death occur simultaneously. However, according to quantum mechanics, for Wigner, who is outside after the experiment begins, the cat is in a superposition of being alive and dead at any given time. And it is only when Wigner asks Charlie about the cat's life or death that the cat collapses into either a living or dead state. If the theory of quantum mechanics is correct, that is how it happens.

However, many people cannot accept this explanation on an emotional level. This 'feeling' problem is the 'measurement problem.' These people, for example, argue as follows.

'In quantum mechanics, Charlie's memory that "the cat is alive now" and Charlie's memory that "the cat is already dead" are in a superposition corresponding to the cat's life or death state. That is absolutely wrong! Charlie is a human with only one consciousness! The consciousness of Charlie inside the room is working continuously, and for that consciousness, at each moment, the cat is either alive or dead. Therefore, for Wigner outside, as an objective fact, neither the cat nor Charlie is in a superposition!'

The point to be careful about with such an argument is that it assumes without basis that "for Wigner outside, as an objective fact, the cat is either alive or dead, and Charlie's memory and consciousness cannot be in a superposition." To Wigner, Charlie is nothing more than a collection of elementary particles. Since Charlie is a collection of elementary particles just like objects such as glasses or robots, and given that superposition has been experimentally verified for such objects, it is very strange to think that "for Wigner, Charlie cannot be in a superposition." The perspective of Charlie, who says "the cat is either alive or dead," and the perspective of Wigner, who says "the cat (and Charlie) is in a quantum superposition," do not contradict each other; they are perfectly compatible. Therefore, the argument of those who cannot yet accept quantum mechanics emotionally, that "Charlie's view should be the same as the external Wigner's view," is just a self-serving assumption, is logically incorrect, and lacks rationality.

This assumption is similar to a common mistake regarding the passage of time for clocks of Alice and Bob in different inertial frames in special relativity. For Bob, the clock of the moving Alice runs slower than Bob's clock. This is an undeniable fact for Bob. However, if Bob were to argue as follows, those who correctly understand relativity would think it is strange, wouldn't they?

'In relativity, it is said that from Alice's perspective, my moving clock runs slow, but from my perspective, Alice is the one moving, and certainly that Alice's clock runs slower. Therefore, it is absolutely wrong for Alice to say from her position that "Bob's (that is, my) clock runs slow"! I am a human with only one consciousness! My consciousness is working continuously, and for me, the speed at which the clock runs is completely normal, and it is Alice's clock that is running slow. Therefore, for Alice too, as an objective fact, Alice's clock must be running slow! Relativity has a flaw!!'

People who cannot accept the results of quantum mechanics on an emotional level are simply arguing something similar to that.

Certainly, in quantum mechanics, for Wigner outside, it is a superposition of both the state where the cat is alive and Charlie thinks the cat is alive, and the state where the cat is dead and Charlie thinks the cat is dead. Therefore, for those who still harbor the "psychological measurement problem," accepting that macroscopic superposition on an emotional level involves great difficulty. That is why the "Wigner's Friend" thought experiment also feels like some kind of great paradox.

On the other hand, it is possible to think of another non-quantum mechanical theory and provide predictions different from quantum mechanics for this "Wigner's Friend" experiment. For example, if the existence of "consciousness" is special and Charlie's "consciousness" must be an objective existence even for Wigner outside, then if a result is obtained that forces us to admit that Charlie cannot be in a quantum superposition even from Wigner's position, it would mean that the theory of quantum mechanics was wrong.

The aforementioned paper proposes the possibility of experimentally verifying an inequality that would hold if a different theory opposing quantum mechanics were correct. Conversely, if quantum mechanics is correct, that inequality is violated, so these theories can be distinguished and verified experimentally. In that process, the authors of the paper have a quantum AI observe "Schrödinger's cat" instead of Charlie.

I would like you to read the paper for details, but I will introduce the essence of the idea. The derivation of the inequality itself, which can be verified by experiment, relies on the presence or absence of quantum entanglement in a two-body system, like Bell's inequality, but that quantum entanglement itself is just a tool here and is not the essence of the "Wigner's Friend" experiment. Hereafter, for simplicity, I will explain this essential part assuming that Charlie observes a two-level spin particle in a pure state of superposition.

In this setup, the initial state of Charlie's brain is, as shown in Figure 3, a "which one?" state where the spin direction is unknown.

After that, Figure 4 shows the case where Charlie interacts with the spin particle to measure it, perceives it as upward, and remembers the result.

And Figure 5 shows the case where he perceives it as downward and remembers the result.

In the non-quantum mechanical theory treated in the paper, it is assumed that even for Wigner outside, it is either Figure 4 or Figure 5; in other words, it is assumed that it is not in a quantum mechanical superposition state like Figure 6.

Since an experiment actually using a human Charlie is not realistic, the authors of the paper conceived an experiment where an intelligent quantum AI acts as a proxy for Charlie, as shown in Figure 7.

If we assume that a quantum AI is an entity with 'consciousness' that causes the wave function to collapse, then as shown in Figures 8 and 9, it perceives and remembers only one of either the spin-up or spin-down state.

Just like Charlie's brain in Figure 6, from the perspective of Wigner, who is outside in quantum mechanics, the superposition state in Figure 10 can be realized.

However, in the non-quantum mechanical theories considered in the paper, even from the perspective of Wigner outside, only one of either Figure 8 or Figure 9 is realized, not Figure 10.

The essential difference between quantum mechanics and the non-quantum mechanical theories in this paper ends here. To produce a verifiable inequality, the experimental setup in the paper is made even more complex. However, this is merely a technical matter, so please refer to the original paper for details. In the explanation above, for simplicity, I assumed that Charlie's spin particle was in a single pure state, but as shown in Figure 11, the paper prepares an entangled state between that spin particle C of Charlie and a spin particle B held by Bob, who is outside.

And another observer, Alice, who is outside, boldly treats Charlie himself as experimental material. With a 50% probability, Alice orders Charlie to measure spin particle C and report whether it is up or down. With the remaining 50% probability, after Alice has Charlie perform the measurement, she performs a 'time reversal' of Charlie and spin particle C, including Charlie's records and memories. Time reversal is the process of returning the process from the future to the past, like playing a movie film in reverse.

Charlie should have once perceived whether spin particle C was up or down, but due to this grand 'nasty' time reversal by Alice, he loses all those records and memories. If quantum mechanics is correct, spin particle C should have returned to the original entangled state of Figure 11. However, in non-quantum mechanical theories that assume 'even for Alice outside, Charlie's consciousness after the measurement perceives only one of either spin particle C being up or down,' it does not return to the state of Figure 11 even after time reversal. Just as in the verification of the violation of Bell's inequality, the basic idea of the aforementioned paper is to quantify that difference with an inequality and verify it through experiments. And they have substituted this role of Charlie with a quantum AI that can be experimented on.

Although it is a complex setup using quantum entanglement, the essence of the physics itself is contained in the setup of Figure 10 for the quantum AI. If quantum mechanics is correct, the state of Figure 10 of the quantumly entangled spin particle C should also have returned as shown in Figure 12 by Alice's time-reversal device.

On the other hand, in non-quantum mechanical theories, it is impossible to return to the state of Figure 12 through time reversal. I also think it is an interesting paper that shows the possibility of verifying quantum mechanics in future experiments. However, I was personally very concerned that the authors of this paper and the reporter from Scientific American who introduced the paper regard this 'Wigner's Friend' as one of the measurement problems in physics and claim that it will be solved as physics through future time-reversal experiments using quantum AI. This is because that mention is based on very old values from the last century.

I will repeat that there is no defect in quantum mechanics theory itself that can be called a 'measurement problem' as a physical theory. I believe that in the current quantum era, everyone should know that there is only a 'psychological measurement problem' for those who cannot easily accept the claims of quantum mechanics at an emotional level. And I hope you will properly understand that the 'Wigner's Friend' experiment with quantum AI in this paper is only verifying the difference between the theory of quantum mechanics and non-quantum mechanical theories.