The Skyrocketing Cost of Scientific Peer Review in the AI Era and the Reconstruction of Science

The ongoing AI revolution is transforming social structures around the world even more than the Industrial Revolution did with the invention of the steam engine. This is also true in academia, including universities and research institutions. This impact is already appearing on arXiv ({arXiv.org e-Print archive), a preprint server for papers in science fields such as physics. A preprint that came out recently contained research on quantum energy teleportation, one of my research fields, and I was surprised when I looked at the list of cited references at the end.

[2] is indeed my paper, but the problem is reference [3]. A paper that I supposedly wrote with a co-author is cited. However, I have no record of writing such a paper, and the paper itself does not exist. Furthermore, the co-author is a fictional person. I have concluded that this is a hallucination caused by AI generation.

Currently, if you use AI, it is possible to generate a moderately decent paper in the theoretical physics industry in a few hours. Please see the article below regarding this.

While it is becoming common for researchers in every scientific field to use AI, the human author of the paper bears the ultimate responsibility for its content. However, in an era where papers can be generated quickly with AI, it is only a matter of time before irresponsible papers like the one mentioned above are submitted in large numbers to arXiv and peer-reviewed journals, causing problems. It seems likely that the number of scientific researchers who engage in such irresponsible work will continue to increase. In particular, one can sympathize with the psychology of young researchers, for whom the number of papers is a condition for their next position, wanting to write many papers regardless of quality while using generative AI. However, in this AI revolution, the nonsensical evaluation method based on the number of papers will likely be quickly abandoned in each field, and whether the research results are essential will become more of an issue.

Furthermore, not limited to theoretical papers, AI is also having a major impact on the falsification of results in the peer review of experimental papers. For example, in the biological research community, it is common to attach experimental image data to papers, but there has been growing confusion that human peer review cannot detect image data falsified by AI. This is a serious situation.

One of the reasons the world has been able to advance science and mathematics through cooperation until now is the peer review system for papers. However, if a vast number of worthless AI-generated papers are submitted to peer-reviewed journals, this peer review system will completely collapse. With only the conventional human check of information within papers, it will be impossible to eliminate the falsification of data and images, and it will also become extremely difficult to find human researchers who will review a vast number of papers.

Also, human editors of peer-reviewed journals will be required to make proper judgments on the vast number of papers submitted via AI generation, one by one. Eventually, it will become a battle not just of numbers, but of the length of the papers. For example, what should be done if a single AI-generated paper claiming to have solved an unsolved mathematical problem becomes millions of pages long?

With only the current peer review process, it will become impossible to advance science and mathematics simultaneously around the world in the future. Changes to address this will occur in each field, and one possibility is the emergence of "expert appraisal companies" that provide a guarantee of the legitimacy of paper content. This is a format in which a guarantee company, organized around specific globally trusted researchers, selects themes with high importance and scientific legitimacy from a vast number of papers, collects important papers along those themes, and presents them to the world in the form of a "selected collection of papers" under the name of the selector or the company. Of course, it is impossible for that company to read and examine every paper in detail. Therefore, the experts will scrutinize a limited number of papers on themes they are concerned about and provide an evaluation. This will be a system of scientific experts, very similar to the work of art dealers and museum curators in art.

You might mistake this "expert appraisal company" for the current top journals where editors choose which papers to accept, but that is not the case. First, in current peer-reviewed journals, editorial board members look through and make judgments on all submitted papers. However, in the AI era where a vast number of submissions arrive, that will become impossible. It means that general researchers will not only do "research" in the current sense, but will also act as experts, selecting important papers in their field, introducing them in the form of a collection of papers, and selling them, engaging in a curator-type credit business. It is the feeling of individual researchers launching journals that they choose, peer-review in detail, and introduce on their own. The "selected collections of papers" that survive in that competition will likely determine the trend of science for the whole world. Current major top journals will also participate in this, but I predict that in many fields, the agile expert appraisal business of individual researchers will win.

However, even for a limited number of papers, the economic cost of this peer review system will be enormous. Regarding experiments in various fields, for example, it is envisioned that the expert appraisal company will dispatch professional human appraisers to the research group that submitted the paper to actually check the experimental equipment and raw data with their own eyes and perform fact-checking to ensure it is not a fabrication, but that will take a huge amount of time and financial cost.

It is highly likely that the high cost of supporting such a peer review system will become a burden for any country. Then, what might happen next is the emergence of local scientific communities that have given up on sharing science with the world. Trusted researchers in one country or a specific region of countries may take the lead, compiling high-level data and knowledge that can be verified and replicated only within that community, and based on that, science may be advanced only within that region. This is the idea of trying to minimize the influence of external misinformation. The possibility of such a dystopia emerging is also increasing.

There is a harsh reality that organizations consisting only of university personnel who are immersed in their immediate work with the same mindset as a few years ago, while maintaining a normalcy bias, will have no choice but to collapse in the near future. There is a need now for more active discussion across fields about the state of science in the AI era.