Yann LeCun on the 'Limits of LLMs' and the Essence of Next-Generation AI
In recent years, the discussion surrounding AI (Artificial Intelligence) has been relentless, and expectations in society and industry have been rising, especially with the advent of Large Language Models (LLMs). At the 2025 NVIDIA GTC (GPU Technology Conference), an intriguing conversation took place between Yann LeCun and Bill Dally, covering a wide range of topics including the future direction of AI, the computing resources required, and the significance of open source. This article provides an easy-to-understand explanation of what is happening at the forefront of AI and computing, focusing on the content of that discussion.
1. The Direction of AI Beyond Large Language Models (LLMs)
1-1. Declining Interest in LLMs and the 'Improvement Phase' in Industry
At the beginning of the conversation, when asked, 'What is the most exciting advancement in AI?', Yann LeCun surprisingly offered the frank opinion that he is 'no longer that interested in LLMs (Large Language Models).' This stems from his recognition that current LLMs have entered an 'improvement phase' in the industry, where the focus is on fine-tuning performance by pouring in more data and computational resources.
In fact, LLMs have achieved significant results in text generation and dialogue systems, and models represented by 'ChatGPT' and 'LLaMA' have garnered great attention. However, LeCun emphasized that these are merely success stories within the limited space of 'language,' and that to truly approach 'general intelligence,' understanding the physical world, persistent memory, and reasoning and planning capabilities are essential.
1-2. The Importance of Models That Understand the Physical World
For AI that deals with the physical world, the major challenge is how to handle high-dimensional, continuous data rather than 'tokens' (discrete units) like text. For example, it is necessary to implement 'spatial awareness' and 'motion prediction'—things humans do daily—into AI, such as through images, video, and sensing information obtained from autonomous driving and robotics.
The conversation also mentioned that 'predicting the future at the pixel level in video has failed.' This is because video is composed of too many diverse elements (lighting conditions, randomness, unobservable factors, etc.), and simple reconstruction or pixel-level prediction cannot lead to correct learning. Therefore, the proposed method is prediction in an 'abstract representation space'—that is, converting data into high-dimensional features and using them to perform prediction and generation.
1-3. The Concept of 'JEPA (Joint Embedding Predictive Architecture)'
The architecture LeCun highlighted is 'JEPA (Joint Embedding Predictive Architecture).' Instead of conventional generative models trying to reconstruct pixels one by one, this mechanism introduces a system that first converts inputs into a high-dimensional abstract space and then predicts unobserved parts.
For example, trying to perfectly generate the next few frames from a video frame at the pixel level requires enormous computational costs and data. However, JEPA is expected to be more efficient and capable of more semantically and physically valid predictions because it performs predictions in a latent space that captures the features of the video.
2. AGI, AMI, and the System 1 and System 2 Thinking Models
2-1. 'AMI' Instead of 'AGI'—The Timeline for General Intelligence
The conversation also touched upon the term 'AGI (Artificial General Intelligence),' which is frequently heard. LeCun is skeptical of the term 'AGI' and instead supports the concept of 'AMI (Advanced Machine Intelligence).' He points out that human intelligence is more specialized than one might imagine, and it is difficult to call it 'truly general.'
Furthermore, he noted that predictions often heard, such as 'AI that surpasses humans will be born in 10 years,' have appeared and failed repeatedly over the past 70 years. Instead of reaching AGI all at once, he predicts a gradual progression: 'small-scale empirical successes in 3 to 5 years, followed by large-scale intelligence levels on a 10-year scale.'
2-2. System 1 and System 2—The Analogy of Intuition and Deliberation
Toward the end of the conversation, the topic of 'System 1 (intuitive thinking)' and 'System 2 (deliberative thinking)' from psychology was introduced. System 1 refers to automatic and reflexive processing (such as skilled driving), while System 2 refers to conscious and logical processes such as planning and reasoning.
Current LLMs and many reasoning models are closer to a System 1 approach, which 'generates tokens one after another,' and they adopt methods such as searching for promising outputs from a large number of combinations. However, LeCun's view is that a new System 2-like architecture is required for full-scale physical understanding, continuous spatial awareness, and building world models 'the way a baby learns.'
3. The Future of Hardware and Open Source
3-1. Reflections on Spiking Neurons, Analog Computing, and Quantum Computing
Building and learning high-dimensional and continuous world models requires enormous computational resources. Bill Dally reflected on the history of NVIDIA GPUs achieving a performance increase of several thousand times over the past decade, while also mentioning the potential of new technologies such as spiking neurons, analog computing, and quantum computing.
However, LeCun remains skeptical about whether neuromorphic computing, analog computing, or quantum computing will bring about major breakthroughs in the short term. The main reason is that biological neurons also ultimately exchange information in a digital form called 'spikes,' and building large-scale analog networks is currently difficult due to trade-offs with existing semiconductor technology and reusability.
3-2. Advantages and Challenges of Open Source Models
While developing AI technology requires vast resources, it is not realistic for a single company to handle everything on its own. This is where the trend of collaborative development through open source is gaining attention. For example, 'LLaMA' and 'LLaMA 2,' released by Meta (formerly Facebook), have generated significant buzz, and researchers and companies around the world are using them as a base to develop applications.
“An environment where everyone can research on the same project” and “Distributed learning of data that supports diverse languages and cultures” have the advantage of aggregating insights on a scale that a single company could never achieve. On the other hand, the significance of companies releasing the latest models they developed at great cost for 'free,' and the balance with their revenue structures, will remain a challenge. LeCun explained that 'if a company can expect a sufficient return from a specific product generating profit, open-sourcing it is not necessarily disadvantageous to the company,' and emphasized the philosophy that 'open-sourcing platforms supports the diversity of the world.'
4. AI Possibilities, Risks, and What We Can Do
4-1. Current Status of Social Implementation in Fields Like Healthcare and Autonomous Driving
As for examples of AI social implementation, significant results have already been reported in areas such as medical image diagnosis and drug discovery support. In fact, LeCun emphasized practical use in familiar fields, such as 'cases where MRI imaging time can be reduced to 1/4 of the conventional time' and 'the reduction of accidents due to the spread of emergency braking systems in automobiles.'
However, such practical use requires strict institutional and evaluation frameworks where 'malfunctions are not tolerated.' It has been pointed out that the risks of misrecognition in autonomous driving and medical equipment are significant, and implementation will take time.
4-2. Risk Countermeasures and the Importance of Open Research
Many comments also expressed concerns about 'the risk of AI creating serious misinformation or fakes.' However, according to LeCun, social media operating companies are already establishing mechanisms to detect malicious posts using AI, and in reality, no major 'negative tsunami' has been observed.
On the other hand, there is strong skepticism regarding the scenario proposed by experts that 'AGI will suddenly emerge and threaten humanity.' In reality, technological progress happens in stages, and human society has found countermeasures each time. Furthermore, many researchers have expressed the opinion that by having the open research community take the lead, the development of more diverse and safe AI models can be expected.
5. Voices from the User Community
In the actual video comment section, diverse perspectives and praise were shared. For example, '@AravindanUmashankar' offered praise, saying, 'Standing on the shoulders of giants like Yann LeCun, more people will move the world forward,' while also expressing the expectation that 'young leaders who will carry the next generation should pursue a safe form of AI.' Additionally, there were pure words of praise like '@behrad9712' saying, 'He (LeCun) is very open-minded and wonderful,' as well as calm analyses like '@RufusShinra' noting, 'Both open source and closed models are progressing simultaneously, and their economic impact has already begun.'
Furthermore, '@HSt-t7n' summarized the discussion timeline in great detail, introducing 'what was talked about at what point in the video,' which was impressive as it showed users spontaneously organizing content and sharing knowledge as a community. There is no doubt that this kind of interactive exchange of opinions is the driving force behind open AI research.
The conversation between Yann LeCun and Bill Dally was highly suggestive in indicating where AI is headed. While Large Language Models (LLMs) are attracting attention, new challenges such as modeling the physical world, reasoning, and planning lie ahead. New architectures like JEPA are being proposed, and the integration of 'intuition and deliberation' by combining System 1 and System 2 is beginning to show the possibility of leading to Advanced Machine Intelligence (AMI).
In addition, the computing hardware that supports this research will continue to evolve, and the foundation for researchers around the world to cooperate with each other is being prepared through the open-source trend. To maximize the benefits brought by AI and minimize the risks, intellectual exchange across borders and the formation of a highly transparent community will be essential, rather than exclusive development closed off to certain companies or regions.
The future where each of us can master an AI assistant as 'our own subordinate' is not far off. When that time comes, what kind of 'intelligence' will we coexist with, and what kind of society will we build? We are truly being asked to face this question seriously right now.
Related Articles