Can LLMs Become AGI? Decoding the Path to 'Creative AI'
Can Large Language Models (LLMs), which drive the evolution of artificial intelligence (AI), truly reach 'AGI (Artificial General Intelligence)'?
In the Greylock Partners podcast 'Will LLMs Get Us To AGI?', VC Martin Casado and researcher Vishal Val tackled this fundamental question. The discussion focuses on two main points: how far LLMs can mimic human intelligence, and whether they can create new knowledge or science.
1. The Essence of LLMs: Intelligence as a Probability Distribution
1-1. The Mechanism of Predicting the 'Next Word'
According to Val, LLMs are essentially systems that 'choose the next token (word) from a probability distribution'.
For example, if given the sentence 'The cat sat on the...', 'mat' is chosen with the highest probability.
In this way, the model learns statistical patterns of language from vast training data and estimates the 'next word that should come' within a probability space.
1-2. Manifold Theory and Human Reasoning
What is interesting is the idea of viewing LLM thought as a 'geometric manifold'.
The model compresses complex high-dimensional spaces and moves along a low-dimensional 'state space'.
Val points out that 'humans also think by reducing infinite reality into finite conceptual spaces'.
In other words, the reasoning process of an LLM can be called a mathematical analogy of human thought.
2. 'Chain of Thought' and Entropy
2-1. 'Entropy' Governing Prediction Confidence
LLM output can be measured by 'entropy' (uncertainty).
If the probability distribution of the next word is wide, it is high entropy; if the choices are limited, it is low entropy.
The model is more confident in a low-entropy state, and may speak 'confident nonsense' in a high-entropy state.
2-2. Chain-of-Thought
On the other hand, 'Chain-of-Thought' is a process of reducing entropy step by step.
By breaking problems into small steps and building reliable reasoning at each stage, it ultimately reaches a highly reliable conclusion.
This idea mathematically explains why LLMs 'appear to be thinking'.
3. The 'Matrix Model' and Bayesian Inference
In the 'Matrix Model' proposed by Val,
each row corresponds to a 'prompt'
each column corresponds to a 'vocabulary token', and the LLM learns a compressed representation of a portion of that massive matrix.
When an unknown input is given, the model references a part of the matrix learned in the past and generates a response by interpolating the probability distribution in a Bayesian manner.
This is also the mechanism of 'In-Context Learning'.
Val compares this mechanism to 'the process by which humans understand new contexts from limited experience'.
4. Can LLMs 'Evolve Themselves'?
4-1. Is 'Self-Improvement' a Fantasy?
Val is cautious about the vision of 'self-evolving AI' that many commentators talk about.
He states that 'LLM output is merely an inductive closure of the training data', and that it is impossible for it to discover new science or theories on its own outside the training scope.
For example, an AI that only knows Newtonian mechanics cannot derive the theory of relativity.
4-2. AGI Means Creating a 'New Manifold'
According to Val's definition, AGI is an intelligence that does not merely 'navigate' known manifolds, but 'creates' new ones.
In other words, it is an entity capable of constructing unknown physical laws or new mathematics.
He argues that to realize this capability, a new structure beyond the 'Transformer' architecture is required.
5. The Next Step: New Architectures Beyond LLMs
Val points out that simple parameter scaling or data augmentation will eventually hit a limit.
'Just as the evolution of the iPhone stopped at camera performance improvements, LLMs remain stuck on a "smooth manifold".'
He states that the key to the future will be the emergence of new types of AI with 'energy-based models' and 'approximate simulation capabilities'.
By integrating not just language, but vision, action, and physical simulation,AI may advance to the stage of 'generating concepts from experience'.
Conclusion
While LLMs can mimic human thought to a surprising degree,they cannot discover the 'new science' that produces creative leaps.
Realizing AGI requires an architectural leap that goes beyond existing statistical learning.
Until that day comes, LLMs will remain 'tools that extend human intelligence'.