Summary
- OpenAI's Q* is AI technology developed by OpenAI that is possibly related to AGI, which differs greatly from current AI.
- AGI understands concepts, unlike regular AI, which only recognizes patterns. It can apply logic to questions rather than going off of what's been done before.
- Q*, which can potentially apply logic to mathematics problems, could lead to AI capable of solving complex problems. This allegedly sparked concerns and led to the ousting of Sam Altman from OpenAI.
Even after Sam Altman was ousted from OpenAI before his quick return, it was never made clear why he was voted out in the first place. Chief scientist of OpenAI Ilya Sutskever, according to a report from The New York Times, thought that Altman wasn't putting enough time into worrying about the risks of AI. Later, Sutskever took to X (formerly Twitter), sharing that he, "deeply regret[s] my participation in the board's actions. I never intended to harm OpenAI."
Following that debacle, we learned from The Information and Reuters that OpenAI had been working on an algorithm it dubbed Q*, a breakthrough AI discovery that was said could "threaten humanity." While it's not totally clear, one source told Reuters that they believe the breakthrough was related to Artificial General Intelligence, otherwise known as AGI. Interestingly, in an interview with The Verge, Altman unwittingly confirmed that at least parts of the reports are true, as he said he had "no comment on that unfortunate leak." But what is Q*? What is AGI, and why is it concerning?
What is AGI?
Artificial General Intelligence (AGI) differs from regular artificial intelligence in its ability to reason. Artificial Intelligence in the forms of ChatGPT, Google Bard, and Bing Chat are great at pattern recognition, identifying things from their trained datasets that match your question and making inferences from there, but it's not like they can think for themselves. For example, if I ask ChatGPT how to cook a 12-inch pizza margherita in the oven, it can give me the right answer from other recipes and tutorials that it has learned from. All of this is to say that ChatGPT understands data, but it doesn't understand concepts.
AGI is different. If I ask ChatGPT a math question that's never been asked before, the answer will be incorrect since it has no logic. Asking the question will force it to pore through its data to see if it has come across the question before. If it hasn't, then it finds the next nearest thing and uses that to inform a "guess" (though more accurately, a hallucination). In the above example, the actual answer is 42,468, not 42,668. There is no "thought" in its process; it just knows it needs to fill a number into its response.
Where AGI would differ is that it would understand the concept of mathematics and what is being asked and can deduce the answer to the above question. Heuristics in artificial intelligence are essential strategies for problem-solving that make things quicker to solve for an AI where a perfect solution may take too long, and in AGI, a machine may essentially create its own set of heuristics for a task at hand.
Why Q* is so perplexing is that it's allegedly capable of logical-mathematical understanding. If an algorithm like that were to get more advanced, it would be capable of mathematics that some of the world's best mathematicians can't do, or, even more worryingly, could surpass what the best of humankind can currently do.
Understanding AGI and Q*: Machine learning in video games
AIs learning to play games is fascinating, and it's a great visual aid for how machine learning algorithms are trained. Watching the above video from SethBling, you'll see that his MarI/O machine learning Lua plugin resulted in an AI playing Super Mario Bros very strangely. It constantly jumps and spins, but that's simply because it learned through the course of its training that it got further when it was jumping and spinning. It "evolved" other traits along the way that allowed it to get further, but the AI doesn't know what made it better, just that it achieved closer to its goal when it was doing different things, and so it kept those features.
As for how an algorithm like this "learns," it has a "reward function," which is an abstract measure of success to work towards. In this case, that reward function is to get to the end of the level, and over a ton of iterations, the AI learns how to maximize its reward by completing the level. You can run many algorithms like this on a powerful PC at home.
With researchers at OpenAI saying that Q* could "threaten humanity" and the kneejerk reaction of firing Altman allegedly being the result of that, it lends credence to something brewing.
Taking the above concept, imagine the supposed Q* algorithm had an understanding of mathematics and was capable of basic computation. With more computational power and an inherent understanding of mathematical concepts, this kind of trial and error could lead to an AI capable of self-policing itself. If it can understand mathematical concepts and know when it's right or wrong, along with being able to form its own heuristics, it could be given free rein to solve complex mathematical problems. Like in the above video where Mario spinning inadvertently gives the AI an advantage, something that humans would never have done, it's possible that an AGI could think outside the box and discover patterns in the world that humans haven't yet.
Of course, this is hugely speculative, and it may never see the light of day. However, with researchers at OpenAI saying that Q* could "threaten humanity" and the kneejerk reaction of firing Altman allegedly being the result of that, it lends credence to something brewing.
What Q* and AGI means for the future
We don't know how close Q* is to genuine artificial general intelligence, but there's enough noise coming from multiple directions around OpenAI that suggests something is coming. Quoting Altman at the APEC CEO Summit the day before he was fired by OpenAI, he stated, "I think this is going to be the greatest step forward that we've had yet so far, and the greatest leap forward of any of the big technological revolutions we've had so far" in an answer to a question asking why he was devoting his life to this work.
To look into things further, the open letter signed by hundreds of employees at OpenAI stated that interim CEO Mira Murati told the leadership team that allowing the company to be destroyed "would be consistent with the mission." Given that the mission of OpenAI is, as stated on its website, "to ensure that artificial general intelligence benefits all of humanity," reading between the lines seems to imply that the company has AGI in a way that it does not believe is safe.
As it stands now, things appear to be full steam ahead for OpenAI. Where that leads is anyone's guess, but with rumors circulating about what Q* is capable of, we'll likely hear more as the months go on.