Elon Musk announced that the next generation of his company’s AI chatbot Grok may be just weeks away from release, describing it as “scary smart” and claiming it had already outperformed every other AI model in testing.
The xAI CEO made these remarks during the World Governments Summit in Dubai on February 13.
“At times, I think Grok-3 is kind of scary smart,” Musk said. “It comes up with solutions that you wouldn’t even anticipate—you know, not obvious solutions.”
The chatbot developers utilized unique training methods for Grok-3. Instead of using real-world data like ChatGPT, Grok-3 relied on synthetic data and employed a self-correcting mechanism to maintain logical consistency. It got so accurate, Musk claimed, that even when it encountered incorrect information, the system reflected on the data and removed content that didn’t match reality.
The computational demands for training Grok-3 were massive. Experts calculate that it required 200 million GPU hours, dwarfing its Chinese competitor DeepSeek-V3’s 2.7 million hours. It ran on xAI’s Colossus supercluster with 100,000 Nvidia H100 GPUs—ten times more computing power than its predecessor. Even without fine-tuning, Musk claimed the base model performed better than Grok-2.
Grok-3’s integration with X, Musk’s social media platform, gave it the advantage of being able to scrape the social media app in real time instead of relying on browsing the web. The system can pull real-time data from X, and features what the company called “Unhinged Mode“— which, according to xAI’s own FAQ, is “intended to be objectionable, inappropriate, and offensive.”
The system isn’t quite ready for prime time, though. Musk compared the remaining work to finishing a house: “That last 5% where you do the drywall and do the painting and the trimming—even though it’s not much work, it transforms the house.”
However, it may be released sooner than OpenAI’s GPT-4.5, at least, which Sam Altman said could be released in weeks or months.
“Probably (Grok-3) gets released in about a week or two,” Elon said. He didn’t clarify whether the new version would be publicly available or put behind a subscription, as happened with Grok-2 at first.
Competition in the AI space has intensified. While ChatGPT dominated the market share in 2024, Chinese open-source model DeepSeek-V3 emerged as a serious contender, outperforming both GPT-4o and Meta’s Llama 3.1 despite using far fewer resources.
Grok was first made available on X Premium, which substantially limited its availability. It was later released free to all users of Musk’s social media platform, with a new standalone website now available for everyone else.
Image: Grok.com
xAI enters reasoning AI battle
Major AI players are switching focus to reasoning models, developing AI models that are able to reflect on specific problems and find ways to solve them after a long and extensive chain of thought reasoning.
The idea was first explored by Matt Schumer, back when Reflection 70b was announced. The model was trained to incorporate Chain of Thought reasoning, and was supposed to beat Claude 3.5 Sonnet at complex tasks despite being just a Llama 70b finetune.
I’m excited to announce Reflection 70B, the world’s top open-source model.
Trained using Reflection-Tuning, a technique developed to enable LLMs to fix their own mistakes.
405B coming next week – we expect it to be the best model in the world.
Built w/ @GlaiveAI.
Read on ⬇️: pic.twitter.com/kZPW1plJuo
— Matt Shumer (@mattshumer_) September 5, 2024
That didn’t work, but just a few weeks later, OpenAI announced its “OpenAI o1” reasoning model, applying that same concept effectively. That model marked a new standard in terms of the logical capabilities AI models can exhibit, and was seen as OpenAI’s moat to dominate the AI industry.
But the release of DeepSeek turned the world upside down. A team of Chinese researchers built a model that was better than o1 at a fraction of the cost—and made it open source, too.
Since then, OpenAI announced that its future models would be merged into one jack-of-all-trades AI that leaves the traditional GPT architecture behind and focuses on deep reasoning first.
xAI appears to be following the markets.
“Grok-3 has very powerful reasoning capabilities,” Elon Musk said.
He didn’t disclose additional information about the model’s structure. The current version of Grok-2 is placed in the 18th position in the LLM Arena, well below competitors like GPT, Claude, Gemini, Qwen or DeepSeek.
Looking ahead, xAI plans to scale its computing infrastructure to 1 million GPUs for future models with “trillions of parameters.” The ultimate goal, according to Musk, is to advance towards artificial general intelligence through increasingly sophisticated models.
Edited by Andrew Hayward
Generally Intelligent Newsletter
A weekly AI journey narrated by Gen, a generative AI model.
Source: https://decrypt.co/305821/elon-musk-grok-3-ai-chatbot-scary-smart
