Elon Musk’s AI venture, xAI, has recently unveiled the early preview of its latest model, Grok 2. This new model has outperformed other prominent AI systems such as Claude, Gemini, and ChatGPT. Despite Grok-1.5 facing criticism, Grok-2 has made significant strides, showcasing impressive results on the LMSYS leaderboard. Along with Grok-2, a smaller variant known as Grok-2 mini has also been released, providing users with more options in AI technology.
xAI claims that Grok-2 has experienced considerable enhancements in crucial areas, including reasoning and instruction following. The model has achieved remarkable scores in traditional AI benchmarking tests, scoring 87.5% in MMLU and 88.4% in HumanEval. Notably, the MMLU score utilized a 0-shot Chain of Thought (CoT) methodology, highlighting Grok-2's advanced capabilities.
In testing conducted on LMSYS, Grok-2 was identified by the name "sus-column-r." Garnering approximately 12,000 votes, it secured the third position overall, just behind ChatGPT-4o-latest and Gemini-1.5-Pro-Experimental. Impressively, Grok-2 outperformed models like GPT-4o-mini and Claude 3.5 Sonnet, establishing its reputation within the AI community.
In coding and math challenges, Grok-2 has claimed the second spot, while it ranks fourth for tackling hard prompts. xAI has hinted that a multimodal version of Grok-2 is on the horizon, although details regarding the parameter size for both models have not yet been disclosed. Users can begin utilizing the new Grok-2 model on x.com, and developers can access the API to integrate its capabilities into their applications.
