> Grok 4 is [...] the most intelligent model so far
A bit too much praise for a model that's barely ahead of the competition in a subset of benchmarks...
> To be honest, this model not only competes with other AI models but also with humans, making it the first of its kind
I'm out
Rperry2174 · 5h ago
I keep seeing these Grok 4 intelligence claims, so I tried something very simple: "Animate a round robin tournament for 10 people."
Results:
Claude: ~10s, perfect working demo
ChatGPT: ~20s, solid solution
Grok 4: ~1000s, failed completely, gave me a truncated base64 blob
This wasn't some obscure edge case... it was basic data visualization that any decent model should handle. Yet somehow Grok 4 is "competing with humans" and has "99% tool accuracy"...
These benchmarks are either just wrong or measuring something completely divorced from practical utility imo...
ajd555 · 5h ago
Grok 4 has about 99% accuracy in picking the right tools and making tool calls with proper arguments almost every single time.
Where did this number come from? What is "the right tool"? I find this extremely subjective. As most engineers know, there is no right tool, but mostly a compromise where you pick the least worst tool and choose what risks you're willing to manage or not.
mdaniel · 5h ago
I believe in this context it means "tool" as in the MCP definition, e.g. "of the catalog of MCP integrations, it doesn't try to use the playright one to browse the web, it'd use the AWS docs one directly"
This is just my speculation, though, as I've never used Grok anything
ajd555 · 5h ago
Yeah, based on a previous comment, that makes sense. I am a little reassured that is what the author meant.
Byamarro · 5h ago
That's langchain terminology. LLMs usually are exposed to a set of tools. It's usually pretty obvious which are obvious, since there's only one tool that's even remotely associated with the task at hand.
ajd555 · 5h ago
Thanks for the info. This makes the article slightly less intolerable!
patrickhogan1 · 5h ago
On your intelligence graph where it shows Grok 4 and OpenAI o4-mini as comparable (and among the highest intelligence rated models), it doesn’t have OpenAI o3 or o3-pro.
Yet all of my tests show o3 blows o4-mini out of the water.
What are you classifying as intelligence?
4b11b4 · 4h ago
This article seems like pure garbage
knes · 5h ago
Didn't the tldr of grok 4 was their over tuned for bencmhark results but in day to day tasks . It's actual not better than o3 / gpt5
OrvalWintermute · 5h ago
grok4 is tortiously slow compared to all the other LLMs I use :(
amitksingh1490 · 5h ago
Ya, even I feel its slow, Thats why I use it only for architecture planning and finding complex issue
aitacobell · 5h ago
> To be honest, this model not only competes with other AI models but also with humans, making it the first of its kind
Is this a joke
kolektiv · 5h ago
I can't take anything seriously with phrases like "it has not yet achieved AGI, but it is one leap forward in the race to AGI" - based on what? Nobody knows whether LLMs are a viable approach to AGI, nobody really agrees on what AGI is, hell, people don't really agree on what "I" is.
This is just not even science at all at this point, we're just into solid cargo cult.
CamperBob2 · 5h ago
If the answer involves giving even more money to Elon Musk, you asked the wrong question.
A bit too much praise for a model that's barely ahead of the competition in a subset of benchmarks...
> To be honest, this model not only competes with other AI models but also with humans, making it the first of its kind
I'm out
Results: Claude: ~10s, perfect working demo ChatGPT: ~20s, solid solution Grok 4: ~1000s, failed completely, gave me a truncated base64 blob
This wasn't some obscure edge case... it was basic data visualization that any decent model should handle. Yet somehow Grok 4 is "competing with humans" and has "99% tool accuracy"...
I don't buy it..
links: Claude: https://claude.ai/share/7a413a6a-5c01-44a1-aaed-8b237e5e9e94 Chatgpt: https://chatgpt.com/canvas/shared/687a9f9d4304819187ac7d98d3... Grok 4: https://grok.com/share/c2hhcmQtMw%3D%3D_20b61291-e1bb-45e5-a...
These benchmarks are either just wrong or measuring something completely divorced from practical utility imo...
Where did this number come from? What is "the right tool"? I find this extremely subjective. As most engineers know, there is no right tool, but mostly a compromise where you pick the least worst tool and choose what risks you're willing to manage or not.
This is just my speculation, though, as I've never used Grok anything
Yet all of my tests show o3 blows o4-mini out of the water.
What are you classifying as intelligence?
Is this a joke
This is just not even science at all at this point, we're just into solid cargo cult.