GPT-5 models have been the most useless models out of any model released this year despite being SOTA, and it because it slow as fuck.
Ideally I would have both fast and SOTA; if I would have to pick one I’d go with SOTA.
There a report by OpenRouter on what folks tend to pay for it; it generally is SOTA in the coding domain. Folks are still paying a premium for them today.
There is a question if there is a bar where coding models are “good enough”; for myself I always want smarter / SOTA.
I think the bar for when coding models are "good enough" will be a tradeoff between performance and price. I could be using Cerebras Code and saving $50 a month, but Opus 4.5 is fast enough and I value the piece-of-mind I have knowing it's quality is higher than Cerebras' open source models to spend the extra money. It might take a while for this gap to close, and what is considered "good enough" will be different for every developer, but certainly this gap cannot exist forever.
Hard disagree. There are very few scenarios where I'd pick speed (quantity) over intelligence (quality) for anything remotely to do with building systems.
Implicit in your claim are specific assumptions about how expensive/untenable it is to build systemic guardrails and human feedback, and specific cost/benefit ratio of approximate goal attainment instead of perfect goal attainment. Rest assured that there is a whole portfolio of situations where different design points make most sense.
1. law of diminishing returns - AI is already much, much faster at many tasks than humans, especially at spitting out text, so becoming even faster doesn’t always make that much of a difference. 2. theory of constraints - throughput of a system is mostly limited by the „weakest link“ or slowest part, which might not be the LLM, but some human-in-the-loop, which might be reduced only by smarter AI, not by faster AI. 3. Intelligence is an emergent property of a system, not a property of its parts - with other words: intelligent behaviour is created through interactions. More powerful LLMs enable new levels of interaction that are just not available with less capable models. You don’t want to bring a knife, not even the quickest one in town, to a massive war of nukes.
The current SOTA models are impressive but still far from what I’d consider good enough to not be a constant exercise in frustration. When the SOTA models still have a long way to go, the open weights models have an even further gap distance to catch up.
I'd like more speed but prefer more quality than more speed.
We should be glad that the foundation model companies are stuck running on treadmills. Runaway success would be bad for everyone else in the market.
Let them sweat.
Reason is: while these models look promising in benchmarks and seem very capable at an affordable price, I *strongly* felt that OpenAI models perform better most of the times. I had to cleanup Gemini mess or Claude mess after vibe coding too much. OpenAI models are just much more reliable with large scale tasks, organizing, chomping tasks one by one etc. That takes its time but the results are 100% worth it.
Could you elaborate? How is this done and what does this mean?
I also haven’t seen any hard data on how much they do use distillation like techniques. They for sure used a bunch of synthetic generated data to get better at reasoning, something that is now commonplace.
The opposite, I think.
Why do you think that local models are a direct threat to Nvidia?
Why would Nvidia let a few of their large customers have more leverage by not diversifying to consumers? Openai decided to eat into Nvidia's manufacturing supply by buying DRAM; that's concretely threatening behavior from one of Nvidia's larger customers.
If Groq sells technology that allows for local models to be used better, why would that /not/ be a profit source for Nvidia to incorporate? Nvidia owes a lot of their success on the consumer market. This is a pattern in the history of computer tech development. Intel forgot this. AMD knows this. See where everyone is now.
Besides, there are going to be more Groqs in the future. Is it worth spending ~20B for each of them to continue to choke-hold the consumer market? Nvidia can afford to look further.
It'd be a lot harder to assume good faith if Openai ended up buying Groq. Maybe Nvidia knows this.
And likely some of them are going to be in countries that won't let them sell out to Nvidia.
Almost all open source models are trained and mostly run on NVIDIA hardware.
Open source is great for NVIDIA. They want more open source, not less.
Commoditize your complement is business 101.
I would guess their interconnect technology is what NVIDIA wants. You need something like 75 adapters for an 8b parameter model they had some really interesting tech to make the accelerator to accelerator communication work and scale. They were able to do that well before nvl 72 and they scale to hundreds of adapters since large models require more adapters still.
We will know in a few months.
That's a non-charitable interpretation of what happened. The are not "spending $20 billion to handicap Groq". They are handing Groq $20 billion to do whatever they want with it. Groq can take this money and build more chips, do more R&D, hire more people. $20 billion is truly a lot of money. It's quite hard to "handicap" someone by giving them $20 billion.
> Groq added that it will continue as an “independent company,” led by finance chief Simon Edwards as CEO.
Inference hardware company
Your words.
Because it's very good tech for inference?
It doesn't even do training.
And most inference providers for Open Source models use NVIDIA eg Fireworks, Basten, TogetherAI etc.
Most NVIDIA sales go to training clusters. That is changing but it'd be an interesting strategy to differentiate the training and inference lines.
If anything more companies in making models business the higher NVIDIA chip demand will be, till we get some proper competition at least. We badly need some open CUDA equivalent so moving off to competition isn't a problem
Yeah I'm disappointed by this, this is clearly to move them out of the market. Still, that leaves a vacuum for someone else to fill. I was extremely impressed by Groq last I messed about with it, the inference speed was bonkers.
That said, it's completely anti-competitive. Nvidia could design a inference chip themselves, but instead the are locking down one of the only real independents. But... Nobody was saying Groq was making any real money. This might just be a rescue mission.
A. The inevitable breakdown of their massive head start with CUDA and data center hardware. A serious competitor at real scale.
B. Anything that'll cool off the massive data center buildouts that are fueling them.
Seems clear that locking up a major potential competitor especially the minds behind it solves for A. And their ongoing machinations with circular funding of companies funding data centers is all about B - keeping the momentum before it fizzles.
The open source economy relies on the wisdom of crowds. But that implies and equal access to experimentation platforms. The democratization of PC and consumer hardware brings the previous open source era that we all love, I am afraid the tech mongols had identified the chokehold of LLM ecosystem and found ways to successfully monopolized it
> Maybe I'm way off on this.
If by open source, you mean downloadable from huggingface and SOTA you mean opus 4.5, yes you are way off.
The more competition, the more shovels they sell.
It's like saying that Intel would've benefited if only Dell and few others sold servers because they brought in multiple billions per year.
Look at Nvidia nemotron series. They hav become a leading open source training lab themselves and they’re releasing the best training data, training tooling, and models at this point.
It quite obvious that open source models are catching up to closed source models very fast they about 3-4 months behind right now, and yeah they are trained on Nvidia chips, but as the open source models become more usable, and closer to closed source models they will eat into Nvidia profit as these companies aren't spending tens of billion dollars on chips to train and run inference. These are smaller models trained on fewer GPUs and they are performing as good as the pervious OpenAI and Anthropic models.
So obviously open source models are a direct threat to Nvidia, and they only thing open source models struggle at is scaling inference and this is where Groq and Cerberus come into the picture as they provide the fastest inference for open source models that make them even more usable than SOTA models.
Maybe I'm way off on this.