Hacker Neue

Preferences
behnamoh parent
Does this also ship only in x8 batches? I really liked MI300 and could afford one of them for my research, but they only come in batches of x8 in a server rack, so I decided to buy an RTX Pro 6000.
jiggawatts
Of course not.

AMD stubbornly refuses to recognise the huge numbers of low- or medium- budget researchers, hobbyists, and open source developers.

This ignorance of how software development is done has resulted in them losing out on a multi-trillion-dollar market.

It's incredible to me how obstinate certain segments of the industry (such as hardware design) can be.

rfv6723
These ppl are very loud online, but they don't make decisions for hyperscalers which are biggest spenders on AI chips.

AMD is doing just fine, Oracle just announced an AI cluster with up to 131,072 of AMD's new MI355X GPUs.

AMD needs to focus on bringing rack-scale mi400 as quickly as possible to market, rather than those hobbyists always find something to complain instead of spending money.

behnamoh OP
> these people

we're talking about the majority of open source developers (I'm one of them). if researchers don't get access to hardware X, they write their paper using hardware Y (Nvidia). AMD isn't doing fine because most low level research on AI is done purely on CUDA.

Certhas
So nVidia has a huge software lead because of open source developers like you? Or because people employed by nVidia write closed source high performance drivers and kernels? Or because the people employed by Meta and Google that wrote Torch and Tensorflow built it on nVidia?

I am really sympathetic to the complaints. It would just be incredibly useful to have competition and options further down the food chain. But the argument that this is a core strategic mistake makes no sense to me.

imtringued
You don't have to beg for good software quality from Nvidia. Everything else is built on top of that foundation. That's all there is to it.
regularfry
Nit: just writing Torch/TF isn't what made the difference. Having them adopted by a huge audience outside those orgs is, and that's bottlenecked on the hardware platform choice.

AMD has demonstrably not even acknowledged that they needed to play catch-up for a significant chunk of the last 20 years. The mistake isn't a recent one.

wisty
Also theano, keras ... nvidia made it easy to develop on so that's what people use.
motorest
> if researchers don't get access to hardware X, they write their paper using hardware Y (Nvidia).

There are plenty of research institutions that can easily spend >$250k on computational resources. Many routinely spend multiples of that volume.

They'll be fine.

impossiblefork
Many institutions don't.

Look at China. A couple of years ago people thought people in China weren't doing good AI research, but the thing is, there's good AI research from basically everywhere-- even South America. You can't assume that institutions can spend >$250k on computational resources.

Many can, but many can't.

qualifiedeephd
Serious researchers use HPC clusters not desktop workstations. Currently the biggest HPC cluster in North America has AMD GPUs. I think it'll be fine.
pjmlp
Before they became serious researchers they were once upon a time students learning with what their laptops were capable of.
uniclaude
Neither their revenue nor their market share in the space looks like just fine. What exactly in trailing the market for years is “just fine”?

AMD is very far behind, and their earnings are so low that even with a nonsensical pe ratio they’re still less than a tenth of nvidia. No, they are not doing anywhere near fine.

Are hobbyists the reason for this? I’m not sure. However, what AMD is doing is clearly failing.

regularfry
A big chunk of NVidia's current price is a reflection of lacking meaningful competition. So straight comparison isn't quite fair: if AMD started to do better, the gap would shrink from both ends.
creato
When you design software for N customers, where N is very small, and you expect to hold each customers' hand individually, the software is basically guaranteed to be hot garbage that doesn't generalize or actually work except in exactly the use cases you supported (there are exceptions to this, but it requires having exceptional software engineers and leaders that care about doing things correctly and not just closing the next ticket, and in my experience, they are extremely rare).

If you design software for N00000 customers, it can't be shit, because you can't hold the hands of that many people, it's just not possible. By intending to design software for a wide variety of users, it forces you to make your software not suck, or you'll drown in support requests that you cannot possibly handle.

wmf
Now imagine you don't have the resources to satisfy N00000 customers. What do you do?
exceptione
Start selling ice creams. :)

Honestly, if they "don't have the resources to satisfy N00000 customers", they better get them. That will teach them in the hard way to work differently.

almostgotcaught
> These ppl are very loud online, but they don't make decisions for hyperscalers which are biggest spenders on AI chips.

this guy gets it - absolutely no one cares about the hobby market because it's absolutely not how software development is done (nor is it how software is paid for).

pstuart
The hobby market should be considered as a pipeline to future customers. It doesn't mean AMD should drop everything and cater specifically to them, but they'd be foolish to ignore them altogether.
justahuman74
The hobby market is how you get 'market default' N years later
wmf
This probably does work for the first mover. It's not clear that it can work for the underdog.
almostgotcaught
citation please
6 More Comments →
gdiamos
startups and researchers are broke, just like Geoff Hinton in 2006 - https://blog.waqasrana.me/assets/papers/hinton2006.pdf
behnamoh OP
no we're not broke! we constantly write grants and receive funding from various sources. guess what hardware we recommend the University to purchase? it's 99.9% Nvidia, and sometimes Mac Studio just to play with MLX.
gdiamos
I mean broke compared to Meta or OpenAI.

AI research used to be fringe and not well funded.

Back in those days, 99.9% of hardware was Xeon.

ryao
It has gone through many boom and busy cycles. If you go far back enough, it was very well funded. In particular, I recall reading about the US government investing 1 to 2 billion dollars during the Cold War into AI research to translate Russian into English. It had some very impressive demos on preselected Russian texts that had justified the investments. However, it failed to yield results on arbitrary texts. The translation problem has only been mostly solved in recent years.
naveen99
There is no mass middle market… it’s volume or luxury… middle management is for taxes.
latchkey
It isn't 8x batches. It is 8 OAMs on a UBB. The UBB is what enables the 8x to communicate with each other over infinity fabric.

If you don't need 8, then that's exactly why we offer 1xMI300x VM's.

ryao
A number of people want to purchase their own hardware, not rent cloud hardware. I recently purchased a RTX PRO 6000 for the same reason, despite having the option of renting a B200 VM for $1.49 an hour from DeepInfra until the end of June.
latchkey
True, but as time goes on, it will become a wider and wider gap between what is deployed in DC’s and what you can run at home.

We see it now with 8x UBB and it will get worse with direct liquid cooling and larger power requirements. Mi300x is 700w. Mi355 is 1200w. Mi450 will be even more.

Certainly amd should make some consumer grade stuff, but they won’t stop on the enterprise side either. Your only option to get super computer level compute, will be to rent it.

ryao
The higher power requirements of datacenter GPUs are mostly from running hardware well past a reasonable point on the efficiency curve to eke slightly higher generational increases in performance. Nvidia for example has three versions of the RTX PRO 6000. One runs at 600W while the other two run at 300W. The main differences are the power target and the cooling solutions. If I recall correctly, the performance difference between them is less than 20%, despite a 50% reduction in power consumption, for what is effectively the same hardware. This can be confirmed by changing the power target of the 600W version to 300W and benchmarking the before and after. Plenty of people have made similar observations of AMD’s hardware.

That said, I am confident that Nvidia will continue serve those of us who want our own hardware.

latchkey
They all bin their chips after testing. 355x is 35% faster than 300x and uses almost 2x more power.

But 355 has fp4/6 added in, which until udma comes out, likely won’t get emulated.

It is fine if you dont need the features of newer hardware, but if you do… then desktop won’t help you.

3 More Comments →

This item has no comments currently.