Hacker Neue

Preferences
ryao parent
A number of people want to purchase their own hardware, not rent cloud hardware. I recently purchased a RTX PRO 6000 for the same reason, despite having the option of renting a B200 VM for $1.49 an hour from DeepInfra until the end of June.
latchkey
True, but as time goes on, it will become a wider and wider gap between what is deployed in DC’s and what you can run at home.

We see it now with 8x UBB and it will get worse with direct liquid cooling and larger power requirements. Mi300x is 700w. Mi355 is 1200w. Mi450 will be even more.

Certainly amd should make some consumer grade stuff, but they won’t stop on the enterprise side either. Your only option to get super computer level compute, will be to rent it.

ryao OP
The higher power requirements of datacenter GPUs are mostly from running hardware well past a reasonable point on the efficiency curve to eke slightly higher generational increases in performance. Nvidia for example has three versions of the RTX PRO 6000. One runs at 600W while the other two run at 300W. The main differences are the power target and the cooling solutions. If I recall correctly, the performance difference between them is less than 20%, despite a 50% reduction in power consumption, for what is effectively the same hardware. This can be confirmed by changing the power target of the 600W version to 300W and benchmarking the before and after. Plenty of people have made similar observations of AMD’s hardware.

That said, I am confident that Nvidia will continue serve those of us who want our own hardware.

latchkey
They all bin their chips after testing. 355x is 35% faster than 300x and uses almost 2x more power.

But 355 has fp4/6 added in, which until udma comes out, likely won’t get emulated.

It is fine if you dont need the features of newer hardware, but if you do… then desktop won’t help you.

ryao OP
Your comment does not make much sense to me. The 355x and 300x are two different chips, not binned versions of one another. The RTX PRO 6000 has fp4/fp6 support too, so there is no need to use datacenter exclusive hardware to get that.
latchkey
I wasn’t referring to them being binned to each other. Oh nice, I didn’t know that about the 6k.
ryao OP
The consumer 50 series GPUs also have FP4/FP6 support. Nvidia talks about it a little here:

https://images.nvidia.com/aem-dam/Solutions/geforce/blackwel...

FP4 is given a full page advertising it while FP6 support in “RTX Blackwell” is a footnote.

This item has no comments currently.