So those two QSFP ports can be configured for 400GbE IB and/or RoCE?
Also is it fair to say that this can run a desktop considering it has an HDMI port?
dragonwriter
Yes, the whole point is that it is a compact desktop for AI applications, so, yes, it can run a desktop.
moondev
Nvidia DGX and HGX servers packed NVME, 8 H100 GPUs and 8 Infiniband cards don't run a desktop. They don't even have an HDMI port.
andrewSC
Am I missing something here or is inference going to be painful given the "low" memory bandwidth compared to, say, HBM2E?
sliken
273GB/sec with good FP4 performance should be fine for developers playing with inference. This isn't the kind of thing that you'd use for inference workloads supporting millions of users.
I'd like to see a inference benchmark vs the strix halo, which has better memory bandwidth and costs 2/3rds as much.
tanelpoderOP
I guess since these devices aren't meant for production throughput, but rather about having enough RAM for local experimentation with large enough models, it's an ok tradeoff at this price point...
dragonwriter
Have you seen anything with 128GB of HBM2E at anywhere near the DGX Spark's $3,000 price point?
karmakaze
The AMD Radeon VII has 16GB HBM2 and sold for $700 in 2019. I don't know how that would translate to today's HBM2E's pricing like if its price change follows that of GDDR's.
andrewSC
I honestly can't say I have however, that doesn't mean it couldn't physically exist/happen? Perhaps a "little" more cost but I'd be willing to bet people would gladly pay the premium for such a device. I'm also very curious to know what the BOM for an A100 actually is as well as HBM2E per GB.
Heard coming from the Taiwan-infused PC clone manufacturer industry https://youtu.be/wIE5wwvicew
Meanwhile, at whatever planet OpenAI lives on https://youtu.be/615lNmMiOSQ
https://resources.nvidia.com/en-us-accelerated-networking-re...
So those two QSFP ports can be configured for 400GbE IB and/or RoCE?
Also is it fair to say that this can run a desktop considering it has an HDMI port?
I'd like to see a inference benchmark vs the strix halo, which has better memory bandwidth and costs 2/3rds as much.