Preferences

postalrat parent
The 3090 also has 24gb of ram vs 128gb for the spark

Gracana
You'd have to be doing something where the unified memory is specifically necessary, and it's okay that it's slow. If all you want is to run large LLMs slowly, you can do that with split CPU/GPU inference using a normal desktop and a 3090, with the added benefit that a smaller model that fits in the 3090 is going to be blazing fast compared to the same model on the spark.

This item has no comments currently.