kingkongjaffa parent
Can it really run local LLMs at any decent size? I find it hard to believe for the price it can run anything more than 7b or 8b models slowly.
according to [0] it looks like the "Umbrel Home" device they sell (with 16GB RAM and an N150 CPU) can run a 7B model at 2.7 tokens/sec, or a 13B model at 1.5 t/s.
especially when they seem to be aiming for a not-terribly-technical market segment, there seems to be a pretty big mismatch between that performance and their website claims:
> The most transformative technology of our generation shouldn't be confined to corporate data centers. Umbrel Home democratizes access to AI, allowing you to run powerful models on a device you own and control.
0: https://github.com/getumbrel/llama-gpt?tab=readme-ov-file#be...
Wow that's wild that they advertise "Run Deepseek-R1 locally" when the screenshot in the app store refers to "DeepSeek-R1-0528-Qwen3-8B"
It’s all subjective. Personally I think it would border on useless for local inference but maybe some people are happy with low quality models at slow speeds.