Hacker Neue

Preferences
kgeist parent
>I challenge everyone to find another local GUI with that privacy

Llama.cpp's built-in web UI.

trilogic
This is from webui website docs: Once saved, Open WebUI will begin using your local Llama.cpp server as a backend! So you see Llama server not CLI. That´s a big flag there. I repeat no app in the whole world takes seriously privacy like HugstonOne. This is not advertisement, I am just making a point.
kgeist OP
I'm not sure what you're talking about. Llama.cpp is an inference server which runs LLMs locally. It has a built-in web UI. You can't get more private than the inference server itself.

I tried downloading your app, and it's a whopping 500 MB. What takes up the most disk space? The llama-server binary with the built-in web UI is like a couple MBs.

trilogic
With all respect you do seem to not understand much of how privacy works. Llama-server is working in Http. And yes the app is a bit heavy as is loading llm models using llama.cpp cli and multimodal which in itself are quite heavy, also just the dlls for cpu/gpu are huge, (just the one for the nvidial gpu is 500mb if I don't go wrong).
kgeist OP
Unless you expose random ports on the local machine to the Internet, running apps on localhost is pretty safe. Llama-server's UI stores conversations in the browser's localStorage so it's not retrievable even if you expose your port. To me, downloading 500 MB from some random site feels far less safe :)

>the app is a bit heavy as is loading llm models using llama.cpp cli

So it adds an unnecessary overhead of reloading all the weights to VRAM on each message? On some larger models it can take up to a minute. Or you somehow stream input/output from an attached CLI process without restarting it?

rcakebread
Says the guy with a link to a broken privacy policy on their website.
trilogic
I accept critics, and I thank you for it. It will be fixed ASAP.
giantrobot
> With all respect you do seem to not understand much of how privacy works. Llama-server is working in Http.

What in the world are you trying to say here? llama.cpp can run completely locally and web access can be limited to localhost only. That's entirely private and offline (after downloading a model). I can't tell if you're spreading FUD about llama.cpp or are just generally misinformed about how it works. You certainly have some motivated reasoning trying to promote your app which makes your replies seem very disingenuous.

trilogic
I am not here to teach cybersecurity Tcp/ip protocols or ML. HTTP = HyperText Transfer Protocol The standard protocol for transferring data over the web. CLI = Command-Line Interface. Try again after endless nights of informatic work please.
2 More Comments →

This item has no comments currently.