I still happen to trust Apple with my cloud data, with their secure enclave. To that end, an Apple solution where my history/context is kept in my cloud account, perhaps even a future custom Apple chip that could run some measure of a local LLM.... This "aiPhone" might be the mainstream solution that non-technicals will enjoy.
> there is almost zero demand for private LLM right now.
you need some really expensive hardware to run a local LLM, most of which is unavailable to the average user. The demand might just simply be hidden as these users do not know nor want to expend the resources for it.
but i have hope that the hardware costs will come down eventually, enough that it reveals the demand for local LLM.
After all, i prefer my private questions to an LLM not be ever revealed.
> I think that services will win (as they always do)
We can have services but also private history/contexts. Those can be "local" (and encrypted).
Non-technicals don't know how LLMs work, and, more importantly, don't care about their privacy.
For a technology to be widely used, by definition, you need to make it appealing to the masses, and there is almost zero demand for private LLM right now.
That's why I don't think that local llms will win. There are narrow use cases where regulations can force local llm usage (like for medical stuff), but overall I think that services will win (as they always do)