Hacker Neue

Preferences
umeshunni parent
> I do think that open source models will eventually catch up to closed source ones

It felt like that was the direction for a while, but in the last year or so, the gap seems to have widened. I'm curious whether this is my perception or validated by some metric.

msgodel
Already today I can use aider with qwen3 for free but have to pay per token to use it with any of the commercial models. The flexibility is worth the lower performance.
QRY
Do you have anything to share on that workflow? I've been trying to get a local-first AI thing going, could use your insights!
msgodel
It's super easy. I already had llama.cpp/llama-server set up for a bunch of other stuff and actually had my own homebrew RAG dialog engine, aider is just way better.

One crazy thing is that since I keep all my PIM data in git in flat text I now have essentially "siri for Linux" too if I want it. It's a great example of what Karpathy was talking about where improvements in the ML model have consumed the older decision trees and coded integrations.

I'd highly recommend /nothink in the system prompt. Qwen3 is not good at reasoning and tends to get stuck in loops until it fills up its context window.

My current config is qwen2.5-coder-0.5b for my editor plugin and qwen3-8b for interactive chat and aider. I use nibble quants for everything. 0.5b is not enough for something like aider, 8b is too much for interactive editing. I'd also recommend shrinking the ring context in the neovim plugin if you use that since the default is 32k tokens which takes forever and generates a ton of heat.

QRY
I really appreciate you going into such technical specificity, thank you! I'll have to steal that siri for linux setup, that sounds awesome. Exploring ways to make use of compute people have lying around to do useful things without the vendor dependencies. But I'm relatively new to the AI scene, so your input really boosts my learning speed, thank you again!
no_wizard
This was how early browsers felt too, the open source browser engines were slower at adapting than the ones developed by Netscape and Microsoft, but eventually it all reversed and open source excelled past the closed source software.

Another way to put it, is that over time you see this, it usually takes a little while for open source projects to catch up, but once they do they gain traction quite quickly over the closed source counter parts.

umeshunni OP
That's a good analogy. Makes a lot of sense. The only caveat I see is that there is a lot of context locked up in proprietary data sets (e.g. YT, books, podcasts) and I'm not sure how OSS models get access to that.
tayo42
Those were way simpler projects in the beginning when that happened. Like do you think a new browser would catch up today chrome now?
no_wizard
The tech behind LLMs has been open source for a very long time. Look at DeepSeek and LLAMA for example. They aren’t yet as capable as say Gemini but they aren’t “miles behind” either, especially if you know how to tune the models to be purpose built[0].

The time horizons will be different as they always are, but I believe it will happen eventually.

I’d also argue that browsers got complicated pretty fast, long cry from libhtml in a few short years.

[0]: of which I contend most useful applications of this technology will not be the generalized ChatGPT interface but specialized highly tuned models that don’t need the scope of a generalized querying

This item has no comments currently.