Hacker Neue

Preferences
no_wizard parent
A large contention of this essay (which I’m assuming the talk is based on or is transcribed from depending on order) I do think that open source models will eventually catch up to closed source ones, or at least be “good enough” and I also think you can already see how LLMs are augmenting knowledge work.

I don’t think it’s the 4th wave of pioneering a new dawn of civilization but it’s clear LLMs will remain useful when applied correctly.

bix6
Why would open source outpace? Isn’t there way more money in the closed source ones and therefore more incentive to work on them?
no_wizard OP
I didn’t say outpace, but I do believe the collective nature of open source will allow it to catch up, much like it did with browser tech, and at which point you’ll see a shift of resources toward that by major companies. It’s a collective works thing. I think it also is attractive to work on in open source, much like Linux or web browsers (hence the comparison to one) and that will also help it along over time.

I stick by my general thesis that OSS will eventually catch up or the gap will be so small only frontier applications will benefit from using the most advanced models

oblio
They didn't say "outpace", they said "catch up to good enough levels".
umeshunni
> I do think that open source models will eventually catch up to closed source ones

It felt like that was the direction for a while, but in the last year or so, the gap seems to have widened. I'm curious whether this is my perception or validated by some metric.

msgodel
Already today I can use aider with qwen3 for free but have to pay per token to use it with any of the commercial models. The flexibility is worth the lower performance.
QRY
Do you have anything to share on that workflow? I've been trying to get a local-first AI thing going, could use your insights!
msgodel
It's super easy. I already had llama.cpp/llama-server set up for a bunch of other stuff and actually had my own homebrew RAG dialog engine, aider is just way better.

One crazy thing is that since I keep all my PIM data in git in flat text I now have essentially "siri for Linux" too if I want it. It's a great example of what Karpathy was talking about where improvements in the ML model have consumed the older decision trees and coded integrations.

I'd highly recommend /nothink in the system prompt. Qwen3 is not good at reasoning and tends to get stuck in loops until it fills up its context window.

My current config is qwen2.5-coder-0.5b for my editor plugin and qwen3-8b for interactive chat and aider. I use nibble quants for everything. 0.5b is not enough for something like aider, 8b is too much for interactive editing. I'd also recommend shrinking the ring context in the neovim plugin if you use that since the default is 32k tokens which takes forever and generates a ton of heat.

QRY
I really appreciate you going into such technical specificity, thank you! I'll have to steal that siri for linux setup, that sounds awesome. Exploring ways to make use of compute people have lying around to do useful things without the vendor dependencies. But I'm relatively new to the AI scene, so your input really boosts my learning speed, thank you again!
no_wizard OP
This was how early browsers felt too, the open source browser engines were slower at adapting than the ones developed by Netscape and Microsoft, but eventually it all reversed and open source excelled past the closed source software.

Another way to put it, is that over time you see this, it usually takes a little while for open source projects to catch up, but once they do they gain traction quite quickly over the closed source counter parts.

umeshunni
That's a good analogy. Makes a lot of sense. The only caveat I see is that there is a lot of context locked up in proprietary data sets (e.g. YT, books, podcasts) and I'm not sure how OSS models get access to that.
tayo42
Those were way simpler projects in the beginning when that happened. Like do you think a new browser would catch up today chrome now?
no_wizard OP
The tech behind LLMs has been open source for a very long time. Look at DeepSeek and LLAMA for example. They aren’t yet as capable as say Gemini but they aren’t “miles behind” either, especially if you know how to tune the models to be purpose built[0].

The time horizons will be different as they always are, but I believe it will happen eventually.

I’d also argue that browsers got complicated pretty fast, long cry from libhtml in a few short years.

[0]: of which I contend most useful applications of this technology will not be the generalized ChatGPT interface but specialized highly tuned models that don’t need the scope of a generalized querying

This item has no comments currently.