For the last year, I've tried all sorts of models both as hosted services and running locally with llama.cpp or ollama. I've used both the continue.dev vscode extension and cursor more recently.
The results have been frustrating at best. The user interface of the tools is just awful. The output of any models from Deepseek to quen to Claude to whatever other model is mediocre to useless. I literally highlight some code that includes comments about what I need and I even include long explicit descriptions etc in the prompts and it's just unrelated garbage out every time.
The most useful thing has just been ChatGPT when there's something I need to learn about. Rubber ducking basically. It's alright at very simple coding questions or asking about obscure database questions I might have, but beyond that it's useless. Gotta keep the context window short, or it starts going off the rails every single time.
They're still useful tools for exploring new disciplines, but if you're say a programmer and you think ChatGPT or DeepSeek is good at programming, that's a good sign you need to start improving.
https://www.youtube.com/watch?v=VHHT6W-N0ak&pp=ygUMTGludXMgQ...
I do believe the benefit decreases the more senior or familiar the work is but there is still a noticeable benefit and I think it largely depends on the velocity and life cycle of the product. I think you get less benefit the slower the velocity or the more mature of a product. To deny it like in your post is simply being an intellectual minimalist.
You make a polite but still ad hominem "attack" about me instead of addressing my points with demonstrations of evidence.
Make a video or blog article actually showing how your use of LLMs in coding is making you more productive. Show what it's doing to help you that has a multiplier effect on your productivity.
I don't think it deserves a video or blog, like I already said the multiple posts that have made HN front page have covered it well. - Autocomplete saves me keystrokes usually - Features like Cursor's composer/agent allow me to outsource junior level changes to the code base. I can copy/paste my requirements and it gives me the diffs of the changes when its done. Its often at a junior level or better and tackles multi-file changes. I usually kick this off and go make other changes to the code base.
Now like I have said before, this depends a lot on the velocity of the team and the maturity of the code base. I think more mature products you will have less benefit on feature implementation and most likely more opportunity in the test writing capabilities. Likewise, teams with a slower cadence, thinking a bluechip software company compared to a startup, are not going to get as much benefit either.
Instead of being so aggressive, simply say why it does not work for you. These tools strive in web dev which you may not be involved in!
they don't let me walk at the pace of a SUV.
AI is like the good shoes. they help, and make many tasks a bit easier. but they can't make me into an SUV.
and if they can, then no programmers will have jobs. which is the end-state of this whole LLM thing as far as I can tell.
This comment is like saying, “This diet didn’t work for me” without providing any details about your health circumstances. What’s your weight? Age? Level of activity?
In this context: What language are you working in? What frameworks are you using? What’s the nature of your project? How legacy is your codebase? How big is the codebase?
If we all outline these factors plus our experiences with these tools, then perhaps we can collectively learn about the circumstances when they work or don’t work. And then maybe we can make them better for the circumstances where they’re currently weak.
Whereas measuring productivity and usefulness is way more opaque.
Many simple software systems are highly productive for their companies.
From what I have seen the people that have the most success have AI building something from scratch using well known tooling (read: old tooling).
The problem is that doesn't immediately help most people. We are all stuck in crap jobs with massive, crusty code bases. Its hard for AI because its hard for everyone.
I might go back and tweak some stuff, add some extra tags and what not, but often its pretty good at doing what I ask.
Sometimes its suggestions aren't what I was really wanting to do in my codebase, a handful of times it has made up methods or parameters of even well-known libraries. But usually, its suggestions are better than a basic IntelliSense-style autocomplete at least in my experiences.
I haven't used many of the other developer assistant plugins like say GitHub Copilot. I couldn't really say which is better or worse. But I do think using Q Developer has made me faster in many tasks.
I wouldn't expect a tool that doesn't have access to the context of my editor and the files I have open to be very useful for actually coding. There's a lot of context to understand in even a basic application. If you're just asking a locally running app in ollama "give me a method to do x", don't be surprised if it doesn't know everything else happening in your app. Maybe it'll give you a halfway decent example of doing something, but devoid of how it actually plugs in to whatever you're making it might be entirely worthless.
Maybe whatever language you are coding it or whatever project you are working on is not a good fit? It is an equally perplexing situation for myself when I hear anecdotes like yours which don't align with my experience. The fact that you say everything is garbage calls into question either how you are using the tool or something else.
I can reliably use cursor's composer to reference a couple files, give a bullet list of what we are trying to do and point it to one of the better models and the output is junior engineer level or better output. When I say junior, I mean a junior who has experience with the codebase.
Myself, I do find it quite useful in a few respects. First and foremost, as a "better Google/StackOverflow." If something's not working, I can describe my exact scenario and usually get pointed in the right direction. Sometimes the LLM just wastes my time by very confidently telling me some function/library that solves my exact problem exists when in fact it doesn't.
Second, IntelliJ's local LLM is sort of a smarter autocomplete. It makes some outright wrong suggestions, but when there's areas where I have to do a lot of repetitive tasks that follow a simple pattern (like for instance, mapping fields from one type of object to another), it does a pretty good job of making correct suggestions. I definitely appreciate it but it's certainly not doing things like writing a significant portion of code in my style.
And this is coming from someone who uses LLMs daily at the subscription, API (vscode and 3 nextjs apps) and local level. I have a custom langchain stack, prompt repo, you name it. And regardless of how little or how much I use what I have, or what soup de jour prompt or process (from Keep it simple to Prompt enhancers) I can’t say it’s made a meaningful difference in my life. Even with all of the customization and planning.
It’s a great search engine though.
I swear half the time when I use it to look up the nuances of system API stuff, it's replaying forum, mailing list or Stackoverflow conversations that Google ought to be able to find but somehow can't.
It's very difficult to measure productivity of most people, certainly most people in office jobs, so while you can have a gut feeling that you're doing better, it's no more measurable than pre-AI individual productivity measurement was
Maybe you should stop using the Ai slop tools that don't work?
We have been living in a fake economy for quite some time where money is printed and distributed to the "tech" sector. Which isn't really "tech", but mostly entertainment (YouTube, Netflix, Facebook, ...).
Growth of the economy means nothing. The money that has been printed goes to shareholders. What the common man gets is inflation and job losses.
If you want to grow the real economy, build houses and reduce the cost of living.
Yes, I wonder why it is so hard for Western countries to understand that there's no future in a place where housing is more expensive than your average salary. If may look cool for a few years until most people have left or are living on the streets.
don't get me wrong, everyone want cheaper housing but not their house
https://www.cnbc.com/2023/06/26/microsoft-ceo-nadella-said-r...
Some regions or sectors might have experienced higher growth spurts, but the main point stands: the overall economic growth was quite low by modern standards - even though I don't think GDP numbers alone adequately describe the huge societal changes of such sustained growth compared to agrarian cultures before the Industrial Revolution.
[0] https://web.archive.org/web/20071127032512/http://minneapoli... [1] https://www.bankofengland.co.uk/explainers/how-has-growth-ch... [2] https://academic.oup.com/ereh/article/21/2/141/3044162
All of the things you site are available via search engines, or better handled with expertise so you know how much of the response is nonsense.
Every time I use AI, it's a time waste.
On balance AI gets more things wrong than the best humans and fewer things wrong than average humans.
I get a lot of correct answers from llms, but sometimes they make shit up. Most of the time, it's some function in a library that doesn't actually exist. Sometimes even the wrong answers are useful because they tell me where to look in the reference docs. Ask it to search the web and cite sources, makes it easier to verify the answer.
I don't appreciate what's going on with AI art and AI generated slop, but the idea that they aren't a useful tool is just wild to me.
Because improving the productivity of every employee by 10% does not translate to the company being 10% more productive.
Processes and systems exist precisely to slow employees down so that they comply with regulations, best practices etc rather than move fast and break things.
And from experience with a few enterprise LLM projects now they are a waste of time. Because the money/effort to fix up the decades of bad source data far exceeds the ROI.
You will definitely see them used in chat bots and replacing customer service people though.
during the industrial revolution(steam/electricity/internet), the world was growing, there're trains, cars, netflix
bussiness grown with productivity growing, even so, we lived through 2 world wars and dozens of economic crisis
but now is very different, when you repair the tank with LLM's help, when the labour value of repairers is decreased, there's no addition value are produced
there's a very simple thought experiment abt the result of productivity growing alone:
let's assume robotics become to a extremely high level, everything humen work can be reduced to 1/100 with help of robots, what will happen next?
Prior to computerization and databases, millions of people were required for filing, typing, and physically transporting messages and information. All of those jobs, entire fields of work were deleted by computerization.
We work 35 hour years instead of 35 hour weeks?
That seems like a low bar because it already is- it's just not equally distributed yet.
My own productivity has grown far more than 10% thanks to AI, and I don't just mean in terms of dev. It reads my bloodwork results, speeds up my ability to repair a leak in my toilet tank, writes a concise "no I won't lend you money; I barely know you" message... you name it.
Normally all of those things would take much longer and I'd get worse results on my own.
If that's what I can do at the personal level, then surely 10% is an easily-achievable improvement at the enterprise level.