I've found Claude's usefulness is highly variable, though somewhat predictable. It can write `jq` filters flawlessly every time, whereas I would normally spend 30 minutes scanning docs because nobody memorizes `jq` syntax. And it can comb through server logs in every pod of my k8s clusters extremely fast. But it often struggles making quality code changes in a large codebase, or writing good documentation that isn't just an English translation of the code it's documenting.
I think the reason for this is because these systems get all their coding and design expertise from training, and while there is lots of training data available for small scale software (individual functions, small projects), there is much less for large projects (mostly commercial and private, aside from a few large open source projects).
Designing large software systems, both to meet initial requirements, and to be maintainable and extensible over time, is a different skill than writing small software projects, which is why design of these systems is done by senior developers and systems architects. It's perhaps a bit like the difference between designing a city and designing a single building - there are different considerations and decisions being made. A city is not just a big building, or a collection of buildings, and large software system is not just a large function or collection of functions.
But I also have it document and summarise its own work.
Could you share some of your prompts or CLAUDE.md? I'm still learning what works.
take care
> I'm not an expert in this language or this project but I used AI to add a feature and I think its pretty good. Do you want to use it?
I find myself writing these and bumping into others doing the same thing. It's exciting, projects that were stagnant are getting new attention.
I understand that a maintainer may not want to take responsibility for new features of this sort, but its easier than ever to fork the project and merge them yourself.
I noticed this most recently in https://github.com/andyk/ht/pulls which has two open (one draft) PRs of that sort, plus several closed ones.
Issues that have been stale for years are getting traction, and if you look at the commit messages, it's AI tooling doing the work.
People feel more capable to attempt contributions which they'd otherwise have to wait for a specialist for. We do need to be careful not to overwhelm the specialists with such things, as some of them are of low quality, but on the whole it's a really good thing.
If you're not noticing it, I suggests hanging out in places where people actually share code, rather than here where we often instead brag about unshared code.
That does not mean that they are more capable, and that's the problem.
> We do need to be careful not to overwhelm the specialists with such things, as some of them are of low quality, but on the whole it's a really good thing.
That's not what the specialists who have to deal with this slop say. There have been articles about this discussed here already.
Which is also fine and great and very useful and I am also making those, but it probably does not generalize to projects that require higher quality standards and actual maintenance.
Solving the problems of the business that isn’t a software business.
I worked at an insurance company a decade ago and the majority of their software was ancient. There were a couple desktops in the datacenter lab running Windows NT for something that had never been ported. They'd spent the past decade trying to get off the mainframe and a majority of requests still hit the mainframe at some point. We kept versions of Java and IBM WebSphere on NFS shares because Oracle or IBM (or both) wouldn't even let us download versions that old and insecure.
Software businesses are way more willing to continually rebuild an app every year.
Building something that works ? Not so easy
Pushing that thing in production ? That the hardest part
https://play.google.com/store/apps/details?id=com.blazingban...
https://play.google.com/store/apps/details?id=com.blazingban...
https://play.google.com/store/apps/details?id=com.blazingban...
Are they perfect? No probably not, but I wouldn't have been able to make any of these without LLMs. The last app was originally built with GPT-3.5.
There is a whole host of other non-public projects I've built with LLMs, these are just a few of the public ones.
You might not go for a run when the socks are not there, but I don't think you would start questioning your ability to run.
I have learnt so much in this process, nowhere near as much as someone that wrote every line (which is why I think being a good developer will be a hot commodity) but I have had so much fun and enjoyment, alongside actually seeing tangible stuff get created, at the end of the day, that's what it's all about.
I have a finite amount of time to do things, I already want to do more than I can fit into that time, LLMs help me achieve some of them.
I think if you stick with a project for a while, keep code organized well, and most importantly prioritize having an excellent test suite, you can go very far with these tools. I am still developing this at a high pace every single day using these tools. It’s night and day to me, and I say that as someone who solo founded and was acquired once before, 10 years ago.
You can see by Contributors which ones Claude has done.
I have no idea if the code is any good, I’ve never looked at it and I have no idea how to code in Rust or Racket or Erlang anyway.
In that case, are you really producing multiple projects per week? If you've never looked at the code, have you verified that they work?
the list could go on
You can see all of Claude’s commits.
I’ve shipped so much with ai.
Favorite has been metrics dashboards of various kinds - across life and business.
Because vibing the air about those gains without any evidence looks too shilly.
Why build them if other can just generate them too, where is the value of making so many projects?
If the value is in who can sell it the best to people who can't generate it, isn't it just a matter of time before someone else will generate one and they may become better than you at selling it?
No offence to anyone but these generated projects are nothing ground-breaking. As soon as you venture outside the usual CRUD apps where novelty and serious engineering is necessary, the value proposition of LLMs drops considerably.
For example, I'm exploring a novel design for a microkernel, and I have no need for machine generated boilerplate, as most of the hard work is not implementing yet another JSON API boilerplate, but it's thinking very hard with pen and paper about something few have thought before, and even fewer LLMs have been trained on, and have no intelligence to ponder upon the material.
To be fair, even for the most dumb side-projects, like the notes app I wrote for myself, there is still a joy in doing things by hand, because I do not care about shipping early and getting VC money.
I've just added a ATA over Ethernet server in Rust, I thought of doing it in the car on the way home and an hour later I've got a working version.
I type this comment using a voice to text system I built, admittedly it uses Whisper as the transcriber but I've turned it into a personal assistant.
I make stuff every day I just wouldn't bother to make if I had to do it myself. and on top of that it does configuration. So I've had it build full wireguard configs that is taking on our pay addresses so that different destinations cause different routing. I don't know how to do that off the top of my head. I'm not going to spend weeks trying to find out how it works. It took me an evening of prompting.
> I'm not going to spend weeks trying to find out how it works.
Then what is the point? For some of us, programming is an art form. Creativity is an art form and an ideal to strive towards. Why have a machine to create something we wouldn’t care about?
The only result is a devaluation to zero of actual effort and passion, whose only beneficiary are those that only care about creating more “product”. Sure, you can pump out products with little effort now, all the while making a few ultrabilionaires richer. Good for you, I guess.
Hobby Lobby has US$723m in annual revenue.
I don't make "products" I solve problems
Here’s a hint: Nobody should ever write a CRUD app, because nobody should ever have to write a CRUD app; that’s something that can be generated fully and deterministically (i.e. by a set of locally-executable heuristics, not a goddamn ocean-boiling LLM) from a sufficiently detailed model of the data involved.
In the 1970s you could wire up an OS-level forms library to your database schema and then serve literally thousands of users from a system less powerful than the CPU in modern peripheral or storage controller. And in less RAM too.
People need to take a look at what was done before in order to truly have a proper degree of shame about how things are being done now.
When you're doing CRUD, you're spending most of the time with the extra constraints designed by product. It's dealing with the CRUD events, the IAM system, the Notification system,...
I mostly agree, but I do find them useful for fuzzing out tests and finding issues with implementations. I have moved away from larger architectural sketches using LLMs because over larger time scales I no longer find they actually save time, but I do think they're useful for finding ways to improve correctness and safety in code.
It isn't the exciting and magical thing AI platforms want people to think it is, and it isn't indispensable, but I like having it handy sometimes.
The key is that it still requires an operator who knows something is missing, or that there are still improvements to be made, and how to suss them out. This is far less likely to occur in the hands of people who don't know, in which case I agree that it's essentially a pachinko machine.
Down with force-multiplying abstractions! Down with intermediate languages and CPU agnostic binaries! Down with libraries!
Or Stackoverflow is really good.
I’m producing multiple projects per week that are weeks of work each.