> When to look at the code or when to just "vibe" test it and move on.
I'm really curious how you're ensuring the code output by whatever LLM you're using, is actually doing what you think it's doing.
Here's a recent example where I used this pattern: I was working on a (micro) service that implements a chat based assistant. I designed it a bit differently than the traditional "chat bot" that's prevalent right now. I used a "chat room" approach, where everyone (user, search, LLM, etc) writes in a queue, and different processes trigger on different message types. After I finished, I had tested it with both unit tests and scripted integration tests, with some "happy path" scenarios.
But I also wanted to see it work "live" in a browser. So, instead of waiting for the frontend team to implement it, I started a new session, and used a prompt alongt he lines of "Based on this repo, create a one page frontend that uses all the relevant endpoints and interfaces". The "agent" read through all the relevant files, and produced (0 shot) an interface where everything was wired correctly, and I could test it, and watch the logs in real-time on my machine. I never looked at the code, because the artifact was not important for me, the important thing was the fact that I had it, 5 minutes later.
Fun fact, it did allow me to find a timing bug. I had implemented message merging, so the LLM gets several messages at once, when a user types\n like\n this\n and basically adds new messages while the others are processing. But I had a weird timing bug, where a message would be marked as "processing", a user would type a message, and the compacting algo would all act "at the same time", and some messages would be "lost" (unprocessed by the correct entity). I didn't see that from the integration tests, because sometimes just playing around with it reveals such weird interactions. For me being able to play around with the service in ~5 minutes was worth it, and I couldn't care less about the artifact of the frontend. A dedicated team will handle that, eventually.
I recently ran across a package in my team's codebase that has a bunch of interrelated DB tables, and we didn't already have a nice doc explaining how everything fits together - so I asked the AI to make me a detailed README.md for the package. I'm currently reviewing that, removing a bunch of nonsense I didn't ask for, and I'm going to run it by my team. It's actually pretty good to start with because the code and DB models are well documented, just piecemeal all over the place, and half of what the AI is doing is just collating all that info and putting it in one doc.
Disagree. Vibe coding is even more powerful if you know what you're doing. Because if you know what you're doing, and you keep up with the trends, you also know when to use it, and when not to. When to look at the code or when to just "vibe" test it and move on.