Someone wanted to use FFmpeg to (losslessly) convert video files from one container to another and then delete the original. ChatGPT made a script which created a list of filenames and passed them into a loop which executed ffmpeg on each file. Thus it ran headfirst into several edgecases, for example how FFmpeg by default (unless you -nostdin) consumes stdin byte-by-byte, which messed up the list of filenames. Of course it appeared to work fine, as it clearly was processing a bunch of files.

Secondly, it forgot to map the input streams correctly, so several files came out with subtly missing tracks.

But more crucially, it failed to check the returncode for each file, assuming everything went fine... And deleting the lone copy of several files which failed to convert. Of course there were no backups.

So if you must rely on GPT's, then do so with extreme caution. Don't get fooled by their "self-confidence".