This seems consistent with the letter of the licence. https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html
See also https://www.gnu.org/licenses/old-licenses/gpl-2.0-faq.en.htm...
I wouldn't have written my comment if it weren't.
Right, I was agreeing with you.
That's a good point and I'm glad you mentioned it. However, one set of circumstances is enough :-) If you do not wish to distribute the source code to other third parties, you do not need to, as long as you distribute it to your customers. I definitely could have described that better.
The confusion is on your end. Section 3 of GPLv2 requires you make the source available to any third party, not just your customer (and GPLv3's provisions are similar).
There is exactly one set of circumstances that permit you to say "no" to any and all non-customers, and it involves not limiting the initial distribution to just binaries. That is, iff you provide binaries together with the source to the customer who's giving out copies of the binaries themselves, then it is their responsibility and not yours, and only in that instance are you allowed to turn down third-party requests for source. This would also hinge on having a customer who would be fine giving out copies of the binaries they received from you in step 1 but for some reason wants not to give out copies of the source code that they received with it.