It's not a good reason but I believe this happens because voice command has to go to Google server, Google talk to light vendor API, light vendor communicates with your device, and lights go off only if all of this succeeds.

Meanwhile, button in vendor app will not use internet so lot less can go wrong.