rEFInd is more or less exactly that.
How do you anticipate users choosing Windows or Linux at boot time without a bootloader?
If the idea is they go cold turkey full Linux, good luck with that.
if the idea is they use their UEFI firmware boot menu, you're forgetting how unintuitive that is for most users with most uefi interfaces (spam hotkey at boot, wait for slow loading uefi, navigate to subscreen with boot order, find right menu item, either reorder and save or press F-key combo to "boot once now")
I've not had a MB where you had to spam any key. Just hold it on boot and wait until you see the menu. Select the one you want to boot now with either mouse or keyboard. This doesn't usually change the default boot order.
If you managed to install linux then this really shouldn't be a thing to get hung up on.
Replace Grub with nothing. If you're not doing bootable snapshots like openSUSE, then there is virtually no benefit in a "boot loader". The linux kernel + cmdline (+other stuff like ucode or secure boot signing stuff) can easily be packed into a single bootable .efi file.
That efi file will then get an entry in your uefi boot device list just like windows already has/had. This way is better anyway, since windows will overwrite your uefi boot order with every significant update, meaning users will already need to know how to boot other os's.