Right, it's the right alt, not left:) It's muscle memory to the point I had to check myself doing it to be sure :)
The shortcut to change was definitely something with Ctrl and Shift because I remember accidently switching layout when I was selecting text by whole words with Ctrl+Shift+Left/Right.
Tilde works funny on linux - it makes alternative version of every letter, not only from the current locale. I was accused of being a Russian pretending to be Polish on some Polish forum long ago because I wrote something with a Greek (or cyrylic?) letter by accident because I did something with home directory in the background and only pressed ~ once instead of twice :)
If I remember correctly shortcuts to change layout/language are by default Ctrl+Shift and Alt+Shift respectively (correct me if I'm wrong). These are incredibly annoying, especially in some games. Luckily though you can disable them from the settings. Instead there's Win+Space, which is a Godsend and should've always been the only default.
Fun fact: on Windows Polish programmer's keyboard you can use the Tilde key (Shift+Grave) to input Polish characters as well, e.g. press Shift+Grave (it won't put in any symbol at this point), release and then press 's' to input 'ś'. However it makes it problematic to input the tilde symbol itself, so I've modified my layout with the MS Keyboard Layout Creator to get rid of that functionality/flaw (aside from other minor improvements) https://www.microsoft.com/download/details.aspx?id=22339