I don't worry about security too much given it is just bound to localhost, but I do use a simple password (and make it modifiable by the user). Avoiding port collisions in the real world isn't a big issue, just ask an AI for the least assigned default ports and chance of collision is minor (in worst case, also user modifiable). In return, you get free "remotability", which is kind of a big deal IMO.

I do wish gRPC allowed for easy usage of UNIX domain sockets and perhaps named pipes, however. Sometimes all you need is IPC, but in my case, I'm happy to have remote usage builtin.