As someone who has both spent quite a bit of time writing scrapers and later lots of headache on blocking malicious bots from accessing websites, I can tell you this has become futile. Bot makers aren't stupid. If you put in a check for how fast actions are performed, they will put in a sleep timer in their script. If you start blocking residential IPs because many people use it, you are probably just blocking a school or dormitory, while the real bots will quickly move to another IP once they smell something is off. Today with modern multimodal LLMs, you can bypass almost every "human-check" imaginable. And if they can't pass something, most of your users sure as hell won't either. Not because it is too hard, but because it will take too long to solve. The sweet 3-15s actionable human intelligence threshold has been passed by now. The cats and dogs type captchas were already solved more than 12 years ago by simple CV machine learning. The tech has progressed an insane amount since then. In the end I always ended up basically doing what SoundCloud did here if my service was sensitive: Block entire countries, all tor exit nodes and all known VPN ASNs. That will get it down by like 90%. Bear in mind that anyone who wants to put in some effort will still easily bypass this, but at least the low-effort guys from third world countries will take a while before they catch on. So you can go back to doing some actual work in the meantime.

This is the main issue here, the web has become actively hostile to normal people in the quest to monetize every second of online activity.

Snowden showed the NSA has taps upstream, so in my book: that's over. I'm fairly convinced if your company reaches a size where it could potentially be a national security threat, the government comes knocking (Facebook, Apple, Twitter, etc.), so that seems like it's over. You have the AI companies scraping god knows what. And, I imagine most countries have corollaries.

Really, all the bad actors I'd encounter in my daily travels would be ones who want to steal money from me. That's a simple ideology. I can handle that. My identity gets stolen, my bank account...there's multiple levels of billion dollar companies with vested interest in me not losing faith in "the system," so I'm not worried about it really.

If a company wants to associate my phone number to glean all my purchases forever in order to target tailored ads to me, fine. Again, it's in the spirit of taking my money, which is a simple ideology.

If the neighbors want to snoop on my traffic, hats off to them for having the capacity to live two lives: both theirs, and mine after they figure out my day-to-day dealings. Doubt they have time to do much about it. Hard enough to live one life in 24 hours.

If the government wants to try and keep tabs on everything to see who's making ICBMs and who isn't, or whatever else they want to do, that's their prerogative but it seems like a complex goal that doesn't affect me.