If any puppeteer developers would also find it useful, let us know - it's always good to know something is wanted before you make it.
This outputs a series of clicks and keyboard events. It doesn't handle the logic necessary for most JS-heavy web-apps.
Scraping an infinite scroll is a good example. A naive algorithm is - scroll, wait, scrape, scroll, wait, scrape...
We have a slightly more sophisticated algorithm to deal with this. Also, what happens when you encounter a page with multiple iframes?
I'm essentially talking about a library which works with higher-level abstractions than just click and type, perhaps 'ScrapeInfiniteScroll' - and similar operations.
See also https://www.testfront.io/ for an actual description of how it works, but don't bother signing up. I'm really bad at marketing/getting the word out, and google took forever to approve the extension so I lost interest/ran out of money. Maybe if others show interest, I'll be motivated to resume the plans I had for it.
Here's a 4 minute video of it in action, covering every user join/login/update possibility: https://www.youtube.com/watch?v=ifkaptIYeL8
We switched it out for puppeteer as it was more suited as a library for our-case. Cypress is a great batteries-included E2E solution though.
As someone who has recently started working with puppeteer, I definitely think a library to alleviate that pain would be immensely valuable.