Preferences

Just so I understand: Could this library be used for iterating large collections without blocking the main thread? Basically as a replacement for a worker with synchronous iteration?

Yes, but I'd add a caveat.

It will be very efficient when the async operations are few, and slower. It will not be very efficient when the async operations are many and fast.

That's because the await keyword itself blocks the main thread every time a function call is made. It has to because the `await` keyword is defined to return to the event loop and resume processing only during the next "tick" and after other queued ticks have run.

For a large collection the overhead on the event loop can be calculated as: COLLECTION_SIZE * FUNCTION_CALLS_PER_ELEMENT. Since function calls per element goes up as you wrap layers of helper functions, even with this library you would face a strong perverse incentive to avoid using (async) function calls or nested iterators (if at all possible) when handling large collections, especially with high desired throughput, especially in situations where you want the event loop to stay unclogged so that you can, for example, redraw the UI.

There's a rather famous blog post that cuts to the heart of all this: https://blog.izs.me/2013/08/designing-apis-for-asynchrony/
In the most ridiculous scenario you try to process two high-throughput async data streams concurrently and you end up with this nightmarish "bouncing" where each "thread" of processing can only process until it sees an await keyword before being forced to cede control back the other thread. At this point it would be reasonable to expect that the amount of overhead would exceed the amount of real work being done, and in such a way that the text of the EcmaScript spec makes impossible to optimize or fix.
Thank you for taking the time to explain! While the details seem to be well over my head, I think I‘ll stick to a worker then, as the specific thing I was talking about was taxonomy faceting of a pretty large dataset. Thank you again!
Having had a quick read of the concept, I believe it's concurrent in the sense that Promise.all() is compared to awaiting in a for loop?

In other words, put on the stack at the same time but not a different thread.

Do correct me if I misunderstood though.

This item has no comments currently.

Keyboard Shortcuts

Story Lists

j
Next story
k
Previous story
Shift+j
Last story
Shift+k
First story
o Enter
Go to story URL
c
Go to comments
u
Go to author

Navigation

Shift+t
Go to top stories
Shift+n
Go to new stories
Shift+b
Go to best stories
Shift+a
Go to Ask HN
Shift+s
Go to Show HN

Miscellaneous

?
Show this modal