Preferences

Would that kind of data not compress like crazy? Or would they need to keep all that data hot and fast.

From just a single exchange you can reach up to 1 million messages of order book change per second

https://www.nasdaqtrader.com/snippets/inet2.html

   Message Volume  1,684,103,265
   Messages per Second  1,134,640
   Order Volume  871,875,595
   Orders per Second  581,696
   Share Volume  12,814,454,760
   Executions per Second  193,350
Also if you look at equity derivative products which have parameters like type call/put, strike, maturity can be hundreds of financial products for one underlying stock.

I worked in this sector and volume of data is a real challenge, no wonder you often get custom software to handle that :)

Thanks for the insight!
How do you propose lossless compression for all orderbook data? Of course if you are willing to lose granularity/information, it can be compressed a lot

This item has no comments currently.

Keyboard Shortcuts

Story Lists

j
Next story
k
Previous story
Shift+j
Last story
Shift+k
First story
o Enter
Go to story URL
c
Go to comments
u
Go to author

Navigation

Shift+t
Go to top stories
Shift+n
Go to new stories
Shift+b
Go to best stories
Shift+a
Go to Ask HN
Shift+s
Go to Show HN

Miscellaneous

?
Show this modal