Stop aggregating away the signal in your data

Not written by me...but I wanted to share a great post with you created some time ago - I think the author is absolutely right about everything in regards to processing and visualizing of data.

You can head over to stack overflow and read it there: https://stackoverflow.blog/2022/03/03/stop-aggregating-away-the-signal-in-your-data/

Although being a nice idea in theory, with classic visualization tools like Tableau, it is hard to make this reality. Tableau stalls easily when datasets get too large. You need fast data transfer and also enough processing capacity on the frontend side - especially important in case of mobile devices.

The recent progress with Arrow IPC and Arquero in combination with specialized and custom developed frontends might provide the capabilities needed for detailed visualization of data as layed out in the post.