Improving funnels processing from ~20 mins to seconds

Editor’s note: we recently shared details of a major GameAnalytics update which includes a substantial overhaul to our product and infrastructure. Check it out for more context.

At GameAnalytics we have been using Apache Druid and Imply (Imply.io) for over four years  to power our analytical backend, allowing us to achieve a responsive frontend experience for our users. This is due to the low latency querying that Druid enables through the usage of approximation algorithms.

Recently we had the opportunity to present the work behind our Druid implementation in the BigDataLDN 2022 event, where we covered our new funnels feature and the usage of the Theta Sketch algorithm from the Apache DataSketches library (https://datasketches.apache.org/).

Here’s a recording of the talk. Enjoy!