Data Export is going away

Update: DataSuite features are now available in a single, affordable, bundle. Find out how to save on your data processing costs for games here.

While we work hard to add more and more features to GameAnalytics, we occasionally have to say goodbye to a feature. Today we’re announcing the well-deserved retirement of our Data Export.

If you currently use our Data Export, then please read on. There are some key changes you need to be aware of, and some next steps you may need to take.

Here are the details.

We’re retiring our Data Export

We’re suspending access to the Data Export on May 31st, 2021, a little under three months from now. We’re replacing it with a series of new services we’ve been developing in collaboration with studios and publishers in our community.

• The Player Warehouse is an instant data warehouse that provides access to player-level data via Google BigQuery. We process the data for you, aggregate it at the daily level, and share it in tables so you can instantly query it with SQL. Historical data is available from the start of 2020 onwards.
• The Event Export provides granular event-level data, per event type, with no aggregation, in parquet format. The data is delivered directly to your own AWS S3 or Google Cloud Storage and can be easily ingested in most data warehouses or distributed databases that support the parquet file format.
• The Raw Export provides all raw data (in JSON format) delivered to your AWS S3 or Google Cloud Storage. Your data team will probably need to process or transform the data to combine it with other data sources or query it for analysis or machine learning purposes.

Which one should you pick?

The Player Warehouse is the easiest and fastest way to access and query your player data, as you won’t need to do any preprocessing or aggregation, and historical data is built-in. Anyone on your team who knows a bit of SQL can analyze this data meaningfully, and build reports and visualizations.

That said, some use cases may require a higher level of granularity. For instance, advanced funnels can be highly dependent on the order players performed certain actions, and you may need individual timestamps to assess when each action occurred. If certain actions in your game have inter-dependencies or are highly time-sensitive, the Event Export may be a better option.

Event Export delivers each event with its own timestamp, grouped in different files per event type, and including only fields relevant for that specific type. With the parquet file format, the data will be compatible with your existing data warehouse, whether you use a Hadoop-based ecosystem, RedShift, or BigQuery.

Raw Export is most similar to the existing Data Export, and it may be useful if you need the data in a very specific format – either to match the format of a different data source or a particular advanced use case specific to your game. Processing and transforming data will also require additional infrastructure costs, so unless there is a strong reason you’d prefer raw JSON data, we generally recommend opting for either of the two previous services.

One thing to note about both the Event and Raw Export compared to the current Data Export is that both services will deliver the data to your setup directly. Therefore, your team will save time from extracting and pipe the data over to your infrastructure, and you won’t have to worry about extra data transfer costs.

Why is the Data Export going away?

As our community of studios has grown, more game teams are looking for access to data in a way that gives them the benefits of a powerful data stack, but without the cost of hiring an entire data engineering team. To make these new use cases possible, we needed to support new data formats, types of aggregation, and ways of delivering data to studios.

The data export tech powering was built several years ago and has reached its scalability and customization limits. To develop these new capabilities we’ve had to start from scratch and build a new generation of services, designed for resilience, scalability, and easier maintenance.

This will be a premium service

Please note, that these new capabilities are premium DataSuite. We always try to provide like-for-like free capabilities for features we reengineer, such as the GameAnalytics web and mobile apps. But with large-scale and programmatic DataSuite such as the Data Export, delivery is very resource-intensive from an engineering and infrastructure perspective. As a result, all programmatic access to data is now part of our premium offering.

Although we believe that the value they deliver to studios is worth their price, we have made every possible effort to keep these services as efficient and affordable as possible for studios of all sizes. If you’d like to learn more about our approach to free, freemium, and premium services, we’ve explained it here.

What’s next

If you’re using the Data Export currently and would like to transition to one of the new options, you can find more details on our DataSuitepage. The Event Export and Raw Export are now in closed beta, so to access these services or get more information, get in touch with us. We’re also working on more detailed docs, which should be out in the next couple of weeks, along with more pricing information, and onboarding perks.