· 9 min read
Wildlife Studios: Why Your Studio Needs Data Scientists
Editor’s Note: This article was originally published by Vitor Margato, Senior Data Science Manager at Wildlife Studios. You can read the original version here.
Data science has been a hot topic for the past few years, but it’s often hard to understand what data scientists actually do in real companies. They seem to be behind whatever is going on when your favorite streaming app recommends you new songs, or when a ride-hailing app assigns you a driver, but what exactly is their role?
In this post, I don’t try to describe how data scientists can help companies in general — I’ll focus on mobile gaming companies, because that’s where I have firsthand experience. Hopefully, some concrete examples will help you form a clearer picture of data science. But first, let me just take a step back to explain why I think those companies absolutely deserve your attention.
Mobile gaming is extremely popular, reaching 2.4 billion people in 2019. It’s also hugely profitable: games overall are expected to generate almost $150 billion in revenue this year, with almost half that amount coming from mobile. Games earn more money than movies and music combined. That means a lot of data as well.
At Wildlife, as millions of people play our games every day, we get more and more terabytes of fresh data that we can use to produce great products. But how exactly? Below, I list ten different ways to do it. It doesn’t cover everything, of course, but it should serve as a good starting point.
1. Data lets you understand how players actually interact with your games
As user-experience designers know all too well, you are not the user. That means developers can’t just rely on their personal preferences when designing products for people all over the world. You have to measure how people interact with your games in the wild. Do they ever get to use that multiplayer mode you cherish? Are they not making purchases because many of them can’t even see the store in the first place?
Mobile game developers can instrument their products using either third-party tools or in-house solutions, and thus collect very granular data on how people actually enjoy their games. That can lead to higher-quality roadmaps, centered on what really matters to customers. Without such data, you risk spending months developing features that won’t move the needle in player engagement and profits.
2. You’re able to focus on KPIs that matter — instead of vanity metrics
Companies love charts that only go up and to the right — total (historical) downloads, revenue, etc. That’s natural, but those metrics are usually not well-suited to running day-to-day operations. At Wildlife, we often rely on cohort-level data, i.e., data describing activity patterns of users having installed our games on a given day. That’s important because it produces better measures of the health of our business.
For example, it’s perfectly possible that a game is performing worse in important ways even as its daily revenue keeps growing. If that revenue growth comes only from the fact that it’s had a recent boom in downloads that is now over, and if more recent cohorts seem to generate increasingly lower revenue per user, then that’s a big problem — one that doesn’t show up unless you look at the right metrics. (The Lean Startup by Eric Ries is a great reference on this.)
3. You can distinguish causation from correlation using randomized tests
Yes: causation and correlation are not the same thing. Source: Spurious Correlations
As this guy illustrates much better than I could, the correlation between variables does not imply a cause-effect relationship. Say you implement a feature in your game that lets users join clans (or guilds or another funny word for “groups”). Your co-workers get excited because players who join a clan are much more engaged than those who don’t, which clearly demonstrates the feature is a success… right?
Not really: clan-belonging players are probably different from their non-clan-belonging peers in important ways that have nothing to do with clans. Specifically, they probably enjoy your game a lot, and that affects both how long they are expected to remain active and how much revenue they are expected to generate.
In general, you have to perform a controlled experiment where some of your users are chosen at random and are not exposed to your new feature. You can then use statistical techniques to compare players who see and who don’t see the feature to tease out the causal effect.
4. User-level data makes fine-grained calibration possible
Building great products is about vision, but it’s also about lots of incremental improvements. The most fundamental aspects of our games are usually the result of careful calibration done over the course of months or years. Analyzing user-level data helps us understand whether our onboarding is smooth, whether a given mission is too hard and causing players to churn, whether people ever go to the shop, etc. That’s a very different reality from that of most physical products.
5. You can customize content according to each player’s behavior
Just like e-commerce websites can recommend items you may like to purchase, mobile game developers can tailor special offers — or any other content, really, at least in theory — to users’ behavior. From simple, hardcoded rules to complex machine learning models, there’s a lot of room to make games adapt to each player, and thus become more fun and more profitable. If some players get stuck during the game tutorial, you can gently point them to a better way to use the controls. If other players’ behavior indicates they’re looking for a very rare card to complement their deck, you can offer it to them at a special discount. Endless possibilities here.
6. You can price your items in a more scientific, profitable way
It’s not always the case that you want to price each item to maximize the revenue it generates. Some special offers, for example, can appear very profitable just because you’re not taking into account what other purchases they’re affecting negatively. In other situations, some key items should be thought of more as an engagement driver than a revenue driver in itself. However you define your goal, in mobile games there’s a lot of room to experiment with pricing and infer those price-elasticities you’ve heard about in Econ 101.
7. You can predict your players’ lifetime value
Having precise, constantly updated predictions of how much revenue each user segment will generate over time can be a strategic advantage to digital product companies — not just to mobile game developers. That’s true both from a marketing perspective and from a managerial one. On the marketing side, paid user acquisition is increasingly competitive, so good lifetime value (LTV) estimates are essential to reach more users in a profitable way. On the management side, you need to have revenue predictions over a time horizon that can reach years — and you can’t just wait for that revenue to actually be recorded.
8. You can allocate your marketing budget in a much better way
Of course, paid marketing is not just about estimating LTV. Marketing apps is hard. You have to factor in how your campaigns can be scaled up with more resources, how additional marketing investment brings in more paid and organic users, how you will find out which promotional videos attract the most players, etc. At Wildlife, we have a data science team completely devoted to marketing-related projects. Domain knowledge goes a long way here, so it’s important to make sure data scientists have a productive relationship with user acquisition teams.
9. You can increase organic downloads with better keywords and packaging
Quantitative methods are also extremely important to make your game reach more people organically. Keywords and packaging (i.e., everything you see when you go to a game’s page on an app store) have to be constantly improved, and there are challenging technical questions associated with it — from distinguishing the influence of each keyword on downloads to estimating the effect of changes relying only on observational (i.e., non-experimental) data over time.
10. You can detect fraud using unsupervised learning
A large portion of mobile app downloads are fraudulent, and games are severely affected. Machine learning models — specifically, unsupervised learning algorithms — can be very helpful when it comes to identifying suspicious behavioral patterns. They can help you go beyond what any human could do using simple rule-based procedures because there are just so many ways in which fraudsters can act. Moreover, as companies get better at spotting signs of fraud, criminals also become more sophisticated, so those detection algorithms have to be periodically re-trained.
What about data engineers?
The above are just some ways in which data scientists can help mobile gaming companies succeed. That’s absolutely not to say they can do it on their own. They have to work alongside product, marketing, game development, and, most of all, data engineering. I’ll say this with links to make sure I make my point clear: data scientists and data engineers are not the same, and both are fundamental. Future posts will cover both data science and engineering, so stay tuned!
Conclusion
Mobile game developers can build better, more successful products with the help of data scientists. In this article, I tried to describe a few ways in which this can be done. This list is not exhaustive by any means — we’re always looking for new ways to create value for Wildlife, and we’re always working on new projects. But I know what people usually write about data science online, and I know it can often feel too vague, so I hope this provides you with a more concrete understanding of how data scientists can impact tech companies in general and gaming companies in particular. Let me know what you think!