· 10 min read
Customizing Rewarded Ads using Machine Learning – Lessons from Rovio
A couple of months ago, we had the pleasure of attending Google’s GameCamp, which was filled to the brim with gamedev content delivered from the best and brightest in mobile gaming. While there, Elif Büyükcan, the Business Intelligence Director at Rovio Entertainment, shared with us how they’ve been using rewarded ads in their games. We’re here to report on their story, and what you can learn from them.
Elif’s main message was this:
“Getting your monetization model right is incredibly important. It takes a lot of time, thought and testing. You need to be constantly measuring, learning and adapting. This makes machine learning (ML) and artificial intelligence (AI) your best friends.”
And today, we’ll go through everything we learned from their talk (specifically about machine learning), including:
- The process Rovio use to hone their monetization models,
- a case study of Angry Birds Dream Blast and rewarded ads,
- and what their advice is for aspiring game developers.
Without further ado, let’s dig in.
Why Rovio wanted to work on rewarded ads
There are a few, solid monetization routes developers can explore these days. And with so many balls to juggle, it’s tricky to know what to focus on and test. For Rovio, they chose rewarded ads. And these were the reasons why:
Reason one: The crown of in-app purchases (IAP) is slipping
IAP has always been king of F2P monetization. But IAP conversion rates are declining for the top-performing games in the market.
When you rely on IAP, you’re only making money from the small minority of players who are willing to make those purchases. So it makes sense that developers are looking for ways to bring in revenue from the vast majority of players who don’t use IAP.
But at the same time, it’s only natural to want to avoid adding anything that might discourage these precious payers from playing your game. This is a big challenge.
Reason two: In-game ads are becoming more popular
Recent studies have shown that players do value ads, if they’re integrated well and relevant to them. And as ML and AI evolve, developers are getting better at understanding their players and personalizing ads.
So the myth that ads hurt games is quickly dissolving.
Reason three: Striking the right balance brings rewards
IAP conversion rates aren’t infinite, but neither are lifetime values (LTV) from ads. They both tend to decline over time, but for different reasons.
In short: the perfect monetization model wouldn’t rely entirely on IAP or ads. The challenge is to find the right balance between the two. Crack that problem, and you potentially stand to give your revenue a bump.
Case study: Angry Birds Dream Blast
During her talk, Elif went into detail about one of their latest games, Angry Birds Dream Blast. Having launched a year ago, it’s already one of their most successful games. It made it into the 100 top-grossing iOS games and has stayed there.
And in case you haven’t played it, the game uses a physics-based ‘tap to clear’ mechanism – you pop colorful bubbles by tapping on them. Clearing big amounts of bubbles gives you birds, and you can combine birds to clear wider areas.
So, how do they use rewarded ads in one of their performing games? Here’s what they shared.
How Rovio were using ads in Dream Blast
They already had rewarded ads, but they were only doing one thing: Each day, the player can watch an ad in return for a daily reward or a longer playtime that day.
These ads often cross-promote Rovio’s other games, when it makes sense – as players who enjoy Dream Blast are likely to enjoy the other Angry Birds games too.
How Rovio use rewarded ads in other games
Rovio has a few games under their sleeve, but that doesn’t mean ads work the same in each one of them. A few different ways they do use rewarded ads is: getting a free booster at the beginning of a level, retrying a failed level, or even reclaiming lost rewards.
Elif says these models were all working well, but these were mostly ‘one size fits all’ models. They wanted to find ways of personalizing their rewarded ads to up their ad revenue without hurting their IAP conversion rate.
So they set about finding the best way to use rewarded ads in Dream Blast. They did this in three stages.
Stage one: They started testing rewarded ads
The team at Rovio had come up with a good idea for a rewarded ad model that they expected would increase their total revenue. So they tested it during soft launch. They used fairly generous configurations, based on what they’d seen in their other games.
Unfortunately, the results weren’t what they had hoped for. Here’s what she shared.
What they tested:
If a player watches an ad (watchable at least once a day) after failing a level, they get three extra moves to complete the level.
The results:
Adding this feature had a negative impact on the game’s monetization. IAP revenue and conversion rate both dropped.
What they learned:
- Using a ‘one size fits all’ method doesn’t work. The revenue they lost from paying players outweighed the revenue they gained from non-paying players.
- They also realised that testing during a soft launch is tricky. Because the sample size is so small, the figures can be inconclusive and minor changes can have a big effect.
What they planned to do next:
Did these negative results mean they should never show rewarded ads ever? “No, of course not,” says Elif. “We needed to study the results, adapt, optimize and re-test.”
They knew these ads were discouraging paying players – and that’s the last thing they wanted to do. So they thought: ‘what if we could show the rewarded ads only to non-paying players?’
But how do you quickly identify what players will become payers and which won’t?
Stage two: Several teams came together to create a hypothesis and a way to test it
They needed to find a way of predicting who the payers and non-payers would be, and then run an A/B test to see if their hypothesis worked.
To do this, they needed their advertising, business intelligence, technology and game teams to be constantly collaborating. The key players were:
- The game’s product manager – responsible for planning tests and new features.
- Data analysts – responsible for running the A/B tests and providing the data on the game’s performance.
- Ad monetization manager – responsible for proposing ideas about ad placements and how to integrate them smartly.
After lengthy brainstorming, they settled on a hypothesis:
‘If we offer the +3 moves rewarded ad only to predicted non-spenders, we’ll see an increase in our total revenue from both IAP and ads.’
Creating a machine learning model
To test their hypothesis, Rovio needed a system that could assign a value to each user – a value that would predict how likely they were to become a spender and how much they were likely to spend. This would allow them to target only the non-spenders with rewarded ads.
[bctt tweet=”In-app advertising is getting smarter, with help from machine learning and AI. Elif Buyukcan told @GameAnalytics how @Rovio optimizes rewarded ads in Angry Birds Dream Blast. Spoiler: it takes a lot of experimentation. #gamedev #indiedev ” via=”no”]
This is a data science challenge. So with Rovio’s ML and AI capabilities, they were well prepared for it. In fact, they already had a player-level LTV prediction model in production that they could tweak to serve their new purpose.
With their new model up and running, they were ready to test their hypothesis.
Stage three: Testing a new model
What they tested:
They offered the +3 moves rewarded ads only to the players the model predicted would be non-spenders.
They ran the test for a few weeks, and kept an eye on the long-term impact for more weeks after that. Their setup was better this time as they’d launched the game globally – so small sample sizes weren’t an issue.
The results:
Rovio saw no positive impact, but less negative impact. Their total revenue was slightly lower – the extra ad revenue didn’t recover the minor loss in IAP revenue.
Keep in mind, this was still an improvement on the last test though. The game’s conversion rate stayed stable and IAP revenue was almost stable.
What they learned:
- The cost of losing a spender is simply too high for ad revenue to compensate
- The prediction model was mostly accurate, but the few inaccuracies were costly
- When the +3 moves reward affects consumption of other video rewards, ad impressions per user can be lower.
- Although the results were still negative, they were moving in the right direction
What they planned to do next:
Elif said they had three options:
- Run a new A/B test with a lower spender probability score. This was the low-risk option.
- Run a new A/B test with ads appearing more often. This could increase ad revenue, but would risk lowering IAP revenue again.
- Try to make the prediction model more accurate, or try a different model. This was the most technical option, but could bring some fast results.
Did they consider giving up on making rewarded ads work? Elif said, “No, we didn’t. Again – these things take time and patience. We keep adapting, testing and learning.”
They decided to go with the third option. They focused on improving or replacing the prediction model, so they could target non-paying players more effectively, with less risk to IAP revenue.
Finally, what we learned from Elif’s talk
That was all that Elif had time to share, but boy did they cover a lot. There were a lot of tips, advice, and lessons in there, but here are our biggest takeaways from this talk:
1. Well-placed ads can boost total LTV
When it comes to in-game ads, it’s all about how you use them. You just need to show the right ads to the right players in the right ways.
2. Every test is a lesson
Even if your test has negative results, you haven’t wasted your time. Negative tests tell you what doesn’t work and they often tell you why. Every test is another stage in a constant cycle of learning and improving.
3. Personalization, MI and AI are key
Players are a hugely diverse group – even among a carefully targeted audience. A ‘one size fits all’ approach to monetization will never be more effective than tailoring your game to individual players. ML and AI are the ideal tools for personalization, so if you’re not using them, your revenue is never going to be as high as it could be.
4. Game development needs to be data-driven
In our industry, we can experiment, test and learn very quickly. There are always new tests and techniques we can use to improve. And to get monetization models right, we have to take advantage of that.
[bctt tweet=”@Rovio know a fair bit about mobile game monetization. So, @GameAnalytics reported on their story about using machine learning and AI to optimize rewarded ads. #gamedev #indiedev” username=””]
We want to give a massive thank you to the Rovio team for sharing their story, we look forward to hearing more in the future. If you want to learn more about Machine learning, then we strongly recommend you check out Rovio’s video on ‘Machine learning meets Puzzle game design’ here: