Rewarded Video: Retention Impact in Match-3 Games

Editor’s note: the following article is a guest post from Yaniv Nizan, co-founder and CEO of Monetization Measurement Platform, Soomla. The original report is available to download here: video ads retention study in mobile games (PDF).

The Purpose of this Study

Rewarded video ads can have a positive or negative impact on retention – depending who you ask. This is an unbiased 3rd party study to answer this question for Match-3 games.

Test Setup – Data Collection and User Selection to Groups

This study aggregates data from 6 Match-3 mobile games. All the data in this test was collected by the SOOMLA TRACEBACK platform coming mainly from:

• Client side SDK installed in the tested games
• Ad-network reporting APIs – mainly used as a source of data validation

The selection of users to groups was done differently in each test:

• Test 1 – refers to a change that was made in the game to make rewarded videos more prominent via prompts. Users were selected into groups based on activity before and after the change
• Test 2 – users were selected to groups by choosing to engage or not engage with the video ads

The Change – Implementing Frequent Prompts for Videos

The games implemented a change to the integration of the rewarded video into the match 3 game. The basic transaction is “Watch Video and Get 1 Life”:

Before implementing the change (Period A) – the users had to run out of lives (5 lives) before getting the video end screen (below) and could also access the videos by clicking on the lives counter.

After implementing the change (Period B) – users got the video end screen after every time they failed a level instead of the standard end screen.

Test 1 – Retention Before and After the Change

We examined the impact on existing users. Here the selection to groups is based on whether the user had activity before or after the change. Note that some of the users appear in both groups.

Group A – users who played at least one day in period A, before the change was made.

Group B – users played at least one day in period B, after the change was made.

The table compares the return rate after 1 day and in the 3 day period following the play day.

Table 1 shows a consistent impact on existing users. If we look at the D1 return rates, the impact ranges from 0.5% to 0.9% and on the 3 day return rates the impact ranges from 0.2% to 0.5%. These changes however are too small to detect in week by week testing and our recommendation is to implement a split A/B test.

Test 2 – Retention of Video Engagers

While this test involves self selection it is still an interesting data point to look at. In addition, it was important to also reproduce the tests made by ad providers.

Non-Engagers – users who ignored the prompts and never watched video ads

Engagers – users opted-in to watch a video ad in return for lives

Converters – users who installed the apps that were advertised to them

The table compares the return rate after 1 day and in the 3 day period following the play day.

Conclusion 1

Table 2 shows that users who return to the game and video engagers are in positive correlation. However, combined with the results of test 1 the conclusion is that the likely cause is that users who engage with the game also engage with the videos. In other words, it is not the addition of the video ads that resulted in the higher return rate.

Conclusion 2

The high return rate among users who installed advertised apps shows that advertisers are not “stealing” users away.

Revenue Impact of the Change

Here we are presenting the commercial impact of the changes made to the video prompt. Although the changes might look insignificant from a game design perspective, the behavior and commercial impact where quite drastic.

Opt-in rate – the ratio of users who selected to watch videos in proportion to the total active users Note that Daily Ad Revenue is given in US dollars and Ads ARPDAU is in cents.

Table 3 shows a big impact on revenue made by the change in video ads implementation. A small changes in the prompt made a big difference in opt-in rates and revenue with almost no impact on retention.

Final Thoughts

The study shows that doubling video ads exposure had negligible impact on retention. This finding contradicts claims that adding video ads improves retention. Our conclusion is that video watchers show better retention rates because the engaged users are more likely to watch videos and not because the video watchers are more likely to be engaged users.

Another conclusion is that users who install apps that were advertised to them are as likely to come back as a user who didn’t see an ad at all. The final takeaway is that overall the tested change was a positive one. The games doubled the ad-revenue with negligible impact on retention.

?  Further Reading: Increasing Rewarded Video Revenue by Mastering Your Opt-in Ratio