Making Better Game Decisions with Feedback Testing

You might be familiar with the concept of split testing. Also known as A/B testing, split testing compares different versions of your assets to see how each performs with your players. At the conclusion of your test, you adopt the variant that showed the best response.

Split testing is a critical part of the game development process. It’s pretty much baked into any user acquisition campaign, with ad creatives and app store assets quick and easy to A/B test.

But a traditional split test’s quantitative data can only tell you so much. And this kind of testing is only possible once the game is up and running. That’s why feedback testing is so important. It provides qualitative data to show you the why behind what is and isn’t working.

What is feedback testing?

Feedback testing is essentially surveying players to discover the reasons behind any choice they may make. Why do they pick a particular character? Why do they leave after that level? Why do they choose one advertisement over another? It’s scooping up all of these ‘why’s’ on various aspects of your game.

And interestingly, you can begin gathering this kind of feedback before your game even has a dedicated audience. The earlier you can start gathering feedback, the faster you can squash bugs and make important creative decisions about your game.

You can conduct feedback testing through online or in-person surveys, interviews, and focus groups. The goal is not just to find out users’ preferences but also to learn why they prefer one option over another.

Why is the “why” so important in mobile games?

As a mobile game developer, you’re not designing for yourself. You’re designing for your players. You want them to download your game, play it, fall in love, and then ideally recommend it to others.

For this to happen, you need to understand your players’ needs, behaviors, and preferences. What do they look for in their gaming experience? Why do they respond to certain features and not others? What designs and color schemes will keep them engaged? Finding answers to these questions can help you create a game they’ll love.

When should you test?

Many game developers turn to testing late in the development process after key creative decisions have already been made. They miss a massive opportunity. I recommend testing at every stage in the game design process:

• Ideation
• Design
• Development
• Pre-launch

Testing at the ideation stage

As you ideate, solicit feedback on your character sketches, the game’s name, and storyboards. You don’t need to have a working build. At this stage, you can get feedback on your early designs and concepts to make sure you’re heading in the right direction.

Here’s an example. A publisher tested three different names for a new game to find the one that best matched its storyline. The poll asked 400 people between the ages of 18-44 with an app store spend of at least $10, “which app name sounds better?”. The publisher described the game as an action RPG with stunning, cutting-edge 3D visual effects that lead to a magnificent world where good and evil collide.

Survey respondents were presented with three options:

• Option A: Infinity Discord: Apocalypse
• Option B: Furious Wings: Origin
• Option C: Dark Nemesis

They voted in round-robin fashion, meaning voting on Option A vs. Option B, Option B vs. Option C, and Option A vs. Option C. At the end of voting, Option C, Dark Nemesis, emerged as the majority favorite, largely because it communicated the idea of “good versus evil” better than the others. Plus, the word infinity evoked Marvel imagery, while the word Discord reminded people of the chat app.

Not only did the developers get detailed insight into what the name Dark Nemesis evokes, but they also used the feedback to guide them as they developed the game’s design, storylines, and marketing.

Once you get past the early stages, you can test your target audience’s preference for themes, character art, and their preferred user interface. The more feedback you get, the more data you have to inform your decisions, which can ultimately save you time, and more importantly, help you avoid making serious mistakes with your game.

It doesn’t end once you’ve launched

After your game is live, feedback testing is fantastic for specific updates and campaigns. For example, Mojiworks wanted to test the seasonal update of its Snapchat game Ready, Chef, Go! called Jingle Burgers. Here’s what it looked like:

By using feedback testing, Mojiworks could make quickfire decisions about which creatives to use in its campaign launch. They tested proactively rather than reactively, which is especially vital when your game may only be in the charts for a short time.

It’s quick and easy to get started

For effective feedback testing, you need a large group of people to complete the surveys and give their opinions. You’ll need to find the players, organize the sessions, collect their answers, and present the findings in a digestible way – which can all cost precious time and money.

Feedback tools like PickFu (which I co-founded), have emerged to make it easier and more accessible. Of course, I’ll recommend our tool. But whatever testing platform you end up using, do your research to make sure it ticks all of the boxes for your game’s needs.

To learn more about feedback testing, hit me up. I’d love to have a chat.

Justin Chen is a co-founder of PickFu and a serial entrepreneur of 14 years. He has a passion for creating automated technology solutions that solve problems people didn’t even know existed. A firm believer in remote work and work/life balance, he breaks up his day by shuttling two kids between school and all their various activities through Los Angeles traffic.