Newsletter

About this blog

Welcome to the GameAnalytics blog, where we try to help game makers build more engaging experiences through a strong understanding of their core metrics.

Follow us here

How To Conduct User Research In The Prototype Stage For Your Game

Testing ideas in the prototype stage is a vital step in your game's development. But what should you test? And who to? We answer just that...

Content Marketing Manager at GameAnalytics

Whether it’s your game concept or your core game features, prototyping and testing ideas at an early stage can you let you know a lot about the health of your game, and more specifically, if people are actually going to enjoy it.

Ideally, you’ll want to make sure your title is popular the moment it hits the market. Not only is testing in the early stages of development good for getting your game out there, but it can also help you focus your production process on features that will make the most impact on your intended audience.

In this post, you’ll find the steps you need to take when conducting user research during that all-important prototype phase. We’ll include a bunch of different examples of specific test methods (with pros and cons of each) as well as advice for pulling together your very own test plan.

This is a bit of a lengthy post, so feel free to skip to a specific section if you’re looking for something in particular:

Step 1: Plan out your approach

Testing is great. It can tell you things about your game that you’re blind to, or just confirm suspicions that may have been playing on your mind. However, before you test anything about your game, you’ll need to take a moment to reflect on the purpose of your prototype stage.

This stage is about figuring out exactly what you need to test in order to improve your game, but also laying out the steps you need to take to move forward. A few questions you’re going to need to answer are:

  • What is your ultimate objective?
  • What are your plans post-test?
  • What resources do you have?
  • How much time can you dedicate to test?
  • Which features do you need to test?

Now is the time to define your strategy for your user research. The approach you take should be tailored to the data that matters to your game development, whether it’s feedback on game angles, ideas, concepts, gameplay, or, simply just the design.

For example, if I’m creating a hyper-casual title with simple mechanics, it would be foolish to test narrative. A better use of time here would be the games core loop, difficulty or level design.

To help you understand what this requires, we’ve created a useful document with the essential questions and checklists you’ll need when building a prototype test plan. You can download a copy here.

Step 2: Make sure you know what you’re testing, and why

What on earth are you testing?

So you have your approach all together, and you’re ready to take the next step. At this stage, you’re going to have to look at your title and decide exactly on what you’re going to test. Depending on the approach, this could either be a particular in-game feature, aesthetics, the overall UX, or even a monetization strategy.

Knowing exactly what you’re testing will inform the whole structure of the user research. If you’re searching for feedback on your game’s concept, for example, you might want to mock-up an app store screen with descriptions and images to see how new players respond.

Who should you be testing?

Once you know what you’re testing, you’ll have a better idea of who you should be testing it with. This refers to the number of participants you need to gather results from, as well as the demographics and player personas you need to target.

If you want to gain more qualitative feedback on whether a specific kind of player will enjoy your game, playtesting sessions that involve face-to-face interviews might be the best option for you. To do this, you’ll need to be familiar with the type of players your game is targeting, considering things like age, gender, platform, player experience levels and interests.

To give you an idea of what we mean, here’s a few different examples:

  • Specific audience: Hard-core gamers who play for 20+ hours a week, aged 20 – 40, lives in London
  • Less specific: Women, aged 50+ who are interested in puzzle games
  • Open: 10 Beta players

Why are you doing this test?

You should have a pretty good idea what the answer is to this question, but it’s important to write this down. It’s easy to get lost in all of the nitty gritty details when conducting your user research, so this all-important ‘why’ will bring you back to Earth, and keep your vision and goal clear.

A few examples of ‘why’ you’ll be testing could be:

  • See if players simply enjoy the core features
  • Understand first impressions of the game in the app store
  • Find out if levels are too difficult or too easy
  • Learn how players interact with the core game

How are you going to test?

There’s an appropriate tool for every job. Once you’ve answered the above questions, you should be in a good place to decide on which testing methods are the best fit for your purpose. For instance, your decision will vary massively depending on whether you’re searching for in-depth qualitative feedback on player experiences, or if you’re searching for more general feedback from a large sample of participants.

We’ve actually listed some of the most popular testing methods below to aid your user research during the prototype stage…

Step 3: Choose the right testing methods

A/B testing

As you may already be aware, A/B testing lets you test two or more things at the same time in a controlled environment. These types of tests normally are an iteration of one aspect, as it’s important to make sure you’re testing apples against apples.

For example, you can test a red icon colour against a green icon colour. But you can’t test a red icon colour against a different type of font – the results of this will just confuse you and lead to inconclusive results.

Pros Cons
Results provide clear and conclusive evidence Can be time-consuming and resource-heavy during the set up and planning phases
Allows you to test new and innovative ideas and compare them against each other Results are more quantitative and lack depth
Can be performed on a large scale
Useful for determining new approaches and eliminating ineffective options

Online surveys

The use of online market research tools has come a long way in the last decade. There are now several game-specific platforms (like Playtest Cloud, Sekg, Player Research, and more) that are set up to offer you easy survey functionality when it comes to testing elements of your game.

Pros Cons
Easy and quick to set up Results are indicative rather than definitive
Plenty of providers of online surveys Responses to survey questions can be limited
Easy to target different audience segments A lack of interviewer to clarify results can lead to misinterpretation and less reliable data
Reasonable costs
Informative results gathered from large samples
No interviewer or moderator needed

Diary studies

Diary studies are a form of long-term research that takes place over a set duration of time. Typically, users report their activities at regular intervals to create a record that developers can study. It’s up to you how this is structured, but it can include player’s thoughts, frustrations or enjoyability levels over a number of sessions.

Pros Cons
Good for determining long-term behavioural trends Time-consuming process
Good for assessing retention or learnability of different features More complicated and costly to set up the testing process
More in-depth qualitative feedback Requires greater commitment from testers
Opportunity for participants to expand on overall experience with follow-up interviews Reliant on individuals to self-report accurately

Expert reviews

Mobile game experts can offer something unique when it comes to prototype testing. Whether it’s a pro gamer or a professional prototype tester, sometimes you need feedback from someone who has a firm grasp of the medium to inform your development strategy. This should help offer actionable development advice that can help you transform specific features of your game.

Pros Cons
Tester will have specialist knowledge of game mechanics and features Feedback is isolated to a small number of particular players
More practical feedback and information Not completely objective
In-depth analysis Players may have their own bias or preferences
The opportunity to prompt for more information in specific areas

Face-to-face interviews

There are a few variations for how this test method works. A basic model is having a group of testers play your game for a set period of time, allowing them test out different features and experience the game as a whole. Following this, an interview can be performed to gather specific feedback related to their experience.

Pros Cons
Results are based on observed behavior rather than indirect results through a third party Time-consuming and complicated set up process
Ability to register emotions and immediate reactions to features Requires a trained interviewer
A deeper understanding of how players understand and enjoy your game Artificial environment might produce less reliable data
The chance to gain more insight with follow-up questions or conversations with the testers

Step 4: Reviewing the results

When reviewing your test results, everything should tie back in to your test objectives as outlined in your test plan. In the end, your interviews, studies or surveys are all designed to improve your overall development process, so make sure you use the results to inform your strategy going forward.

The data you receive should also be reviewed as objectively as possible. If your team are too close to the situation to judge it fairly, then bring in some fresh eyes to help you understand your next course of action. It’s important that you don’t let emotions get in the way. If you receive some negative feedback, you simply need to suck it up and adapt your future approach – there’s no two ways about it.

If your results confirm your initial ideas then hooray! You can now act with the confidence you need to invest additional resources into further developing and refining your game, bringing it that one step closer to the finish line.

Here’s an example…

As we’re a strong advisor of ‘show don’t tell’, here’s an example of how you can conduct your user research for your next project. Also, don’t forget to download our template, which should help you cut some corners when setting up.

Example: We have an unproven game concept in mind, but we don’t know if players will like it

  • Step 1: Plan approach
    • We need to find out if our concept will resonate with potential players. To achieve this, we’ll need to mock-up an app store page, with fake images, text and gameplay. Our resources are limited and budget tight, so this route will make sure we don’t exhaust ourselves in case we need to do more testing.
  • Step 2: Answer the 4 ‘What, who, why, how’
    • What: A fake app store page, with images, text and mock gameplay
    • Who: Men aged 20 – 35 who work in London and play mobile games (roughly 300 invitations.)
    • Why: To find out if our game’s concept is interesting enough for players to want to download
    • How: Online survey
  • Step 3: Set up
    • Create mock assets for testing
    • Decide and set up online survey questions
    • Identify audience and send out invitation
    • Decide on closing date
  • Step 4: Review the results
    • At this stage, we should have a collection of results that will help decide our next steps.

Remember…

Due to the high costs associated with developing and launching new games, identifying failing ideas as fast as possible is crucial. This could save you a ton of time in the long run, as well as make sure you’re on the path to success.

When figuring out your approach, don’t get stuck within a rigid test format. What might work for one studio might not necessarily work for you, so tailor your user research to suit your target audience, genre and the development stage of your game (as well as the resources that you have).

Finally, use a collection of different test methods for more reliable results (if you can). Any areas that overlap between different methods will give you more solid conclusions, rather than relying on the success of one method alone.

More on game testing

For more information and advice on game testing, we’ve collected some great links that can help you while you’re conducting research during the prototype phase:

Mobile Gaming Report

Great stories about great game developers, and how they thrive in the era of data.

* You get our industry report the minute you sign up.

Learn more about GameAnalytics

GameAnalytics.com

Your are reading

How To Conduct User Research In The Prototype Stage For Your Game