How To Do An Efficient Game Test

We would love to think that our creations provide a great experience from the get-go. That they are rock-solid, that we don’t need to run tedious series of Beta test sessions. But we have a terrible bias in favor of what we do. We invest so much time and energy into our work that it is hard to take a step back. We are so connected to our creation, so deep into our work that we can’t keep track of the big picture all alone. Our vision is often more clouded than we would like to think.

Last year, I drew an Asian-inspired character for a game project. I was quite content with the looks of it, so I shared the illustration publicly. Soon after, a fellow artist took a look at it and commented: “The colors are nice but… the proportions are off. It looks quite deformed at the moment. You should definitely mirror that guy.”

I opened the Photoshop file in disbelief, flipped the piece and… I was shocked. The character was skewed! It looked completely off. And it took someone else’s perspective to open my eyes. Which is common as an artist. We need feedback all the time.

And it is the same with game design. We easily get used to the way we implemented mechanics or controls. We think like designers. We know how our game has been designed and why it has been designed that way. We can justify poor decisions with theory, or get caught thinking that our games are better than they actually are.

That is why it’s critical to get constructive feedback on a regular basis. Both from peers and from our audience. We can find that feedback in honest test sessions. So we have to run our creations through regular sessions of tests, starting from the preproduction phase and with our earliest prototypes.

How can we run efficient tests and get to enjoy a sometimes tedious or unpleasant process? That’s what we are going to approach in this article.

I see at least 3 key things for establishing a good testing workflow

1. Pick clear goals, as always with any design task.
2. Learn to embrace even harsh critiques, to enjoy the process. As long as they are genuine and constructive.
3. Streamline your tools and methods.

You need a specific set of goals

According to Jesse Schell’s “The Art of Game Design”, a test session is a prototype. Not of your game, but of the experience it provides. And any prototype is meant to answer a specific set of questions, to fulfill a clear goal. Do the visuals appeal to your target audience? Does a given mechanic or a level cause frustration? Does level 2 look worse than the others? Which control scheme works best for the majority of testers?

[bctt tweet=”A test session is a prototype. Not of your game, but of the experience it provides”]

Those are the types of questions you should be looking to answer with a test session. They are simple, precise, and useful for you to improve your game.

This also means that for each question, you should have a plan to get answers. If you are looking for feedback on a boss fight, there is no need for the participants to go through the whole game. This would be a waste of your most precious resource: time. Instead, the players could play through a tutorial, followed up by a short level to warm up… and on to the boss! Or maybe they should directly face a weakened version of their archenemy. In case you only needed critiques on the visual feel of the encounter, for example.

You also want to focus the tests on your questions. That may sound obvious, but if you have a goal, it is always interesting to remove any noise that would prevent you from reaching it. Don’t hesitate to make a special version of the game where the player only runs through portions of levels that are relevant to your needs. Yet, this isn’t a reason to shut down your mind to happy accidents and unexpected events! That is what I love most about testing: it comes with its lot of surprises. Be it a strategy you had never thought about it, or a funny bug, you should embrace those opportunities to explore new aspects of your creation.

The 2 schools of game tests

There are mainly 2 schools when it comes to running tests: some designers like to watch the players trying out the game, and some don’t want to influence the player’s experience with their presence.

To me, it is most interesting to watch players trying out your game, as you can watch their emotional reactions live. At least as long as you don’t influence their experience too much. Just let them play and don’t disturb them in any way. Don’t help the player by telling him what he should do, because that’s the game’s role.

[bctt tweet=”Don’t help the player by telling him what he should do, because that’s the game’s role”]

It is especially instructing to see a player discover your game before you because after playing your demo, they won’t remember most of what they went through. You can’t expect them to recall the details of the experiences they had in your game. Playing is most of the time a forward process: we move towards the end of a level, the end of a quest, or any goal that the game provides us with. Few other events mark our mind.

Another school of thought considers that you shouldn’t be present during tests because it influences the players. Even if you don’t talk, your presence might push them to play differently. The idea is not to influence them, to stay out of their way while they are discovering the game. It is certainly true, your presence may alter the player’s experience. But if you’re not looking for scientifically accurate results, it is both funnier and faster to be directly in touch with the testers.

Also, you should never talk while you receive feedback from a player or a fellow professional. This is something I was told early during my education as a designer. Take the time to listen to the whole critique without arguing. Showing emotion might prevent the person you’re speaking to from being honest. If you cut the person you are speaking to in the middle of her speech, you prevent her from developing her entire thought process. It shows that you are not mature enough on an emotional level to accept it, and you’re likely going to waste both your saliva and the tester’s time.

Beta testing the fun way

Anyway, tests don’t have to be a pain to go through  as game designers. It is a matter of mindset. Regardless of the kind of feedback you get, it is only here to help you improve your creation and your craft. You have the right to sort the critiques you should take in account and those that are irrelevant to your project. You have the right not to apply a critique that would take too a lot of effort for a small benefit.

[bctt tweet=”Tests don’t have to be a pain to go through as game designers”]

I see testing as a form of dialogue between our creation and the player. It is an occasion to combine business with pleasure. On one end, we are doing a necessary job: gathering feedback. At the same time, we are looking at an opportunity to connect with our audience, an opportunity to hone our relationship with our community. It depends on your testing environment though. As I work alone, I often share demos online.

Or I showcase my work in local cafés. Their warm ambiance turns testing sessions into a form of fun worktime. It is a real relief from focused production work. A great occasion to take a step back and contemplate all that you achieved. Even if the feedback is not that positive, you can take a deep breath and enjoy a moment of connection with players. At the end of the day, at least this is the reason why I make games: to feel connected with people. It may not be the case for everyone, but our job as designers is still to create games for others. We put our work at the service of the players. This philosophy permits me to deeply appreciate even negative or unpleasant feedback. As long as it is genuine (haters are another story).

Part of being a designer means being able to communicate well. Your ability to understand others and to exchange with them is essential to share your excitement for your own work. But it is also useful to be able to formulate questions well, thus gather more relevant feedback. You can take tests as a great opportunity to improve both yourself and your craft. Being in touch with your audience, your users, you can get to know them and their needs better. Which is essential to then build great games, tailored for their taste.

Streamlining the process

Just as with any other aspect of game design, we want to establish a general workflow to run test sessions efficiently.

In the case of face-to-face tests, the process is straightforward: you can prepare your questions in advance and directly interview the testers one by one. And do so right after they finished playing, when the experience is still fresh in their minds.

I quite like one-on-one testing sessions. They provide a lot of insights on how the game affects the player. You can see the player’s emotions or reactions on his face. However, individual interviews still give a limited amount of technical information.

There comes a point in a project’s developers where we need feedback on a larger scale. Be it to see how smoothly the game runs on a variety of mobile devices, or just in order to verify our demographics. In those cases, individual tests are less than ideal. At that point, we want to run tests online. If you have done any in the past, you’ll know that it’s not easy to gather valuable feedback with random and remote players. Thankfully, there are 2 tools for us to maximize both the amount and the quality of the data we gather: appealing short forms and analytics.

Who doesn’t love beautiful forms?

Actually, few people like forms. Or rather, most people hate long lists of questions, pages filled with large bodies of text. Luckily for us, it is pretty easy to design good-looking forms. There are but a few rules we have to follow. And as you can guess, the first one is to keep it as short and as simple as possible. It should be organized logically and stick to the point, to your objective. If you absolutely need to ask numerous questions, break up your form into multiple pages. Or you could even break up your test sessions into multiple, focused demos.

Another way to create solid surveys is to illustrate every exposed situation with a screenshot from the game. For instance, if you are talking about the entrance of a boss’s lair, just show a picture of it right before the question. It will refresh the player’s mind on the look of the environment. It can even bring back some memories of his or her experience.

You can use a free service like Google forms to quickly build forms and tailor them to your needs. If you can afford it, you could even integrate a survey inside of your game. That way, you’ll increase your chances to see a random online user filling it. It is not uncommon to see it done in advergames or serious games. Curiously, in the entertainment industry, it is pretty rare.

Integrating forms into your game offers another advantage: you can collect the results with your favorite analytics API.

Let your analytics API do the job for you

Game analytics can be reused and repurposed, so they are a great candidate to improve your Beta testing workflow.

Some basic metrics are useful for most games, like how many times a player loses to a given challenge. If you put that in perspective with the results of your survey, you could automatically compare the player’s death count and his overall appreciation of a certain level. This can give you a hint regarding the fairness of your game’s challenges.

You may also get the answers directly from your form, you might say. And you would be right! But your funnels and other general metrics will bring objective, complimentary insights for you to dig. You can’t ask everything you might want to in a form, as you have to keep it short. And various metrics may teach you something unexpected about your game. That is how the authors of Realm of the Mad God discovered that the in game items that were purchased the most were the ones that benefited the whole community. And turned a great game into a financial success.

Plus, once you got the hang of them, analytics are extremely easy to use. It takes but a line of code for the system to collect a stream of data on your behalf. And game analytics are especially good when it comes to tracking player choices, something that is hard to do by hand. Analytics can be one of your test’s best friends in a sense. They work hard in the shadows, living you with time to focus on seeking other kind of feedback during your test sessions.

Key takeaways

This time around, the article’s summary is nothing fancy. You already got it in the intro! If you want to improve the way you run tests and get better feedback for your games, you can follow those 3 recommendations:

1. Pick clear, specific goals.
2. Learn to accept constructive critiques and take the whole process cheerfully.
3. Embrace forms and analytics to streamline your tests.