· 11 min read

Building Tight Games With Game Metrics (Part 1)

Here are a number of game metrics ideas and suggestions inspired by the techniques described in Daniel Cook's blog post about making games easy to understand and play, called "Building tight game systems of cause and effect".

In July 2012 Daniel Cook, Chief Creative Officer at SpryFox, published a blog post about making games easy to understand and play, which he called “Building tight game systems of cause and effect”. Here are a number of game metrics ideas and suggestions inspired by the techniques described in that post.

How to build “tightly”

In his blogpost, Daniel Cook explains that: “a tight system has clearly defined cause and effect. A loose system make is more difficult to distinguish cause and effect relationships. There is no correct ‘tightness’ of a loop.” He then enumerates several methods to tweak the tightness of a game under development. In this post, we tried to translate those aspects into actual game and player metrics. The list below is, of course, far from a complete guide for using game metrics to track game design “tightness”. As Daniel himself writes:

 Not all systems are readily amenable to the intuitive formation of models of cause and effect.

  Not all of the methods apply to all games. Depending on what the game designer is trying to accomplish, some methods will be important and others will not make any sense at all.

As a consequence, the game metrics examples and ideas mentioned here will not be applicable to all games and situations. Each tracking metric is generally designed to answer one specific question or query. The process of data tracking should always start with a question, and then the data to track can be selected. Going the other way (from data to questions) can be just as interesting, but in game development economy and time constraints make such strategies unfeasible. Therefore, the metrics mentioned here are specific to a very narrow set of tasks. Hopefully, they can be useful to game developers or inspire other uses of game metrics.

1. Strength of feedback

 “Tighter: Multiple channels of aligned feedback such as colour, animation, sound, and touch that reinforce one another.”

 “Looser: One channel of feedback that is weakly evident. In multiplayer FPS games often the only sense that you have that another player is near comes from the faint patter of their footsteps. Expert players gain immense satisfaction from being able to predict the location of their opponent by combining knowledge of the levels with tiny hints of where they might be.”

Corresponding Metric: Alarm and reaction index 

If you want to track user response to feedback, there are several methods available depending on the desired feedback. For instance, if the player gets a warning and does not react to it, that indicates that the feedback from the warning is not strong enough. There are, of course, other factors that influence whether a player reacts to the alarm. Also, the player might acknowledge the alarm and choose to not react to it. But if there are more players who do not react than those who react, the feedback is probably not adequate and needs to be strengthened.

 

Hungry Shark Evolution pressure warning
Hungry Shark Evolution pressure warning

It is also interesting to track indicators with game metrics. If the game uses indicators to convey game status or objectives, it could be interesting to track how much the player reacts to them. For instance if there is a pointer on the screen to lead the player towards the game objective, it would make sense to track how much the player actually follows the pointer. Again, there are many factors affecting whether a player follows a pointer: for example, whether or not it is the only objective and what possibility the game offers for exploration. But if the game is designed so that the player follows something and the player more often than not does not follow that thing, it would indicate that the feedback of the indicator is not strong enough.

The problem with tracking for the effect of feedback is that, while game metrics are excellent at determining exactly what the players do and when they do it, metrics are not good at determining player intent – the why. These methods can work but the more complex the game, and the more options the player has, and so the more difficult it is to derive player intent from metrics. To fully understand player intent it is more useful to use qualitative research methods such as think-aloud play sessions and interviews.

2. Noisiness

“Tighter: A clear signal of effect that is related to the cause.”

“Looser: A multiplicity of conflicting, attention sapping signals, which are not related to cause. One of the critical skills in Jeff Minter’s Space Giraffe is learning to see through the visual noise of the psychedelic backgrounds.”

Corresponding Metric: Acoustic and visual clutter

Noise is strongly connected to strength of feedback: too strong feedback from too many sources will create noise. Therefore, when designing increased strength of feedback, designers should be aware that their solutions to weak feedback can create noise and counter-effect the purpose of the strength increase.

A way to measure noise is to track for how many sounds are being played at any one time, and the compared volume of each. If there are too many sounds playing at one time or at the same volume level, it will be more difficult for the player to distinguish between the sounds. When that happens, any meaning or feedback that sounds are conveying will be in danger of being misheard or misunderstood. To use this method it is necessary to distinguish between important and unimportant, as an explosion might be one of a thousand basic enemies dying, or the signal that the end boss is finally vulnerable for attack.

The same method can be used to track how many different things are being displayed on the screen at any one time. This method can also be used to enhance performance, to keep track of game progression, or to make sure that the game expands and speeds up at the desired rate.

3. Sensory type

“Tighter: Visually or tactile feedback is often more clearly perceived. Consider the many billions of dollars spent on improving visual feedback each year so that we can demonstrate the visceral impact of a player’s bullet on simulated flesh with ever greater fidelity. Tight visual feedback is highly functional; it communicates the effect to the player in an elegant efficient fashion. It is not just about making pretty pictures. In a recent update of Triple Town, we changed the colour scheme so that the background was the same general value as the foreground objects. The result was attractive, but players were pissed because the icons weren’t nearly as visible as before.

Looser: Auditory and smell are less clearly perceived. Not as much has been done here, but due to the looseness that comes with such systems it would seem that there are potential systems of mastery. It is perhaps ironic that most music games, a topic typically associated with auditory mastery, can be played with the sound turned off.”

Corresponding Metric: Finger screen blocking 

Deciding what is clearly visible for the player and what is not is at centre of game design, as computer games are perceived mainly visually. With mobile games there is an added challenge with the visual aspect, as the player’s main method of control is by touching the screen. That adds the aspect of what can’t the player see when the fingers are blocking part of the screen.

The visual design of games on mobile devices, especially on phones, becomes very important. It is possible to use game metrics to examine what the player can and cannot see while playing. Tracking which areas are being pressed will enable the creation of a heat map of what parts of the screen are covered and when. In the case of an action game where the player must hold the device in a certain way, it is possible to accurately extrapolate where the rest of the finger is based on the place of contact. By using metrics to track these factors both during development and after launch, it is possible to make a very accurate map of what parts of the screen are hidden from the player during play and for how long. Then the designer must make sure that the hidden parts of the screen do not contain any important or critical information. The data collected could also be used to determine if the game interface should be changed in the event that the game is played in an unexpected way.

4. Tapping existing mental models

“Tighter: Closely map the theme, feedback and system to existing mental models. Due to decades of exposure to pop culture, players know how zombies move and that they should be avoided. One means of quickly communicating the dozens of variables in a particular slow moving group of monsters is to label them ‘zombies’.”

“Looser: Step away from existing models and introduce the player to new systems that they’ve never experienced. Consider the metaphors involved in Tetris. Falling elements are something our brain can process as reasonably familiar. Tetriminos that you fit into lines that disappear to earn points while Russian music plays? That doesn’t fit any known metaphor that I know, yet it results in a great game.”

Corresponding Metric: User pattern recognition 

It is very difficult for metrics to measure whether a player recognizes game design elements, or “natural mapping” as Donald Norman calls it. The problem is, again, the “why” in elusive trough metrics. A player might very well recognize that red barrels usually blow up, but all the metrics can track is that the player shot the barrel. It is possible to track whether or not a player behaves as the designer expected. For instance, if there are also black barrels in the game, metrics can track whether the player mainly shoots them first or shoots them more than the red ones, or if the player continues shooting them after they have shot a red barrel.

This method only works when there are several similar game objects to compare, and game metrics is not the best way to examine if a game uses “natural mapping” effectively. The challenge of identifying and using well known and recognizable thought models falls on the designer.

5. Discreteness

“Tighter: Discrete states or low value numbers. Binary is the tightest. For example, recently we were playing with units moving at various speeds. By making them move a 1, 2, and 4 tiles/sec, it suddenly became very obvious to the player how each unit type was distinct. This is one of my favourite techniques for getting unruly systems under control.“

“Looser: Analogue values or very high value numbers. For example, in Angry Birds, you can give your bird a wide range of angles and velocities. This makes the results surprisingly uncertain. Think of how predictable (and boring) the game would be if you could only pick two distinct angles and velocities.”

Corresponding Metric: Player action / Player selection 

By “discreteness” Daniel Cook seems to mean a mix of player action distinctiveness and consistency, user agency and game elements. That is a somewhat complex collection of concepts to understand, and might be generally hard to track. But it is possible to look at the sub-categories one at the time.

For instance, by looking at user agency it is possible to track how many times the user employs the different actions available. This will reveal what actions the user prefers and will also help indicate what actions are redundant or unimportant. If for instance there are four different types of attacks and the players use only one of them and are still advancing in the game that indicates that the three other types of attack are redundant. The game then needs to be redesigned either by making the other types of attack necessary, by removing them or by modifying them. This kind of tracking is used to determine what classes, weapons and building types are most popular in games.

The race and class census of World of Warcraft
The race and class census of World of Warcraft

There are nine more techniques described in Daniel Cook’s post. They will be covered in the next two parts of this article.