Getting Started with Game Analytics: 3 Key Terms

There are two key terms in game analytics that everyone should know: telemetry and metric. These are sometimes used interchangeably but are in practice very different. Additionally, it is important to know that there are many different types or classes of telemetry-derived data sources and metrics that are used in game analytics and business intelligence for game development.

Telemetry

Telemetry is the collection of measurements over a distance. The term dates back to the 19^th century where the first data transmission circuits were developed, e.g. radio wave transmission. Game telemetry is the term we use for any source of data obtained over a distance, which is used in game development. There are many such sources, with some of the most popular being user telemetry, i.e. data collected from installed clients or servers about the behavior of users. For example, what items they purchase, how much they play and when, interactions with other users, etc.

Telemetry data are measures about the attributes of objects (or items). Object is a broad term that encompasses people, virtual objects, processes, systems or anything really that can be measured in one way or another, i.e. which has attributes. For example, the location of a player character as it navigates a 3D environment. In this case, the location is the attribute, the player character the object. Conversely, the length of customer service calls generated from a newly released patch in an MMORPG sees the length of the calls as the attribute of the customer service calls.

In order to work with telemetry data, they need to be operationalized, which means figuring out how to express them. For example, deciding that locational data about a mobile phone user should be organized as an integer with the unit in meters. Operationalizing attribute data in this way turns them into variables or features. In most cases, how to operationalize an attribute is straightforward, but it is important to make sure that similar attributes are operationalized in a standardized way – e.g. avoiding measuring some distances in “klicks” and others in “yards”. Finally, variables/features have a specific domain. The domain is the set of all possible values – defining the domain is essentially what operationalizing attribute data is all about. For example, a binary domain allows only two values (e.g. 0 or 1).

Metrics

Telemetry data are stored in a database format of some kind, which allows them to be transformed into interpretable measures. For example, “average level completion time” which is calculated based on data about session start, session end, level/-s played, levels completed yes/no, and similar pieces of raw data. Measures such as average weekly bug fix rate, daily revenue, daily active users (DAU), user lifetime value (LTV) etc. are called game metrics. Fundamentally, they are interpretable measures of something. Metrics can be based on simple aggregate values or the result of complex data mining procedures.

To take an example: telemetry data from a game like Medal of Honor could include data on the location of the player in the virtual environment, the weapons used, and information on whether every shot hits or misses, etc. These are different attributes, and they can be converted into variables/features such as “number of hits” or “number of misses” with a domain from 0-1000 (with 1000 being the biggest number of hits scored for a specific level). In turn, these can form the basis for metrcs, e.g. the total number of hits, total number of misses, or the hit/miss ratio for each level or map in the game (e.g. “hit/miss ratio is 1.2 on average for the map X”).

Game metrics are normally calculated as a function of something (a dimension). Time is a typical dimension, others include build (version), level, map, country, player, progression or similar. All metrics are bound to some sort of timeframe, and this will always be from a past period – we cannot (yet) collect telemetry from the future.

However, it is possible to run predictive analyzes based on historical data, which can generate metrics for future behavior, e.g. expected sales, expected churn rate, expected number of players, etc. However, these will always be based on predictions with a specific uncertainty attached, whereas collected telemetry data – if collected correctly – are facts.

The term game metrics is often used as a synonym for measures based on operationalized game telemetry data, but it is worth noting that a game metric does not need to be derived from telemetry data. The connection between telemetry and game metrics is commonly made in game development due to the inspiration of the use of the term “metric” in web analytics and mobile analytics, which have been among the primary inspirational sources for game analytics.

Three common classes of game metrics

Game metrics come in many shapes and sizes, but can be classified into three distinct categories that relate to how they are used in practice by a company:

1)     User metrics:

These are metrics related to the people, or users, who play games, from the dual perspective of them being either customers, i.e. sources of revenue or players, who behave in a particular way when interacting with games. The first perspective is used when calculating metrics related to revenue, e.g. average revenue per user (ARPU), daily active users (DAU) or when performing analyses related to revenue, e.g. churn analysis, customer support performance analysis or micro-transaction analysis. The second perspective is used for investigating how people interact with the actual game system and the components of it and with other players, i.e. focusing on in-game behavior. Examples of metrics are: total playtime per player, average number of in-game friends per player or average damage dealt per player; and common analyzes include time-spent analysis, trajectory analysis, or social networks analysis

2)     Performance metrics:

These are metrics related to the performance of the technical and software-based infrastructure behind a game, notably relevant for online or persistent games. Common performance metrics include the frame rate at which a game executes on a client hardware platform, or in the case of a game server, its stability. Performance metrics are also used when monitoring changing features or the impact of patches and updates on how well the client executes.

3)     Process metrics:

These are metrics related to the actual process of developing games. Game development is a creative process, which – similar to other creative areas in IT – has necessitated the use of agile development methods. In turn, this has prompted the development of ways of monitoring and measuring the development process. For example, by combining task size estimation with burn down charts, or measuring the average turnaround time of new content being delivered, type and effect of blocks to the development pipeline, and so forth. Similar to performance metrics, a number of process metrics and the associated management and monitoring methods are adopted and/or adapted from the methods and strategies in use outside the games sector.

In the next post, we provide suggestions for some of the best material out there on game analytics. Part one focuses on books and online writings