Summarize the mainstream Games User Research Methods here for future reference:
Behavioral Observation
This technique can be as simple as looking over the shoulders of individuals when they are playing a game, or as complex as high-definition video recording of players from different angles in a natural gaming environment, such as a living room. Because of its relative simplicity and the actionable results that it produces, behavioral observation is often regarded as the most valuable GUR technique. Direct
observation is especially valuable to the designers of a game.
Think-Aloud Protocol
The think-aloud protocol was developed in usability testing for products (Lewis 1982) and later made its way from interaction design to GUR. It could be seen as a natural expansion of behavioral observation, because it introduces player narrative to the observations.
Lewis C (1982) Using the thinking-aloud method in cognitive interface design. Vol technical re
port. IBM TJ Watson Research Center, Yorktown Heights
Interviews
Interviews are a common method of qualitative subjective inquiry, providing direct insight into the player experience. Much of the quality of the data collected during an interview session depends on the skill of the interviewer. Interviews allow a greater degree of depth when analyzing player opinions, emotions, and reactions. However, data collection and analysis can be time-consuming.
Questionnaires
Questionnaires or surveys are a common GUR method, because they allow collecting large volumes of self-report data simultaneously from many different players. Surveys are usually used to get insights into value judgments about gameplay moments.
A questionnaire can be delivered to players directly after a gameplay event (e.g., making a choice in the game) or after a gameplay session, so that experience is still present in memory. This is arguably less biased than, for example, recalling an experience during an interview session post-gameplay. Often, gameplay questionnaires feature a Likert-scale (Likert 1932) rating type of a gameplay interaction.
Focus Groups
Focus groups are another method of qualitative inquiry, where a group of players is gathered to talk about their opinions, beliefs, and attitudes towards the game (Poels et al. 2007). The group has complete freedom to talk about their likes and dislikes, but usually a moderator is present to facilitate the discussion and lead the group toward a topic of interest. Again, I will not go into depth about focus group methodology here, since Poels et al. (2010) describe a focus group study about qualitative insights into postgame experiences, which is an excellent overview of the methodology.
Poels K, et al. (2010) Digital games, the Aftermath: qualitative insights into postgame experiences. Evaluating user experience in games. Springer London, 149–163. doi:10.1007/978-1- 84882-963-3_9
Heuristic Evaluation
Heuristic evaluation is a method stemming from usability research (Nielsen and Molich 1990). In this context, the evaluation consists of judging how an interface complies with recognized usability principles, which are called “heuristics”. It is known as a discount usability method, because it is cheap to conduct and can yield significant actionable results for a game. The method includes GUR experts playing a game and evaluating it based on a set of criteria. So, in a way, a heuristic evaluation can be likened to a tightly structured game review. After the expert has played the game, they give feedback on whether the game fits a certain playability guideline, or “heuristic”, and what problems might arise from non-compliance.
Nielsen J, Molich R (1990) Heuristic evaluation of user interfaces. Paper presented at the Proceedings of the SIGCHI conference on Human factors in computing systems: Empowering people, Seattle, Washington, United States doi:10.1145/97243.97281
Game Metrics
Game analytics and metrics are undeniably a recent trend within the GUR community (Seif El-Nasr et al. 2013). Chapter 7 by Drachen in this book focuses on game metrics or more specifically on a part of game metrics that is also referred to as game telemetry (because data are often collected after delivery of the game and not always during testing). The term “game metrics” generally refers to the process of logging player interactions, positions in the game world, camera angles, and all data that relates to the gameplay interaction process in the game. To do this, a programmer has to define appropriate hooks in the game engine that allow the logging of all this data. A huge advantage of metrics is the large amount of data being collected. This can also prove detrimental, as this data has to be analyzed quickly and accurately, which often means that a GUR professional has to use visualization software to make sense of such vast collections of data. The potential of this GUR methodology is promising, because it can measure key gameplay events (e.g., player deaths) and their surrounding circumstances in detail. By integrating these log files with synchronized physiological sensor data, we can craft a more complete picture of player experience.
El-Nasr M, Drachen A, Canossa A (2013) Game analytics: Maximizing the value of player data. Springer London. doi:10.1007/978-1-4471-4769-5
Physiological Game Evaluation
Psychophysiological measures in games can be used in conjunction with other measures to establish relationships between player experience facets and physiological responses. Physiological metrics can be recorded continuously during a games user research session without interrupting play. This makes these methods less obtrusive than subjective measures that either break the experience (by interrupting and prompting with questions) or introduce memory bias (by asking questions about the game in retrospect). The only downside of physiological metrics is that the player has to wear sensors, and that some might find this intrusive (although based on personal experience, many players forget that they are wearing sensors a few minutes into the game).
However, recent research shows that a combined method with above techniques should be more efficient and comprehensive in revealing the actual PX.
Charoenpruksachat, Alongkorn, and Pattama Longani. “Comparative Study of Usability Evaluation Methods on a Hyper Casual Game.” 2021 Joint International Conference on Digital Arts, Media and Technology with ECTI Northern Section Conference on Electrical, Electronics, Computer and Telecommunication Engineering, 2021.
Johnson, Daniel M., et al. “Validation of Two Game Experience Scales: The Player Experience of Need Satisfaction (PENS) and Game Experience Questionnaire (GEQ).” International Journal of Human-Computer Studies / International Journal of Man-Machine Studies, vol. 118, 2018, pp. 38–46.
Tan, Chek Tien, et al. “Combining Think-Aloud and Physiological Data to Understand Video Game Experiences.” Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2014, pp. 381–390.
Scozzi, Monica Visani, et al. “A Mixed Method Approach for Evaluating and Improving the Design of Learning in Puzzle Games.” Proceedings of the Annual Symposium on Computer-Human Interaction in Play, 2017, pp. 217–228.
Toth, Adam J., et al. “Beyond Action Video Games: Differences in Gameplay and Ability Preferences among Gaming Genres.” Entertainment Computing, vol. 38, 2021, p. 100408.
Nacke, Lennart E. “Games User Research and Physiological Game Evaluation.” Game User Experience Evaluation, 2015, pp. 63–86.