Uncovering Hidden States in Gaming Performance: A Neural Perspective

Cutting edge neuroscience shows that when humans perform repetitive tasks, they switch between mental states with unique profiles in decision-making, attention, brain activity, and overall performance. Understanding these shifts in mental states has profound implications for game design and player engagement, and can allow researchers to develop better adaptive game systems that respond to player needs in real-time, improving both gaming experiences and cognitive research methodologies. However, these mental states often reflect subtle shifts in behavioral and neural profiles that are challenging to observe using post-gameplay summary metrics or self-report data alone.

In our recent study published in the journal NeuroImage, led by Dr Vardal, we took an innovative approach to this issue by combining Hidden Markov Models (HMMs) with moment-to-moment behavioral data recorded from individuals playing a laboratory version of Tetris, and simultaneous neural activity recorded with magnetoencephalography (MEG). Our analysis provides compelling evidence that players transition through distinct states with unique profiles spanning behavior and neurophysiological markers of attention.

Decoding Player States Through Unsupervised Learning

We investigated whether distinct states could be inferred from gameplay behavior and whether these states would have a distinct neurophysiological correlate in brain activity relating to visual attention. To this end, we collected high-dimensional behavioral data from players engaged in a laboratory version of Tetris that records detailed moment-to-moment information about button inputs and the Tetris pile. Fitting HMMs to our rich behavioral data set, we first identified three dominant states:

Default State: Characterized by stable motor execution (sequences of button inputs) and moderate pile disarray, this state represented a baseline level of performance.

Opportunity State: Players in this state demonstrated high strategic potential, marked by increased well preparation (setting up high-scoring moves) and rapid decision-making.

Panic State: Defined by poor motor execution, rising messiness of the Tetris pile, and delayed decision-making, this state appeared to capture moments of cognitive overload or failure in planning.

Neural Signatures of Player States

To validate the identification of these states at the neural level, we then examined changes in alpha power in the occipital cortex - a well-established neural marker of visual attention. MEG recordings showed significant differences in occipital alpha power between states, with the highest alpha power observed in the panic state, followed by the default and opportunity states. This suggests that shifts in motor inputs and decision-making correspond to underlying shifts in visual attention.

Implications for Games Research and Beyond

The findings of this study carry broad implications for the study of cognition using digital games. First, they demonstrate the utility of unsupervised learning techniques in parsing dense moment-to-moment performance data, moving beyond traditional aggregate measures like total score or reaction time. Second, they provide a robust neurocognitive framework for understanding how engagement fluctuates dynamically during gameplay.

For researchers in game AI, interaction design, and cognitive science, these findings underscore the potential of combining behavioral analytics with neural data to gain deeper insights into player experience. Future applications could extend to adaptive gaming systems that respond dynamically to shifts in player engagement, enhancing both entertainment value and learning outcomes both in recreational games and games tailor-made for educational purposes.

The paper is available open access here: https://doi.org/10.1016/j.neuroimage.2025.121134