O' game, can you feel my frustration?: improving user's gaming experience via stresscam

ABSTRACT

One of the major challenges of video game design is to have appropriate difficulty levels for users in order to maximize the entertainment value of the game. Game players may lose interests if a game is either too easy or too difficult. This paper presents a novel methodology to improve user's experience in computer games by automatically adjusting the level of the game difficulty. The difficulty level is computed from measurements of the facial physiology of the players at a distance. The measurements are based on the assumption that the players' performance during the game-playing session alters blood flow in the supraorbital region, which is an indirect measurement of increased mental activities. This alters heat dissipation, which can be monitored in a contact-free manner through a thermal imaging-based stress monitoring and analysis system, known as StressCam.

In this work, we investigated on two primary objectives: (1) the feasibility of utilizing the facial physiology in automatically adjusting the difficulty level of the game and (2) the capability of the automatic difficulty level adjustment in improving game players' experience. We employed and extended a XNA video game for this study, and performed an in-depth, comparative usability evaluation on it. Our results show that the automatic difficulty adjustable system successfully maintains game players' interests and substantially outperforms traditional fixed-difficulty mode games. Although a number of issues of this preliminary study remain to be investigated further, this research opens a new direction that utilizes non-contact stress measurements for monitoring and further enhancing a variety of user-centric, interactive entertainment activities.

References

1. Anderson, C.A. and Dill, K.E. Video games and aggressive thoughts, feelings, and behavior in the laboratory and in life. Journal of Personality and Social Psychology, 78, (2000), 772--790.
2. Anderson, C.A. and Bushman, B.J. Effects of violent video games on aggressive behavior, aggressive cognition, aggressive affect, physiological arousal, and prosocial behavior: A meta-analytic review of the scientific literature. Psychological Science, 12, 5 (2001), 353--359.
3. Anderson, C.A. An update on the effects of playing violent video games. Journal of Adolescence, 27, 1 (2004), 113--122.
4. Sherry, J.L. The effects of violent video games on aggression: A meta-analysis. Human Communication Research, 27, 3(2001), 409--431.
5. Ballard, M.E. and Weist, J.R. Mortal Kombat: The effects of violent video game play on males' hostility and cardiovascular responding. Journal of Applied Social Psychology, 26, (1996), 717--730.
6. Carnagey, N.L. and Anderson, C.A. The effects of reward and punishment in violent video games on aggressive affect, cognition, and behavior. Psychological Science, 16, 11 (2005), 882--889.
7. Zellweger, P.T., Bouvin, N.O., Jehøj, H., and Mackinlay, J.D. Fluid Annotations in an Open World. Proc. Hypertext 2001, ACM Press (2001), 9--18.
8. Scott, D. The effect of video games on feelings of aggression. The Journal of Psychology, 129, (1995), 121--132.
9. Ravaja, N., Saari, T., Salminen, M., Laarni, J., Holopainen, J., and Jarvinen A. Emotional Response Patterns and Sense of Presence during Video Games: Potential Criterion Variables for Game Design. NordiCHI '04, (2004), 339--347.
10. Ravaja, N., Saari, T., Turpeinen, M., Laarni, J., Salminen, M., and Kivikangas, M. Spatial Presence and Emotions during Video Game Playing: Does It Matter with Whom You Play? Presence, 15, 4(2006).
11. Nosu, K., Kurokawa, T.A., Horita, H., Ohhazama, Y., and Takeda, K. Real Time Emotion-Diagnosis of Video Game Players from Their Facial Expressions and Its Applications to Voice Feed-Backing to Game Players. Proceedings of the 6th International Conference on Machine Learning and Cybernetics, (2007), 2208--2212.
12. Schneider, E.F., Lang, A., Shin, M. Bradley, S.D. Death with a story: How story impacts emotional, motivational, and physiological responses to first-person shooter video games. Human Communication Research, 30, 3 (2004), 361--375.
13. Nenonen, V., Lindblad, A., Hakkinen, V., Laitinen, T., Jouhtio, M., and Hamalainen, P. Using Heart Rate to Control an Interactive Game. CHI 2007 Proceedings - Games, (2007).
14. Kim, J.H., Gunn, D.V., Schuh, E., Philips, B.C., Pagulayan, R.J., and Wixon, D. Tracking Real-Time User Experience (TRUE): A comprehensive instrumentation solution for complex systems. CHI 2008 Proceedings - Data Collection, (2008).
15. Lin, T., Omata, M., Hu, W., and Imamiya, A. Do physiological data relate to traditional usability indexes?. Proceedings of the 17th Australia Conference on Computer-Human interaction: Citizens online: Considerations For Today and the Future, Canberra, Australia, November 21 - 25, 2005.
16. Lazzaro, N. Why We Play Games: Four Keys to Move Emotion Without Story. XEODesign, 5273 College Ave. Suite 201, Oakland, California, 94618, http://www.xeodesign.com.
17. Puri, C., Olson, L., Pavlidis, I., Levine, J., and Starren, J. StressCam: Non-contact Measurement of Users' Emotional States through Thermal Imaging. CHI 2005 - Late Breaking Results: Posters, (2004).
18. Zhu Z., Tsiamyrtzis P., and Pavlidis I.,"The segmentation of the supraorbital vessels in thermal imagery, in Proceedings of the 5th IEEE International Conference on Advanced Video and Signal Based Surveillance, Santa Fe, New Mexico, September 1-3, 2008.
19. Pavlidis, I., Dowdall, J., Sun, N., Puri, C., Fei, J., and Garbey, M. Interacting with human physiology. Computer Vision and Image Understanding, 108, (2007), 150--170.
20. Palvidis, I. and Levine, J. Thermal image analysis for polygraph testing. IEEE Engineering in Medicine and Biology Magazine, 21, 6 (2002), 56--64.
21. Dowdall, J., Pavlidis, I., Tsiamyrtzis, P. Coalitional tracking. Computer Vision and Image Understanding. vol. 106, no. 2-3, May-June (2007), 205--219.
22. Tsiamyrtzis, P., Dowdall, J., Shastri, D., Pavlidis, I., Frank, M., and Ekman, P. Imaging facial physiology for the detection of deceit. International Journal of Computer Vision 71, 2, 10 (2006), 197--214.
23. Yamakoshi, T., Yamakoshi, K., Tanaka, S., Nogawa, M., Shibata, M., Sawada, Y., Rolfe, P., Hirose, Y. A Preliminary Study on Driver's Stress Index Using a New Method Based on Differential Skin Temperature Measurement. In Proc of IEEE Conf Eng Med Biol Soc. (2007), 722--725.
24. Yamaguchi, M., Wakasugi, J., Sakakima, J. Evaluation of Driver Stress using Biomarker in Motor-vehicle Driving Simulator. In Proc IEEE Conf. Eng Med Biol Soc. (2006), 1834--1837.
25. Healey, J.A., Picard, R.W. Detecting stress during real-world driving tasks using physiological sensors. Intelligent Transportation Systems, IEEE Transactions on. June (2005), 156--166. Volume: 6, Issue: 2
26. FLIR Systems, 70 Castilian Dr., Goleta, California 93117, http://www.flir.com.
27. Quickset International, 3650 Woodhead Drive, Northbrook, IL 60062, http://www.quickset.com
28. SBIR Inc, 30 South Calle Cesar Chavez, Santa Barbara, CA 9310, http://www.sbir.com
29. Moxham, B. J., Krish C., Berkovitz B., Alusi G., and Cheeseman T., Interactive head&neck (CD--ROM), Primal Pictures, December 2002.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • HCI design and evaluation methods

    • Interaction devices

      • Touch screens

    • Interaction paradigms

  • Interaction design

    • Interaction design process and methods

      • Interface design prototyping