Evaluation of real-world and computer-generated stylized facial expressions

ABSTRACT

The goal of stylization is to provide an abstracted representation of an image that highlights specific types of visual information. Recent advances in computer graphics techniques have made it possible to render many varieties of stylized imagery efficiently making stylization into a useful technique, not only for artistic, but also for visualization applications. In this paper, we report results from two sets of experiments that aim at characterizing the perceptual impact and effectiveness of three different stylization techniques in the context of dynamic facial expressions. In the first set of experiments, animated facial expressions are stylized using three common techniques (brush, cartoon, and illustrative stylization) and investigated using different experimental measures. Going beyond the usual questionnaire approach, these experiments compare the techniques according to several criteria ranging from subjective preference to task-dependent measures (such as recognizability, intensity) allowing us to compare behavioral and introspective approaches. The second set of experiments use the same stylization techniques on real-world video sequences in order to compare the effect of stylization on natural and artificial stimuli. Our results shed light on how stylization of image contents affects the perception and subjective evaluation of both real and computer-generated facial expressions.

REFERENCES

• 1: Adolphs, R. 2002. Recognizing emotions from facial expressions: Psychological and neurological mechanisms. Behavioral and Cognitive Neuroscience Reviews 1, 1, 21--61.
• 2: Maneesh Agrawala , Chris Stolte, Rendering effective route maps: improving usability through generalization, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.241-249, August 2001 [doi>10.1145/383259.383286]
• 3: Breidt, M., Wallraven, C., Cunningham, D. W., and Bülthoff, H. H. 2003. Facial animation based on 3d scans and motion capture. SIGGRAPH '03 Sketches & Applications.
• 4: Bull, P. 2001. State of the art: Nonverbal communication. The Psychologist 14, 644--647.
• 5: Douglas W. Cunningham , Martin Breidt , Mario Kleiner , Christian Wallraven , Heinrich H. Bülthoff, How Believable Are Real Faces? Towards a Perceptual Basis for Conversational Animation, Proceedings of the 16th International Conference on Computer Animation and Social Agents (CASA 2003), p.23, May 08-09, 2003
• 6: Douglas W. Cunningham , Manfred Nusseck , Christian Wallraven , Heinrich H. Bülthoff, The role of image size in the recognition of conversational facial expressions: Research Articles, Computer Animation and Virtual Worlds, v.15 n.3-4, p.305-310, July 2004 [doi>10.1002/cav.v15:3/4]
• 7: Douglas W. Cunningham , Mario Kleiner , Christian Wallraven , Heinrich H. Bülthoff, Manipulating Video Sequences to Determine the Components of Conversational Facial Expressions, ACM Transactions on Applied Perception (TAP), v.2 n.3, p.251-269, July 05 [doi>10.1145/1077399.1077404]
• 8: Doug DeCarlo , Anthony Santella, Stylization and abstraction of photographs, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 9: Ekman, P. 1972. Universal and Cultural Differences in Facial Expressions of Emotion. University of Nebraska Press, Lincoln, NB. 207--283.
• 10: Ferwerda, J. 2003. Three varieties of realism in computer graphics. In Proc. of SPIE Human Vision and Electronic Imaging. 290--297.
• 11: Jan Fischer , Dirk Bartz , Wolfgang Straβer, Artistic reality: fast brush stroke stylization for augmented reality, Proceedings of the ACM symposium on Virtual reality software and technology, November 07-09, 2005, Monterey, CA, USA [doi>10.1145/1101616.1101649]
• 12: Fischer, J., Bartz, D., and Strassr, W. 2005b. Illustrative display of hidden iso-surface structures. In Proc. of IEEE Visualization. 663--670.
• 13: Fischer, J., Cunningham, D., Bartz, D., Wallraven, C., Bülthoff, H., and Strassr, W. 2006a. Measuring the discernability of virtual objects in conventional and stylized augmented reality. In Eurographics Symposium on Virtual Environments (EGVE).
• 14: Fischer, J., Eichler, M., Bartz, D., and Strassr, W. 2006b. Model-based hybrid tracking for medical augmented reality. In Eurographics Symposium on Virtual Environments (EGVE).
• 15: Bert Freudenberg , Maic Masuch , Thomas Strothotte, Real-time halftoning: a primitive for non-photorealistic shading, Proceedings of the 13th Eurographics workshop on Rendering, June 26-28, 2002, Pisa, Italy
• 16: Amy Ashurst Gooch , Peter Willemsen, Evaluating space perception in NPR immersive environments, Proceedings of the 2nd international symposium on Non-photorealistic animation and rendering, June 03-05, 2002, Annecy, France [doi>10.1145/508530.508549]
• 17: Bruce Gooch , Erik Reinhard , Amy Gooch, Human facial illustrations: Creation and psychophysical evaluation, ACM Transactions on Graphics (TOG), v.23 n.1, p.27-44, January 2004 [doi>10.1145/966131.966133]
• 18: Paul Haeberli, Paint by numbers: abstract image representations, Proceedings of the 17th annual conference on Computer graphics and interactive techniques, p.207-214, September 1990, Dallas, TX, USA
• 19: Kleiner, M., Wallraven, C., and Bülthoff, H. 2004. The mpi videolab - a system for high quality synchronous recording of video and audio from multiple viewpoints. Tech. Rep. 123. May.
• 20: Peter Litwinowicz, Processing images and video for an impressionist effect, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.407-414, August 1997 [doi>10.1145/258734.258893]
• 21: Anthony Santella , Doug DeCarlo, Visual interest and NPR: an evaluation and manifesto, Proceedings of the 3rd international symposium on Non-photorealistic animation and rendering, June 07-09, 2004, Annecy, France [doi>10.1145/987657.987669]
• 22: William A. Stokes , James A. Ferwerda , Bruce Walter , Donald P. Greenberg, Perceptual illumination components: a new approach to efficient, high quality global illumination rendering, ACM Transactions on Graphics (TOG), v.23 n.3, August 2004
• 23: Thomas Strothotte , Stefan Schlechtweg, Non-photorealistic computer graphics: modeling, rendering, and animation, Morgan Kaufmann Publishers Inc., San Francisco, CA, 2002
• 24: C. Tomasi , R. Manduchi, Bilateral Filtering for Gray and Color Images, Proceedings of the Sixth International Conference on Computer Vision, p.839, January 04-07, 1998
• 25: Christian Wallraven , Douglas W. Cunningham , Martin Breidt , Heinrich H. Bülthoff, View dependence of complex versus simple facial motions, Proceedings of the 1st Symposium on Applied perception in graphics and visualization, August 07-08, 2004, Los Angeles, California [doi>10.1145/1012551.1012603]
• 26: Christian Wallraven , Martin Breidt , Douglas W. Cunningham , Heinrich H. Bülthoff, Psychophysical evaluation of animated facial expressions, Proceedings of the 2nd symposium on Applied perception in graphics and visualization, August 26-28, 2005, A Coroña, Spain [doi>10.1145/1080402.1080405]
• 27: Holger Winnemöller , Sven C. Olsen , Bruce Gooch, Real-time video abstraction, ACM SIGGRAPH 2006 Papers, July 30-August 03, 2006, Boston, Massachusetts
• 28: Yip, A. W. and Sinha, P. 2002. Contribution of color to face recognition. Perception 31, 8, 995--1003.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Animation

Additional Classification:
  J. Computer Applications
  J.4 SOCIAL AND BEHAVIORAL SCIENCES
      Subjects: Psychology

General Terms:
Experimentation

Keywords:
Evaluation of facial animations, avatar, facial expressions, perceptually adaptive graphics, psychophysics, stylization