Motion cues for illustration of skeletal motion capture data

ABSTRACT

There are many applications for which it is necessary to illustrate motion in a static image using visual cues which do not represent a physical entity in the scene, yet are widely understood to convey motion. For example, consider the task of illustrating the desired movements for exercising, dancing, or a given sport technique. Traditional artists have developed techniques to specify desired movements precisely (technical illustrators) and suggest motion (cartoonists) in an image.

In this paper, we present an interactive system to synthesize a 2D image of an animated character by generating artist-inspired motion cues derived from 3D skeletal motion capture data. The primary cues include directed arrows, noise waves, and stroboscopic motion. First, the user decomposes the animation into short sequences containing individual motions which can be represented by visual cues. The system then allows the user to determine a suitable viewpoint for illustrating the movement, to select the proper level in the joint hierarchy, as well as to fine-tune various controls for the depiction of the cues themselves. While the system does provide adapted default values for each control, extracted from the motion capture data, it allows fine-tuning for greater expressiveness. Moreover, these cues are drawn in real time, and maintain a coherent display with changing viewpoints.

We demonstrate the benefit of our interactive system on various motion capture sequences.

REFERENCES

• 1: Aseem Agarwala , Mira Dontcheva , Maneesh Agrawala , Steven Drucker , Alex Colburn , Brian Curless , David Salesin , Michael Cohen, Interactive digital photomontage, ACM Transactions on Graphics (TOG), v.23 n.3, August 2004
• 2: Maneesh Agrawala , Doantam Phan , Julie Heiser , John Haymaker , Jeff Klingner , Pat Hanrahan , Barbara Tversky, Designing effective step-by-step assembly instructions, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 3: Jackie Assa , Yaron Caspi , Daniel Cohen-Or, Action synopsis: pose selection and illustration, ACM Transactions on Graphics (TOG), v.24 n.3, July 2005
• 4: Jernej Barbič , Alla Safonova , Jia-Yu Pan , Christos Faloutsos , Jessica K. Hodgins , Nancy S. Pollard, Segmenting motion capture data into distinct behaviors, Proceedings of Graphics Interface 2004, p.185-194, May 17-19, 2004, London, Ontario, Canada
• 5: Matthew Brand , Aaron Hertzmann, Style machines, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.183-192, July 2000 [doi>10.1145/344779.344865]
• 6: Christoph Bregler , Lorie Loeb , Erika Chuang , Hrishi Deshpande, Turning to the masters: motion capturing cartoons, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 7: Gabriel J. Brostow , Irfan Essa, Image-based motion blur for stop motion animation, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.561-566, August 2001 [doi>10.1145/383259.383325]
• 8: Stephen Chenney , Mark Pingel , Rob Iverson , Marcin Szymanski, Simulating cartoon style animation, Proceedings of the 2nd international symposium on Non-photorealistic animation and rendering, June 03-05, 2002, Annecy, France [doi>10.1145/508530.508553]
• 9: Collomosse, J. P., and Hall, P. M. 2005. Motion analysis in video: Dolls, dynamic cues and modern art. In Proc. International Conference on Vision, Video and Graphics, 109--116.
• 10: J. P. Collomosse , D. Rowntree , P. M. Hall, Rendering cartoon-style motion cues in post-production video, Graphical Models, v.67 n.6, p.549-564, November 2005 [doi>10.1016/j.gmod.2004.12.002]
• 11: Collomosse, J. P. 2004. Higher Level Techniques for the Artistic Rendering of Images and Video. PhD thesis, University of Bath, United Kingdom.
• 12: Cutting, J. E. 2002. Representing motion in a static image: constraints and parallels in art, science, and popular culture. Perception 31, 10, 1165--1193.
• 13: Dony, R., Mateer, J., and Robinson, J. 2005. Techniques for automated reverse storyboarding. IEE Journal of Vision, Image & Signal Processing 152, 4, 425--436.
• 14: Eisner, W. 1985. Comics And Sequential Art. Poorhouse Press.
• 15: Pascal Glardon , Ronan Boulic , Daniel Thalmann, Robust on-line adaptive footplant detection and enforcement for locomotion, The Visual Computer: International Journal of Computer Graphics, v.22 n.3, p.194-209, March 2006 [doi>10.1007/s00371-006-0376-9]
• 16: Dan B Goldman , Brian Curless , David Salesin , Steven M. Seitz, Schematic storyboarding for video visualization and editing, ACM Transactions on Graphics (TOG), v.25 n.3, July 2006
• 17: Michael Haller , Christian Hanl , Jeremiah Diephuis, Non-photorealistic rendering techniques for motion in computer games, Computers in Entertainment (CIE), v.2 n.4, October 2004 [doi>10.1145/1037851.1037869]
• 18: Hart, J. 1999. The Art of the Storyboard: storyboarding for file, TV and animation. Focal Press.
• 19: Julie Heiser , Doantam Phan , Maneesh Agrawala , Barbara Tversky , Pat Hanrahan, Identification and validation of cognitive design principles for automated generation of assembly instructions, Proceedings of the working conference on Advanced visual interfaces, May 25-28, 2004, Gallipoli, Italy [doi>10.1145/989863.989917]
• 20: Siu Chi Hsu , Irene H. H. Lee, Drawing and animation using skeletal strokes, Proceedings of the 21st annual conference on Computer graphics and interactive techniques, p.109-118, July 1994 [doi>10.1145/192161.192186]
• 21: Hutchinson, A. 1984. Dance Notation: the process of recording movement on paper. Dance Horizons.
• 22: Joshi, A., and Rheingans, P. 2005. Illustration-inspired techniques for visualizing time-varying data. In Proc. IEEE Visualization '05, 679--686.
• 23: Kawagishi, Y., Hatsuyama, K., and Kondo, K. 2003. Cartoon blur: Non-photorealistic motion blur. In Proc. Computer Graphics International '03, 276--281.
• 24: Lucas Kovar , Michael Gleicher, Automated extraction and parameterization of motions in large data sets, ACM Transactions on Graphics (TOG), v.23 n.3, August 2004
• 25: Lucas Kovar , John Schreiner , Michael Gleicher, Footskate cleanup for motion capture editing, Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation, July 21-22, 2002, San Antonio, Texas [doi>10.1145/545261.545277]
• 26: Adam Lake , Carl Marshall , Mark Harris , Marc Blackstein, Stylized rendering techniques for scalable real-time 3D animation, Proceedings of the 1st international symposium on Non-photorealistic animation and rendering, p.13-20, June 05-07, 2000, Annecy, France [doi>10.1145/340916.340918]
• 27: Ce Liu , Antonio Torralba , William T. Freeman , Frédo Durand , Edward H. Adelson, Motion magnification, ACM Transactions on Graphics (TOG), v.24 n.3, July 2005
• 28: Maic Masuch , Stefan Schlechtweg , Ronny Schulz, Speedlines: depicting motion in motionless pictures, ACM SIGGRAPH 99 Conference abstracts and applications, p.277, August 08-13, 1999, Los Angeles, California, United States [doi>10.1145/311625.312369]
• 29: McCloud, S. 1993. Understanding Comics-The Invisible Art. Kitchen Sink Press.
• 30: Tomoaki Moriya , Tokiichiro Takahashi , Akimichi Tanaka, Non-photorealistic rendering technique for depicting 3D objects motion, ACM SIGGRAPH 2006 Research posters, July 30-August 03, 2006, Boston, Massachusetts [doi>10.1145/1179622.1179754]
• 31: Nienhaus, M., and Döllner, J. 2003. Dynamic glyphs - Depicting dynamics in images of 3D scenes. In Proc. Symposium on Smart Graphics '03, 102--111.
• 32: Marc Nienhaus , Jurgen Dollner, Depicting Dynamics Using Principles of Visual Art and Narrations, IEEE Computer Graphics and Applications, v.25 n.3, p.40-51, May 2005 [doi>10.1109/MCG.2005.53]
• 33: Syoichi Obayashi , Kunio Kondo , Toshihiro Konma , Ken-ichi Iwamoto, Non-photorealistic motion blur for 3D animation, ACM SIGGRAPH 2005 Sketches, July 31-August 04, 2005, Los Angeles, California [doi>10.1145/1187112.1187224]
• 34: Ramesh Raskar , Remo Ziegler , Thomas Willwacher, Cartoon dioramas in motion, Proceedings of the 2nd international symposium on Non-photorealistic animation and rendering, June 03-05, 2002, Annecy, France [doi>10.1145/508530.508532]
• 35: Matthew Thorne , David Burke , Michiel van de Panne, Motion doodles: an interface for sketching character motion, ACM Transactions on Graphics (TOG), v.23 n.3, August 2004
• 36: Edward R. Tufte, Visual explanations: images and quantities, evidence and narrative, Graphics Press, Cheshire, CT, 1997
• 37: Jue Wang , Steven M. Drucker , Maneesh Agrawala , Michael F. Cohen, The cartoon animation filter, ACM Transactions on Graphics (TOG), v.25 n.3, July 2006
• 38: Andy Wheeler , Joanne Thiel , Greg Hart, Cartoon motion blur for 3D animation, ACM SIGGRAPH 2006 Sketches, July 30-August 03, 2006, Boston, Massachusetts [doi>10.1145/1179849.1179968]
• 39: Williams, R. 2001. The Animator's Survival Kit. Faber and Faber Limited.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Animation

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.3 Picture/Image Generation
          Subjects: Viewing algorithms; Display algorithms