skip to main content
10.1145/1457515.1409077acmconferencesArticle/Chapter ViewAbstractPublication Pagessiggraph-asiaConference Proceedingsconference-collections
research-article

Video puppetry: a performative interface for cutout animation

Published: 01 December 2008 Publication History

Abstract

We present a video-based interface that allows users of all skill levels to quickly create cutout-style animations by performing the character motions. The puppeteer first creates a cast of physical puppets using paper, markers and scissors. He then physically moves these puppets to tell a story. Using an inexpensive overhead camera our system tracks the motions of the puppets and renders them on a new background while removing the puppeteer's hands. Our system runs in real-time (at 30 fps) so that the puppeteer and the audience can immediately see the animation that is created. Our system also supports a variety of constraints and effects including articulated characters, multi-track animation, scene changes, camera controls, 2 1/2-D environments, shadows, and animation cycles. Users have evaluated our system both quantitatively and qualitatively: In tests of low-level dexterity, our system has similar accuracy to a mouse interface. For simple story telling, users prefer our system over either a mouse interface or traditional puppetry. We demonstrate that even first-time users, including an eleven-year-old, can use our system to quickly turn an original story idea into an animation.

Supplementary Material

MOV File (a124-barnes-mp4_hi.mov)

References

[1]
Agarwala, A., Hertzmann, A., Salesin, D. H., and Seitz, S. M. 2004. Keyframe-based Tracking for Rotoscoping and Animation. ACM Trans. Graphics (Proc. SIGGRAPH), Vol. 23, No. 3, 584--591.
[2]
Agarwala, A. 2002. SnakeToonz: A Semi-Automatic Approach to Creating Cel Animation from Video. In Proc. NPAR, 139--146.
[3]
Alvarado, C., and Davis, R. 2001. Resolving Ambiguities to Create a Natural Sketch Based Interface. In Proc. IJCAI, 1365--1371.
[4]
Baecker, R. 1969. Picture-Driven Animation. In Proc. Spring Joint Computer Conference, 273--288.
[5]
Bousseau, A., Neyret, F., Thollot, J., and Salesin, D. 2007. Video Watercolorization using Bidirectional Texture Advection. ACM Trans. Graphics (Proc. SIGGRAPH), Vol. 26, No. 3, 104: 1--104: 7.
[6]
Collomosse, J., and Hall, P. 2005. Video Paintbox: The Fine Art of Video Painting. Computers & Graphics, Vol. 29, No. 6, 862--870.
[7]
Collomosse, J., and Hall, P. 2006. Video Motion Analysis for the Synthesis of Dynamic Cues and Futurist Art. Graphical Models, Vol. 68, No. 5--6, 402--414.
[8]
Davis, J., Agrawala, M., Chuang, E., Popović, Z., and Salesin, D. 2003. A Sketching Interface for Articulated Figure Animation. In Proc. SCA, 320--328.
[9]
Davis, R. C., Colwell, B., and Landay, J. A. 2008. K-Sketch: A "Kinetic" Sketch Pad for Novice Animators. In Proc. SIGCHI, 413--422.
[10]
Dontcheva, M., Yngve, G., and Popović, Z. 2003. Layered acting for character animation. ACM Trans. Graphics (Proc. SIGGRAPH), Vol. 22, No. 3, 409--416.
[11]
Fiala, M. 2005. ARTag, a Fiducial Marker System Using Digital Techniques. Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, Vol. 2, 590--596.
[12]
Gottschalk, S., Lin, M. C., and Manocha, D. 1996. OBB-Tree: A Hierarchical Structure for Rapid Interference Detection. In Proc. SIGGRAPH, 171--180.
[13]
Horn, B. K. P. 1986. Closed-Form Solution of Absolute Orientation using Unit Quaternions. Journal of the Optical Society of America, Vol. 4, 629--642.
[14]
Igarashi, T., Moscovich, T., and Hughes, J. F. 2005. As-Rigid-as-Possible Shape Manipulation. ACM Trans. Graphics (Proc. SIGGRAPH), Vol. 24, No. 3, 1134--1141.
[15]
Ishii, H., and Ullmer, B. 1997. Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms. In Proc. SIGCHI, 234--241.
[16]
Kim, J., Seitz, S., and Agrawala, M. 2004. Video-based Document Tracking: Unifying Your Physical and Electronic Desktops. In Proc. UIST, 99--107.
[17]
Knep, B., Hayes, C., Sayre, R., and Williams, T. 1995. Dinosaur Input Device. In Proc. SIGCHI, 304--309.
[18]
LaViola Jr, J., and Zeleznik, R. 2004. MathPad 2: A System for the Creation and Exploration of Mathematical Sketches. ACM Trans. Graphics (Proc. SIGGRAPH), Vol. 23, No. 3, 432--440.
[19]
Lee, G., Kim, G., and Billinghurst, M. 2005. Immersive authoring: What You eXperience Is What You Get (WYXIWYG). Communications of the ACM, Vol. 48, No. 7, 76--81.
[20]
Lowe, D. G. 1999. Object Recognition from Local Scale-Invariant Features. In International Conference on Computer Vision, 1150--1157.
[21]
Milgram, P., and Kishino, F. 1994. A Taxonomy of Mixed Reality Visual Displays. IEICE Transactions on Information and Systems, Vol. 77, No. 12, 1321--1329.
[22]
Moscovich, T., and Hughes, J. 2001. Animation Sketching: An Approach to Accessible Animation. Tech. Rep. CS04-03, Brown University CS Department.
[23]
Oore, S., Terzopoulos, D., and Hinton, G. 2002. A Desktop Input Device and Interface for Interactive 3D Character Animation. Graphics Interface, Vol. 2, 133--140.
[24]
Popović, J., Seitz, S., and Erdmann, M. 2003. Motion Sketching for Control of Rigid-Body Simulations. ACM Trans. Graphics, Vol. 22, No. 4, 1034--1054.
[25]
Rekimoto, J., and Ayatsuka, Y. 2000. CyberCode: designing augmented reality environments with visual tags. Proceedings of DARE 2000 on Designing augmented reality environments, 1--10.
[26]
Rus, D., and deSantis, P. 1997. The Self-Organizing Desk. Tech. Rep. PCS-TR97-305, Dartmouth University CS Department.
[27]
Sturman, D. 1998. Computer Puppetry. IEEE Computer Graphics and Applications, Vol. 18, No. 1, 38--45.
[28]
Thomas, F., and Johnston, O. 1981. Disney Animation: The Illusion of Life. Walt Disney Productions, New York.
[29]
Thorne, M., Burke, D., and van de Panne, M. 2004. Motion Doodles: An Interface for Sketching Character Motion. ACM Trans. Graphics (Proc. SIGGRAPH), Vol. 23, No. 3, 424--431.
[30]
Tomasi, C., and Kanade, T. 1991. Detection and Tracking of Point Features. Tech. Rep. CMU-CS-91-132, Carnegie Mellon University, Pittsburgh, PA, USA.
[31]
Trucco, E., and Plakas, K. 2006. Video Tracking: A Concise Survey. IEEE Journal of Oceanic Engineering, Vol. 31, No. 2, 520--529.
[32]
Wang, J., Xu, Y., Shum, H., and Cohen, M. 2004. Video Tooning. ACM Trans. Graphics (Proc. SIGGRAPH), Vol. 23, No. 3, 574--583.
[33]
Wang, J., Bhat, P., Colburn, R., Agrawala, M., and Cohen, M. 2005. Interactive video cutout. ACM Trans. Graphics (Proc. SIGGGRAPH), Vol. 24, No. 3, 585--594.
[34]
Wang, J., Drucker, S., Agrawala, M., and Cohen, M. 2006. The Cartoon Animation Filter. ACM Trans. Graphics (Proc. SIGGRAPH), Vol. 25, No. 3, 1169--1173.
[35]
Wellner, P. 1993. Interacting with Paper on the DigitalDesk. Communications of the ACM, Vol. 36, No. 7 (July), 86--97.
[36]
Wilson, A. 2005. Play Anywhere: a compact interactive tabletop projection-vision system. In Proc. UIST, 83--92.
[37]
Winnemöller, H., Olsen, S., and Gooch, B. 2006. Realtime Video Abstraction. ACM Trans. Graphics (Proc. SIGGRAPH), Vol. 25, No. 3, 1221--1226.
[38]
Wisniewski, D., and Wisniewski, D. 1996. Worlds of Shadow: Teaching with Shadow Puppetry. Teacher Ideas Press.

Cited By

View all
  • (2024)PortalInk: 2.5D Visual Storytelling with SVG Parallax and Waypoint TransitionsProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676376(1-16)Online publication date: 13-Oct-2024
  • (2024)SketchAnim: Real-Time Sketch Animation Transfer from VideosProceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation10.1111/cgf.15176(1-11)Online publication date: 21-Aug-2024
  • (2023)Tangible Avatar : Enhancing Presence and Embodiment During Seated Virtual Experiences with a Prop-Based Controller2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct60411.2023.00121(572-577)Online publication date: 16-Oct-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SIGGRAPH Asia '08: ACM SIGGRAPH Asia 2008 papers
December 2008
581 pages
ISBN:9781450318310
DOI:10.1145/1457515
  • Editor:
  • John C. Hart
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 December 2008

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. animation
  2. real-time
  3. tangible user interface
  4. vision

Qualifiers

  • Research-article

Funding Sources

Conference

SIGGRAPH '08
Sponsor:

Acceptance Rates

SIGGRAPH Asia '08 Paper Acceptance Rate 59 of 320 submissions, 18%;
Overall Acceptance Rate 178 of 869 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)40
  • Downloads (Last 6 weeks)3
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)PortalInk: 2.5D Visual Storytelling with SVG Parallax and Waypoint TransitionsProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676376(1-16)Online publication date: 13-Oct-2024
  • (2024)SketchAnim: Real-Time Sketch Animation Transfer from VideosProceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation10.1111/cgf.15176(1-11)Online publication date: 21-Aug-2024
  • (2023)Tangible Avatar : Enhancing Presence and Embodiment During Seated Virtual Experiences with a Prop-Based Controller2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct60411.2023.00121(572-577)Online publication date: 16-Oct-2023
  • (2023)New Experience to Operate Traditional Glove Puppetry on Digital Interactive Stage2023 IEEE 6th International Conference on Knowledge Innovation and Invention (ICKII)10.1109/ICKII58656.2023.10332624(92-95)Online publication date: 11-Aug-2023
  • (2021)Flexible Low-Cost Digital Puppet SystemVirtual, Augmented and Mixed Reality10.1007/978-3-030-77599-5_46(676-694)Online publication date: 24-Jul-2021
  • (2020)StoryMakAR: Bringing Stories to Life With An Augmented Reality & Physical Prototyping Toolkit for YouthProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376790(1-14)Online publication date: 21-Apr-2020
  • (2020)An Exploratory Study of Augmented Reality Presence for Tutoring Machine TasksProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376688(1-13)Online publication date: 21-Apr-2020
  • (2020)XRDirector: A Role-Based Collaborative Immersive Authoring SystemProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376637(1-12)Online publication date: 21-Apr-2020
  • (2019)Supporting the Cross-cultural Appreciation of Traditional Chinese Puppetry Through a Digital Gesture LibraryJournal on Computing and Cultural Heritage 10.1145/334188212:4(1-19)Online publication date: 17-Nov-2019
  • (2018)Live SketchProceedings of the 2018 CHI Conference on Human Factors in Computing Systems10.1145/3173574.3174236(1-12)Online publication date: 21-Apr-2018
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media