skip to main content
10.1145/1449715.1449719acmconferencesArticle/Chapter ViewAbstractPublication PagesuistConference Proceedingsconference-collections
research-article

Video object annotation, navigation, and composition

Published: 19 October 2008 Publication History

Abstract

We explore the use of tracked 2D object motion to enable novel approaches to interacting with video. These include moving annotations, video navigation by direct manipulation of objects, and creating an image composite from multiple video frames. Features in the video are automatically tracked and grouped in an off-line preprocess that enables later interactive manipulation. Examples of annotations include speech and thought balloons, video graffiti, path arrows, video hyperlinks, and schematic storyboards. We also demonstrate a direct-manipulation interface for random frame access using spatial constraints, and a drag-and-drop interface for assembling still images from videos. Taken together, our tools can be employed in a variety of applications including film and video editing, visual tagging, and authoring rich media such as hyperlinked video.

Supplementary Material

JPG File (p3-goldman.jpg)
MOV File (p3-goldman.mov)

References

[1]
A. Agarwala, M. Dontcheva, M. Agrawala, S. Drucker, A. Colburn, B. Curless, D. Salesin, and M. Cohen. Interactive digital photomontage. ACM Trans. Graph. (Proc. SIGGRAPH), 23(4):294--301, 2004.
[2]
A. Agarwala, A. Hertzmann, D. H. Salesin, and S. M. Seitz. Keyframe-based tracking for rotoscoping and animation. ACM Trans. Graph. (Proc. SIGGRAPH), 23(3):584--591, 2004.
[3]
ASTAR Learning Systems. http://www.astarls.com, 2006. {Online; accessed 5-January-2008}.
[4]
C. Bregler, M. Covell, and M. Slaney. Video rewrite: Driving visual speech with audio. In Proc. SIGGRAPH 97, pages 353--360, 1997.
[5]
Y.-Y. Chuang, A. Agarwala, B. Curless, D. H. Salesin, and R. Szeliski. Video matting of complex scenes. ACM Trans. Graph., 21(3):243--248, 2002.
[6]
J. Dakss, S. Agamanolis, E. Chalom, and V. M. Bove Jr. Hyperlinked video. In Proc. SPIE, volume 3528, pages 2--10, 1999.
[7]
P. Dragicevic, G. Ramos, J. Bibliowicz, D. Nowrouzezahrai, R. Balakrishnan, and K. Singh. Video browsing by direct manipulation. In CHI, pages 237--246, 2008.
[8]
D. B Goldman, B. Curless, S. M. Seitz, and D. Salesin. Schematic storyboarding for video visualization and editing. ACM Trans. Graph. (Proc. SIGGRAPH), 25(3):862--871, 2006.
[9]
J. Goldman. Kind of a Blur. http://phobos.apple.com/WebObjects/MZStore.woa/wa/viewMovie?id=197994758, 2005. {Short film available online; accessed 5-January-2008}.
[10]
R. I. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision, page 109. Cambridge University Press, ISBN: 0521540518, second edition, 2004.
[11]
T. Karrer, M. Weiss, E. Lee, and J. Borchers. DRAGON: A direct manipulation interface for frame-accurate in-scene video navigation. In CHI, pages 247--250, 2008.
[12]
D. Kimber, T. Dunnigan, A. Girgensohn, F. Shipman, T. Turner, and T. Yang. Trailblazing: Video playback control by direct object manipulation. In ICME, pages 1015--1018, 2007.
[13]
D. Kurlander, T. Skelly, and D. Salesin. Comic chat. In SIGGRAPH '96, pages 225--236, 1996.
[14]
Y. Li, J. Sun, and H.-Y. Shum. Video object cut and paste. ACM Trans. Graph. (Proc. SIGGRAPH), 24(3):595--600, 2005.
[15]
C. Morningstar and R. F. Farmer. The lessons of Lucasfilm's Habitat. In M. Benedikt, editor, Cyberspace: First Steps, pages 273--301. MIT Press, Cambridge, MA, 1991.
[16]
Y. Pritch, A. Rav-Acha, A. Gutman, and S. Peleg. Webcam synopsis: Peeking around the world. In Proc. ICCV, pages 1--8, 2007.
[17]
Y. Pritch, A. Rav-Acha, and S. Peleg. Non-chronological video synopsis and indexing. IEEE Trans. PAMI, 2008. (to appear).
[18]
G. Ramos and R. Balakrishnan. Fluid interaction techniques for the control and annotation of digital video. In Proc. UIST '03, pages 105--114, 2003.
[19]
A. Rav-Acha, Y. Pritch, D. Lischinski, and S. Peleg. Dynamosaicing: Video mosaics with non-chronological time. In Proc. CVPR, pages 58--65, 2005.
[20]
A. Rav-Acha, Y. Pritch, and S. Peleg. Making a long video short: Dynamic video synopsis. In Proc. CVPR, pages 435--441, 2006.
[21]
E. Rosten, G. Reitmayr, and T. Drummond. Real-time video annotations for augmented reality. In Proc. Intl. Symp. on Visual Computing, 2005.
[22]
P. Sand. Long-Range Video Motion Estimation using Point Trajectories. PhD thesis, MIT, 2006.
[23]
P. Sand and S. Teller. Particle video: Long-range motion estimation using point trajectories. In Proc. CVPR '06, pages 2195--2202, 2006.
[24]
A. Schödl and I. A. Essa. Controlled animation of video sprites. In Proc. ACM/Eurographics Symp. on Comp. Animation, pages 121--127, 2002.
[25]
A. Schödl, R. Szeliski, D. H. Salesin, and I. Essa. Video textures. In SIGGRAPH '00, pages 489--498, 2000.
[26]
J. Sivic, F. Schaffalitzky, and A. Zisserman. Object level grouping for video shots. Intl. J. of Comp. Vis., 67(2):189--210, 2006.
[27]
J. M. Smith, D. Stotts, and S.-U. Kum. An orthogonal taxonomy for hyperlink anchor generation in video streams using OvalTine. In Proc. ACM Conf. on Hypertext and Hypermedia, pages 11--18, 2000.
[28]
Sportvision. Changing The Game. http://www.sportvision.com, 2006. {Online; accessed 5-January-2008}.
[29]
J. Wang, P. Bhat, R. A. Colburn, M. Agrawala, and M. F. Cohen. Interactive video cutout. ACM Trans. Graph. (Proc. SIGGRAPH), 24(3):585--594, 2005.
[30]
Wikipedia. Telestrator. http://en.wikipedia.org/w/index.php?title=Telestrator&oldid=180785495, 2006. {Online; accessed 5-January-2008}.
[31]
A. Yilmaz, O. Javed, and M. Shah. Object tracking: A survey. ACM Computing Surveys, 38(4):13, December 2006.
[32]
L. Zhang, N. Snavely, B. Curless, and S. Seitz. Spacetime faces: high resolution capture for modeling and animation. ACM Trans. Graph. (Proc. SIGGRAPH), 23(3):548--558, 2004.

Cited By

View all
  • (2024)RealityEffects: Augmenting 3D Volumetric Videos with Object-Centric Annotation and Dynamic Visual EffectsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661631(1248-1261)Online publication date: 1-Jul-2024
  • (2024)VideoMap: Supporting Video Exploration, Brainstorming, and Prototyping in the Latent SpaceProceedings of the 16th Conference on Creativity & Cognition10.1145/3635636.3656192(311-327)Online publication date: 23-Jun-2024
  • (2023)RealityReplayProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36108887:3(1-25)Online publication date: 27-Sep-2023
  • Show More Cited By

Index Terms

  1. Video object annotation, navigation, and composition

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    UIST '08: Proceedings of the 21st annual ACM symposium on User interface software and technology
    October 2008
    308 pages
    ISBN:9781595939753
    DOI:10.1145/1449715
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 19 October 2008

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. direct manipulation
    2. video annotation
    3. video interaction
    4. video navigation

    Qualifiers

    • Research-article

    Conference

    UIST08

    Acceptance Rates

    Overall Acceptance Rate 561 of 2,567 submissions, 22%

    Upcoming Conference

    UIST '25
    The 38th Annual ACM Symposium on User Interface Software and Technology
    September 28 - October 1, 2025
    Busan , Republic of Korea

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)19
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 14 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)RealityEffects: Augmenting 3D Volumetric Videos with Object-Centric Annotation and Dynamic Visual EffectsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661631(1248-1261)Online publication date: 1-Jul-2024
    • (2024)VideoMap: Supporting Video Exploration, Brainstorming, and Prototyping in the Latent SpaceProceedings of the 16th Conference on Creativity & Cognition10.1145/3635636.3656192(311-327)Online publication date: 23-Jun-2024
    • (2023)RealityReplayProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36108887:3(1-25)Online publication date: 27-Sep-2023
    • (2023)VideoDoodles: Hand-Drawn Animations on Videos with Scene-Aware CanvasesACM Transactions on Graphics10.1145/359241342:4(1-12)Online publication date: 26-Jul-2023
    • (2023)Multiple Planar Object Tracking2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.02144(23403-23413)Online publication date: 1-Oct-2023
    • (2022)Designing for Collaborative Video EditingNordic Human-Computer Interaction Conference10.1145/3546155.3546664(1-11)Online publication date: 8-Oct-2022
    • (2022)VideoSticker: A Tool for Active Viewing and Visual Note-taking from VideosProceedings of the 27th International Conference on Intelligent User Interfaces10.1145/3490099.3511132(672-690)Online publication date: 22-Mar-2022
    • (2022)Exploring the User Interaction with a Multimodal Web-Based Video AnnotatorIntelligent Technologies for Interactive Entertainment10.1007/978-3-030-99188-3_2(13-22)Online publication date: 25-Mar-2022
    • (2021)Unsupervised learning for cuboid shape abstraction via joint segmentation from point cloudsACM Transactions on Graphics10.1145/3450626.345987340:4(1-11)Online publication date: 19-Jul-2021
    • (2021)Bijective and coarse high-order tetrahedral meshesACM Transactions on Graphics10.1145/3450626.345984040:4(1-16)Online publication date: 19-Jul-2021
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media