Annotating and navigating tourist videos

ABSTRACT

Due to the rapid increase in video capture technology, more and more tourist videos are captured every day, creating a challenge for organization and association with metadata. In this paper, we present a novel system for annotating and navigating tourist videos. Placing annotations in a video is difficult because of the need to track the movement of the camera. Navigation of a regular video is also challenging due to the sequential nature of the media. To overcome these challenges, we introduce a system for registering videos to geo-referenced 3D models and analyzing the video contents. We also introduce a novel scheduling algorithm for showing annotations in video. We show results in automatically annotated videos and in a map-based application for browsing videos. Our user study indicates the system is very useful.

References

1. S. A. Ay, R. Zimmermann, and S. H. Kim. Viewable scene modeling for geospatial video search. ACM International Conference on Multimedia, pages 309--318, 2008.
2. R. E. Bellman. Dynamic programming. Princeton University Press, 1957.
3. B. Chen, G. Ramos, E. Ofek, M. Cohen, S. Drucker, and D. Nister. Interactive techniques for registering images to digital terrain and building models. Technical report, Microsoft Research, 2008.
4. L. Cheong and H. Huo. Shot change detection using scenebased constraint. Multimedia Tools and Applications, 2001.
5. P. L. Cho. 3D organization of 2D urban imagery. Applied Image Pattern Recognition Workshop, 0:3--8, 2007.
6. B. Epshtein, E. Ofek, Y. Wexler, and P. Zhang. Hierarchical photo organization using geo-relevance. In ACM International Symposium on Advances in Geographic Information Systems, pages 1--7, New York, NY, USA, 2007. ACM.
7. Google. Google Maps. http://maps.google.com.
8. W. Heng and K. Ngan. An object-based shot boundary detection using edge tracing and tracking. Journal of Visual Communication and Image Representation, 2001.
9. J. Kopf, B. Neubert, B. Chen, M. Cohen, D. Cohen-Or, O. Deussen, M. Uyttendaele, and D. Lischinski. Deep photo: Model-based photograph enhancement and viewing. ACM Trans. on Graphics (Proceedings of SIGGRAPH Asia 2008), 2008.
10. M. Kroepfl, Y. Wexler, and E. Ofek. Efficiently locating photographs in many panoramas. submitted to ACM GIS 2010, 2010.
11. D. G. Lowe. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision, 60(2):91--110, 2004.
12. Q. Luan, S. M. Drucker, J. Kopf, Y.-Q. Xu, and M. F. Cohen. Annotating gigapixel images. In ACM Symposium on User Interface Software and Technology, 2008.
13. Microsoft. Bing Maps. http://maps.bing.com/.
14. H. Nicolas, A. Manury, J. Benois-Pineau, W. Dupuy, and D. Barba. Grouping video shots into scenes based on 1D mosaic descriptors. Proc. Intl. Conf. on Image Proc., 2004.
15. S. Pongnumkul, J. Wang, and M. Cohen. Creating map-based storyboards for browsing tour videos. In ACM symposium on User Interface Software and Technology, pages 13--22, New York, NY, USA, 2008. ACM.
16. T. Rattenbury and M. Naaman. Methods for extracting place semantics from Flickr tags. ACM Trans. on Web, 3(1):1--30, 2009.
17. I. Simon, N. Snavely, and S. M. Seitz. Scene summarization for online image collections. IEEE International Conference on Computer Vision, 2007.
18. N. Snavely, S. M. Seitz, and R. Szeliski. Photo tourism: exploring photo collections in 3D. In ACM SIGGRAPH 2006 Papers, pages 835--846, New York, NY, USA, 2006.
19. K. Toyama, R. Logan, and A. Roseway. Geographic location tags on digital images. ACM International Conference on Multimedia, 2003.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

        • Video segmentation

      • Computer vision tasks

        • Video summarization

      • Image and video acquisition