A motion based scene tree for browsing and retrieval of compressed videos

ABSTRACT

This paper describes a fully automatic content-based approach for browsing and retrieval of MPEG-2 compressed video. The first step of the approach is the detection of shot boundaries based on motion vectors available from the compressed video stream. The next step involves the construction of a scene tree from the shots obtained earlier. The scene tree is shown to capture some semantic information as well as to provide a construct for hierarchical browsing of compressed videos. Finally, we build a new model for video similarity based on global as well as local motion associated with each node in the scene tree. To this end, we propose new approaches to camera motion and object motion estimation. The experimental results demonstrate that the integration of the above techniques results in an efficient framework for browsing and searching large video databases.

REFERENCES

• 1: B. L. Yeo and B. Liu. Rapid scene analysis on compressed video. IEEE Transactions On Circuits and Systems for Video Technology, 5:553--544, Dec. 1995.
• 2: J. Feng, K. -T. Lo, and M. H. Scene change detection algorithm for mpeg video sequence. In IEEE International Conference on Image Processing, volume 1, pages 821--824, 1996.
• 3: B. Gunturk, Y. Altunbasak, and R. Mersereau. Super-resolution reconstruction of compressed video using transform-domain statistics. IEEE Transactions on Image Processing, 13(1):33--43, jan 2004.
• 4: International Organization for Standardization. MPEG21 Overview (CODING OF MOVING PICTURES AND AUDIO), iso/iec jtc1/sc29/wg11/n4318 edition, July 2001.
• 5: International Organization for Standardization. Overview of the MPEG-7 Standard, iso/iec/jtc1/sc29/wg11 n4031 edition, Mar 2001.
• 6: ITU-T. Video Coding for Low Bit Rate Communication, ITU-T Recommendation, h. 263 edition, Feb 1998.
• 7: ITU-T. Joint Final Committee Draft (JFCD) of Joint Video Specification (ITU-T Rec. H. 264---ISO/IEC 14496-10 AVC), h. 264 edition, July 2002.
• 8: E. Katz. The film encyclopedia, 2nd ed. New York: Harper Collins, 1994.
• 9: J. R. Kender , B. -L. Yeo, Video Scene Segmentation via Continuous Video Coherence, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.367, June 23-25, 1998
• 10: R. Lienhart. Comparison of automatic shot boundary detection algortihms. Proceedings of SPIE conference on Storage and Retrieval for Image and Video Databases VII, 3656:290--301, Jan. 1999.
• 11: J. Meng, Y. Juan, and S. -F. Chang. Scene change detection in a MPEG video sequence. In Proceedings of SPIE Conference on Multimedia Computing and Networking, volume 2417, pages 180--191, San Jose, CA, Feb 1995.
• 12: J. Nam , A. Tewfik, Combined Audio and Visual Streams Analysis for Video Sequence Segmentation, Proceedings of the 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '97) -Volume 4, p.2665, April 21-24, 1997
• 13: JungHwan Oh , Kien A. Hua, Efficient and cost-effective techniques for browsing and indexing large video databases, Proceedings of the 2000 ACM SIGMOD international conference on Management of data, p.415-426, May 15-18, 2000, Dallas, Texas, United States
• 14: Z. Rasheed and M. Shah. Scene boundary detection in hollywood movies and TV show. In IEEE International Conference on Computer Vision and Pattern Recognition, Madison, WI, June 2003.
• 15: M. A. Robertson and R. L. Stevenson. Restoration of compressed video using temporal information. In SPIE conference on Visual Communications and Image Processing, volume 4310, pages 21--29, San Jose, CA, 2001.
• 16: C. Saraceno and R. Leonardi. Identification of story units in audio-visual sequences by joint audio and video processing. In Proceeding of International Conference on Image Processing, pages 358--362, Chicago, IL, USA, 1998.
• 17: D. Schonfeld and D. Lelescu. VORTEX: Video retrieval and tracking from compressed multimedia databasesmultiple object tracking from mpeg-2 bitstream. Journal of Visual Communications and Image Representation, Special Issue on Multimedia Database Management, 11:154--182, 2000.
• 18: I. K. Sethi and N. V. Patel. Statistical approach to scene change detection. In Proceedings of SPIE Conference on Storage and Retrieval for Image and Video Databases, volume 2420, pages 329--338, San Jose, CA, feb 1995.
• 19: B. T. Truong, S. Vekatesh, and C. Dorai. Scene extraction in motion picture. IEEE Transactions on Circuits and Systems for Video Technology, 13(1):5--15, jan 2003.
• 20: H. Wang, A. Divakaran, A. Vetro, S. -F. Chang, and H. Sun. Survey of compressed-domain features used in audio-visual indexing and analysis. Journal of Visual Communication and Image Representation, 14(2):50--183, June 2003.
• 21: B.-L. Yeo , B. Liu, On the extraction of DC sequence from MPEG compressed video, Proceedings of the 1995 International Conference on Image Processing (Vol.2)-Volume 2, p.2260, October 23-26, 1995
• 22: B. L. Yeo and B. Liu. Rapid scene analysis on compressed video. IEEE Transactions On Circuits and Systems for Video Technology. , 5:553--544, Dec. 1995.
• 23: H. Yi, D. Rajan, and L. -T. Chia. A unified approach to detection of shot boundaries and subshots in compressed video. In IEEE International Conference on Image Processing, Barcelona, Spain, Sept. 2003.
• 24: Ramin Zabih , Justin Miller , Kevin Mai, A feature-based algorithm for detecting and classifying scene breaks, Proceedings of the third ACM international conference on Multimedia, p.189-200, November 05-09, 1995, San Francisco, California, United States [doi>10.1145/217279.215266]
• 25: HongJiang Zhang , Atreyi Kankanhalli , Stephen W. Smoliar, Automatic partitioning of full-motion video, Multimedia Systems, v.1 n.1, p.10-28, June 1993 [doi>10.1007/BF01210504]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis

General Terms:
Algorithms

Keywords:
shot boundary detection, video browsing, video indexing, video retrieval, video similarity