A new general framework for shot boundary detection and key-frame extraction

ABSTRACT

Video shot boundary detection is an important step in many video applications. Since the rapid development of video editing technology, especially, the extensive use of sub-window in news video, the original method of video segmentation cannot efficiently detect the video shot boundary caused by special video technique. In this paper, previous temporal multi-resolution analysis (TMRA) work was extended by first using SVM (Supported Vector Machines) classify the video frames within a sliding window into normal frames, gradual transition frames and CUT frames, then clustering the classified frames into different shot categories. The experimental result on ground truth, which has about 21 hours (10,250 shots) news video clip, shows that the new framework has relatively good accuracy for the detection of shot boundaries. It basically resolves the difficulties of shot boundaries detection caused by sub-window technique in video. The framework also greatly improves accuracy of gradual transitions of shot.

REFERENCES

• 1: J. Adcock, A. Girgensohn, M. Cooper. FXPAL Experiments for TRECVID 2004{A}, Proceedings of TRECVID 2004, 2005.
• 2: T.M. Bae, S.H. Jin, Y. M. Ro. Video Segmentation Using Hidden Markov Model with Multimodal Features, CIVR2004, LNCS 3115, Dublin, 2004, 401--409.
• 3: Chih-Chung Chang and Chih-Jen Lin. LIBSVM: a Library for Support Vector Machines 2004 Available at http://www.csie.ntu.edu.tw/~cjlin
• 4: Chang, C.-C. and C.-J. Lin. LIBSVM: a library for support vector machines. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm. 2001.
• 5: A. Cohen, R. Ryan. Wavelet and Multiscale Signal Processing, Chapman and Hall Publishers, 1995.
• 6: Naiyang Deng, Yingjie Tian. A New Method in Data Mining-Support Vector Machine. Beijing: Science Publisher 2004.
• 7: HuaMin Feng, A Chandrashekhara, T.S. Chua. An Automatic Temporal Multi-resolution Analysis Framework for Shot Boundary Detection. MMM 2003, the 9th International Conference on Multi-Media Modeling, 2003, 224.
• 8: YiHong Gong. An Accuracy and Robust Method for Detecting Video Shot boundaries, ICMCS, 1999, Vol. 1 850--854.
• 9: A. Hampapur , T. Weymouth , R. Jain, Digital video segmentation, Proceedings of the second ACM international conference on Multimedia, p.357-364, October 15-20, 1994, San Francisco, California, United States [doi>10.1145/192593.192699]
• 10: F. Idris, S. Panchanathan. Review of Image and Video Indexing Techniques. Journal of Visual Communication and Image Representation, 1997 June, 8(2), 146--166.
• 11: Lin, H.-T. and C.-J. Lin. A study on sigmoid kernels for SVM and the training of non-PSD kernels by SMO-type methods. Technical report, Department of Computer Science and Information Engineering, National Taiwan University. 2003.
• 12: S. Knerr, L. Personnaz, and G. Dreyfus. Single-layer learning revisited: a stepwise procedure for building and training a neural network. In J. Fogelman, editor, Neurocomputing: Algorithms.
• 13: Architectures and Applications. Springer-Verlag, 1990.
• 14: S. Sathiya Keerthi , Chih-Jen Lin, Asymptotic behaviors of support vector machines with Gaussian kernel, Neural Computation, v.15 n.7, p.1667-1689, July 2003 [doi>10.1162/089976603321891855]
• 15: Y. Lin, M. S. Kankanhalli, T.S. Chua. Temporal Multi-resolution Analysis for video Segmentation, Proc. SPIE Conf. Storage and Retrieval for Media Database VIII, SPIE 2000 Vol. 3970, 494--505.
• 16: Yu-Ping Wang , S. L. Lee, Scale-Space Derived From B-Splines, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.20 n.10, p.1040-1055, October 1998 [doi>10.1109/34.722612 ]
• 17: Hong Heather Yu, Wayne Wolf. Multi-resolution video segmentation using wavelet transformation, Storage and retrieval for Image and Video Database (SPIE), San Jose, CA, USA, 1998, 176--187.
• 18: Hongjiang Zhang , Chien Yong Low , Stephen W. Smoliar, Video parsing and browsing using compressed data, Multimedia Tools and Applications, v.1 n.1, p.89-111, March 1995 [doi>10.1007/BF01261227]
• 19: HongJiang Zhang , Atreyi Kankanhalli , Stephen W. Smoliar, Automatic partitioning of full-motion video, Multimedia Systems, v.1 n.1, p.10-28, June 1993 [doi>10.1007/BF01210504]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Video (e.g., tape, disk, DVI)

General Terms:
Algorithms, Design, Experimentation

Keywords:
SVM, TMRA, blocked color histogram