Rushes video summarization using audio-visual information and sequence alignment

ABSTRACT

This paper describes our system and methodologies for the BBC rushes video summarization task of TRECVID 2008. The procedure of the system is composed of three major steps: shot detection, irrelevant and repetitive subshot removal, and final summary generation. First, we segment the original rushes video into subshots according to the difference and accumulative difference of local color histogram between consecutive frames. Second, we recognize the irrelevant subshots, such as subshots of color bar, pure gray frames, and clapper board. We propose a novel video sequence alignment algorithm to detect repetitive subshots. After removing the irrelevant and repetitive subshots, we generate the final summary using the remaining informative and representative subshots. The evaluation from TRECVID 2008 shows that our system can generate good video summaries.

References

1. Takao, S., Haru, T., and Ariki, Y. Summarization of News Speech with Unknown Topic Boundary. In Proceedings of the IEEE International Conference on Multimedia and Expo. pp: 615--618, 2001.
2. Ekin, A., Tekalp, A. M., and Mehrotra, R. Automatic Soccer Video Analysis and Summarization. IEEE Transactions on Image Processing. vol. 12, no. 7, pp. 796--807, July 2003.
3. Choudary, C. and Liu. T.-C. Summarization of Visual Content in Instructional Videos. IEEE Transactions on Multimedia. vol. 9, no. 7, pp. 1443--1455, Nov. 2007.
4. Over, P., Smeaton, A. F., and Kelly, P. The TRECVID 2008 BBC rushes summarization evaluation. In Proceedings of the International Workshop on TRECVID Video Summarization (TVS'08), Vancouver, British Columbia, Canada, October 31, 2008, ACM Press, New York, NY, pp. 1--20, 2008.
5. Zhang, H. J., Kankanhalli, A., and Smoliar, S.W. Automatic Partitioning of Full-Motion Video. Journal of Multimedia Systems 1, 10--28. 1993.
6. Liu, Y., Liu, Y., and Zhang, Y. The Hong Kong Polytechnic University at TRECVID 2007 BBC Rushes Summarization. In Proceedings of the International Workshop on TRECVID Video Summarization (TVS'07), Augsburg, Bavaria, Germany, September 28, 2007, ACM Press, New York, NY, 2007.
7. Pfeiffer, S., Lienhart, R., Fischer, S., and Effelsberg, W. Abstracting Digital Movies Automatically. Journal of Visual Communication and Image Representation, 7, 4, pp: 345--353. Dec. 1996.
8. Yan, J. X, Dou, W. B., and Dong, Z. W. Time-Domain Detection Method of Transient Signals in Audio Coding. Journal of Electronics & Information Technology, vol. 28, no. 2, Feb. 2006.
9. Chang, C. C. and Lin, C. J. LIBSVM: a library for support vector machines, 2001.
10. Manjunath, B. S., Ohm, J. R., Vasudevan, V. V., and Yamada, A. Color and Texture Descriptors. IEEE Transactions on Circuits and Systems for Video Technology, vol. 11, pp. 703--715, 2001.
11. Needleman, S. and Wunsch, C. A General Method Applicable to the Search for Similarities in the Amino Acid Sequence of Two Proteins. Journal of Molecular Biology, vol. 48, no. 3, pp. 443--453, 1970.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision tasks

        • Video summarization

• Information systems

  • Information retrieval

    • Document representation

• Theory of computation

  • Semantics and reasoning

    • Program reasoning

      • Abstraction