Scalable Video Summarization: A Comparative Study

ABSTRACT

The amount of videos over the internet and media storage systems has dramatically increased. This poses challenges in video content understanding and management. A video is a complex and resource consuming media. In addition, efficient use of video data requires the data to be understood and accessed without having to watch it entirely. For those reasons, video summarization (VS) has been a hot topic of recent researches. VS is the process of creating a compact representation that can provide the user with concise information about the video content. VS helps in efficient storage, quick browsing, and retrieval of video data maintaining its main features. In video codec and streaming contexts, Scalable Video Coding (SVC) enables dynamic adaptation based on network conditions and device capabilities. This paper reviews the recent work on scalable video summarization (SVS) and discusses its role in current research directions.

References

1. "YouTube Statistics," 7, 2016; http://www.youtube.com/yt/press/statistics.html.
2. D. Donchev. "27 Mind Blowing YouTube Facts, Figures and Statistics -- 2016," 8-2016; http://fortunelords.com/27-mind-blowing-youtube-facts-figures-and-statistics-backed-by-data/.
3. M. Ajmal, M. H. Ashraf, M. Shakir, Y. Abbas, and F. A. Shah, "Video summarization: techniques and classification," Computer Vision and Graphics, Springer, vol. 7594, 2012, p. 1--13.
4. Z. Xiong, R. Radhakrishnan, A. Divakaran, Y. Rui, and T. S. Huang, A unified framework for video summarization, browsing & retrieval: with applications to consumer and surveillance video: Academic Press, 2006.
5. B. T. Truong, and S. Venkatesh, "Video abstraction: A systematic review and classification," ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP), vol. 3, no. 1, 2007, p. 37.
6. S. Alexandro, and C. Luca, "Live Key Frame Extraction in User Generated Content Scenarios for Embedded Mobile Platforms," MultiMedia Modeling, Springer, vol. 8325, 2014, p. 291--302.
7. H. Farouk, K. ElDahshan, and A. Abozeid, "The State of the Art of Video Summarization for Mobile Devices: Review Article," Graphics, Vision and Image Processing GVIP, vol. 14, no. 2, 2014, p. 37--50.
8. H. Schwarz, D. Marpe, and T. Wiegand, "Overview of the scalable video coding extension of the H. 264/AVC standard," IEEE Transactions on circuits and systems for video technology, vol. 17, no. 9, 2007, p. 1103--1120.
9. J. M. Boyce, Y. Ye, J. Chen, and A. K. Ramasubramonian, "Overview of SHVC: Scalable Extensions of the High Efficiency Video Coding Standard," IEEE Transactions on Circuits and Systems for Video Technology, vol. 26, no. 1, 2016, p. 20--34.
10. A. G. Money, and H. Agius, "Video summarisation: A conceptual framework and survey of the state of the art," Journal of Visual Communication and Image Representation, vol. 19, no. 2, 2008, p. 121--143.
11. S. E. F. de Avila, and A. P. B. Lopes, "VSUMM: A mechanism designed to produce static video summaries and a novel evaluation method," Pattern Recognition Letters, Elsevier vol. 32, no. 1, 2011, p. 56--68.
12. H. Farouk, Kamal A. ElDahshan, and A. Abozeid, "Effective and Efficient Video Summarization Approach for Mobile Devices," International Journal of Interactive Mobile Technologies (iJIM), vol. 10, no. 1, 2016, p. 19--26.
13. L. Zhu, Z. Fan, and K. Aggelos K, "Joint video summarization and transmission adaptation for energy-efficient wireless video streaming," EURASIP Journal on Advances in Signal Processing, vol. 2008, 2008.
14. J. Almeida, N. J. Leite, and R. d. S. Torres, "Online video summarization on compressed domain," Journal of Visual Communication and Image Representation, vol. 24, no. 6, 2013, p. 729--738.
15. R. M. Jiang, A. H. Sadka, and D. Crookes, "Advances in video summarization and skimming," Recent Advances in Multimedia Signal Processing and Communications, pp. 27--50: Springer, 2009.
16. E. Asadi, and N. M. Charkari, "Video summarization using fuzzy c-means clustering," in 20th Iranian Conference on Electrical Engineering (ICEE), Tehran, Iran, 2012, pp. 690--694.
17. S. CVETKOVIC, M. JELENKOVIC, and S. V. NIKOLIC, "Video summarization using color features and efficient adaptive threshold technique," Przegląd Elektrotechniczny, vol. 89, 2013.
18. S. Bekhet, A. Ahmed, and A. Hunter, "Video Matching Using DC-image and Local Features."
19. R. Pal, A. Ghosh, and S. K. Pal, "Video Summarization and Significance of Content: A Review," Handbook on Soft Computing for Video Surveillance, pp. 79--102: CRC Press, 2012.
20. J. Niu, D. Huo, K. Wang, and C. Tong, "Real-time generation of personalized home video summaries on mobile devices," Neurocomputing, ScienceDirect, vol. 120, 2013, p. 404--414.
21. G. Abdollahian, C. M. Taskiran, Z. Pizlo, and E. J. Delp, "Camera motion-based analysis of user generated video," IEEE Transactions on Multimedia, vol. 12, no. 1, 2010, p. 28--41.
22. J. Ren, and J. Jiang, "Hierarchical modeling and adaptive clustering for real-time summarization of rush videos," IEEE Transactions on Multimedia, vol. 11, no. 5, 2009, p. 906--917.
23. X. Zhu, X. Wu, J. Fan, A. K. Elmagarmid, and W. G. Aref, "Exploring video content structure for hierarchical summarization," Multimedia Systems, vol. 10, no. 2, 2004, p. 98--115.
24. L. Herranz, and J. M. Martinez, "A framework for scalable summarization of video," IEEE Transactions on Circuits and Systems for Video Technology, vol. 20, no. 9, 2010, p. 1265--1270.
25. Y. Cong, J. Yuan, and J. Luo, "Towards scalable summarization of consumer videos via sparse dictionary selection," IEEE Transactions on Multimedia, vol. 14, no. 1, 2012, p. 66--75.
26. P. Etezadifar, and H. Farsi, "Scalable video summarization via sparse dictionary learning and selection simultaneously," Multimedia Tools and Applications, 2016, p. 1--25.
27. R. Panda, S. K. Kuanar, and A. S. Chowdhury, "Scalable Video Summarization Using Skeleton Graph and Random Walk." pp. 3481--3486.
28. N. Paragios, Y. Chen, and O. D. Faugeras, Handbook of mathematical models in computer vision: Springer Science & Business Media, 2006.
29. K. R. Perez-Daniel, M. N. Miyatake, J. Benois-Pineau, S. Maabout, and G. Sargent, "Scalable video summarization of cultural video documents in cross-media space based on data cube approach." pp. 1--6.
30. J. Gray, S. Chaudhuri, A. Bosworth, A. Layman, D. Reichart, M. Venkatrao, F. Pellow, and H. Pirahesh, "Data cube: A relational aggregation operator generalizing group-by, cross-tab, and sub-totals," Data mining and knowledge discovery, vol. 1, no. 1, 1997, p. 29--53.
31. "MPEG-DASH Standard," 7-2016; http://mpeg.chiariglione.org/standards/mpeg-dash.
32. H. Farouk, K. A. El Dahshan, and A. Abozeid, "Context-Aware Joint Video Summarization and Streaming (CVSS) Approach," IEEE International Symposium on Multimedia (ISM), 2016, p. 597--602.
33. L. Herranz, and J. M. Martínez, "On the advantages of the use of bitstream extraction for video summary generation," Advances in Multimedia Modeling, pp. 755--760: Springer, 2010.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision tasks

        • Video summarization