A unified framework for semantic shot representation of sports video

ABSTRACT

The development of mid-level shot description helps to bridge the gap between low-level feature and high-level semantics in video indexing and analysis. In this paper, we present a unified framework for semantic shot representation in field-ball sports genres, in which a video shot is characterized via three essential properties, namely, camera shot size, subject in a scene and video production technology. The three properties clearly represent the primary factors of a shot, and provide a unified viewpoint of semantic shot definition. Based on this framework, we design an effective architecture for semantic shot management comprising three main components as: 1) flexible shot clustering and retrieval by adjusting the weights of three properties according to different requirements; 2) semantics based video temporal segmentation for further event recognition; and 3) comprehensive sports video semantics analysis. Extensive experiments on soccer, basketball and tennis demonstrate the effectiveness and validity of this framework.

REFERENCES

• 1: Ling-Yu Duan , Min Xu , Tat-Seng Chua , Qi Tian , Chang-Sheng Xu, A mid-level representation framework for semantic sports video analysis, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957020]
• 2: L. Duan, M. Xu, and Q. Tian, "Semantic shot classification in sports video", In Proc. of SPIE Storage and Retrieval for Media Database 2003, pp. 300--313, 2003.
• 3: R. Dahyot, N. Rea, and A. Kokaram, "Sports video shot segmentation and classification", In Proc. of Visual Communications and Image Processing, pp.8--11, 2003.
• 4: J. Wang, E. Chng, and C. Xu, "Soccer replay detection using scene transition structure analysis", In Proc. of ICASSP, pp. 433--437, 2005.
• 5: Jürgen Assfalg , Marco Bertini , Carlo Colombo , Alberto Del Bimbo, Semantic Annotation of Sports Videos, IEEE MultiMedia, v.9 n.2, p.52-60, April 2002 [doi>10.1109/93.998060 ]
• 6: A. Ekin, A. Tekalp, and R. Mehrotra, "Automatic soccer video analysis and summarization", IEEE Trans. Image Processing, 12(7), pp. 796--807, 2003.
• 7: Arnold W. M. Smeulders , Marcel Worring , Simone Santini , Amarnath Gupta , Ramesh Jain, Content-Based Image Retrieval at the End of the Early Years, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.12, p.1349-1380, December 2000 [doi>10.1109/34.895972 ]
• 8: Stephen W. Smoliar , HongJiang Zhang, Content-Based Video Indexing and Retrieval, IEEE MultiMedia, v.1 n.2, p.62-72, June 1994 [doi>10.1109/93.311653 ]
• 9: M. Naphade, and T. Huang, "A probabilistic framework for semantic video indexing, filtering, and retrieval", IEEE Trans. Multimedia, 3(1), pp. 141--151, 2001.
• 10: Ichiro Ide , Koji Yamamoto , Hidehiko Tanaka, Automatic Video Indexing Based on Shot Classification, Proceedings of the First International Conference on Advanced Multimedia Content Processing, p.87-102, November 09-11, 1998
• 11: C. Snoek, M. Worring, "Multimedia event based video indexing using time intervals", IEEE Trans.Multimedia, 2005 (in press).
• 12: X. Tong, L. Duan, H. Lu, C. Xu, Q.Tian, and J. Jin, "A mid-level visual concept generation framework for sports analysis", In Proc. of ICME 2005.
• 13: Xiao-Feng Tong , Han-Qing Lu , Qing-Shan Liu, An Effective and Fast Soccer Ball Detection and Tracking Method, Proceedings of the Pattern Recognition, 17th International Conference on (ICPR'04) Volume 4, p.795-798, August 23-26, 2004 [doi>10.1109/ICPR.2004.151]
• 14: Xinguo Yu , Changsheng Xu , Hon Wai Leong , Qi Tian , Qing Tang , Kong Wah Wan, Trajectory-based ball detection and tracking with applications to semantic analysis of broadcast soccer video, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957018]
• 15: Kongwah Wan , Xin Yan , Xinguo Yu , Changsheng Xu, Real-time goal-mouth detection in MPEG soccer video, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957079]
• 16: Xiaofeng Tong , Hanqing Lu , Qingshan Liu , Hongliang Jin, Replay Detection in Broadcasting Sports Video, Proceedings of the Third International Conference on Image and Graphics (ICIG'04), p.337-340, December 18-20, 2004 [doi>10.1109/ICIG.2004.124]
• 17: X. Tong, Q. Liu, and H.Lu, "Semantic units based events detection in soccer video", In Proc. of ICIP 2004.
• 18: Lei Wang , Michael Lew , Guangyou Xu, Offense based temporal segmentation for event detection in soccer video, Proceedings of the 6th ACM SIGMM international workshop on Multimedia information retrieval, October 15-16, 2004, New York, NY, USA [doi>10.1145/1026711.1026754]
• 19: J. Dai, L. Duan, X. Tong, C. Xu, Q. Tian, H. Lu, J. Jin, "Replay scene classification in soccer video using web broadcast text", In Proc. of ICME, Netherlands, July, 2005.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Clustering

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Abstracting methods; Indexing methods
      H.3.3 Information Search and Retrieval
          Subjects: Search process

General Terms:
Algorithms, Design, Experimentation

Keywords:
framework, mid-level shot representation, sports video, video retrieval