3D reconstruction and enrichment of broadcast soccer video

ABSTRACT

Recently, it has become a new trend to reconstruct sports video for various purposes. This paper presents a 3D reconstruction and enrichment system that not only reconstructs broadcast soccer video but also enriches reconstructed video with music and illustrations of the video contents. The system can reconstruct not only the goalmouth scene but also the midfield scene, which cannot be reconstructed by the existing systems. To quickly find the feature points for calibrating the camera, we propose a fast algorithm to detect the lines in the goalmouth scene and use the algorithm proposed in our previous papers to detect the partial ellipses in the midfield scene. The reconstruction is conducted on several video sequences of two scenes. The reconstructed videos eliminate the ball deformation and unnecessary camera changes through smoothing the camera parameters. This system also serves as an experimental system for our project that reconstructs the on-going soccer game in real time.

REFERENCES

• 1: T. Bebie and H. Bieri, A video-based 3D-reconstruction of soccer games, Eurographics, Vol. 19 (3), 2000.
• 2: Jonathan Foote , Matthew Cooper , Andreas Girgensohn, Creating music videos using automatic media analysis, Proceedings of the tenth ACM international conference on Multimedia, December 01-06, 2002, Juan-les-Pins, France [doi>10.1145/641007.641119]
• 3: Chua Hock Chuan, Automatic Parsing of TV Soccer Programs, Proceedings of the International Conference on Multimedia Computing and Systems, p.167, May 15-18, 1995
• 4: Richard Hartley , Andrew Zisserman, Multiple View Geometry in Computer Vision, Cambridge University Press, New York, NY, 2003
• 5: Tacone Kim , Yongduek Seo , Ki-Sang Hong, Physics-Based 3D Position Analysis of a Soccer Ball from Monocular Image Sequences, Proceedings of the Sixth International Conference on Computer Vision, p.721, January 04-07, 1998
• 6: Takayoshi Koyama , Itaru Kitahara , Yuichi Ohta, Live Mixed-Reality 3D Video in Soccer Stadium, Proceedings of the The 2nd IEEE and ACM International Symposium on Mixed and Augmented Reality, p.178, October 07-10, 2003
• 7: Y. Liu , T. S. Huang , O. D. Faugeras, Determination of Camera Location from 2-D to 3-D Line and Point Correspondences, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.12 n.1, p.28-37, January 1990 [doi>10.1109/34.41381 ]
• 8: K. Matsui , M. Iwase , M. Agata , T. Tanaka , N. Ohnishi, Soccer Image Sequence Computed by a Virtual Camera, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.860, June 23-25, 1998
• 9: Xinguo Yu , Changsheng Xu , Hon Wai Leong , Qi Tian , Qing Tang , Kong Wah Wan, Trajectory-based ball detection and tracking with applications to semantic analysis of broadcast soccer video, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957018]
• 10: X. Yu, H. W. Leong, C. Xu, and Q. Tian. A robust Hough-based algorithm for partial ellipse detection in broadcast soccer video, Proc. of ICME 2004, Taiwan, June, 2004.
• 11: Xinguo Yu , Hon Wai Leong , Changsheng Xu , Qi Tian, A robust and accumulator-free ellipse hough transform, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027585]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.5 Reconstruction
          Subjects: Transform methods

General Terms:
Algorithms, Design, Experimentation

Keywords:
3D reconstruction, calibration, soccer video, video enrichment