An object-based video coding framework for video sequences obtained from static cameras

ABSTRACT

This paper presents a novel object-based video coding framework for videos obtained from a static camera. As opposed to most existing methods, the proposed method does not require explicit 2D or 3D models of objects and hence is general enough to cater for varying types of objects in the scene. The proposed system detects and tracks objects in the scene and learns the appearance model of each object online using incremental principal component analysis (IPCA). Each object is then coded using the coefficients of the most significant principal components of its learned appearance space. Due to smooth transitions between limited number of poses of an object, usually a limited number of significant principal components contribute to most of the variance in the object's appearance space and therefore only a small number of coefficients are required to code the object. The rigid component of the object's motion is coded in terms of its affine parameters. The framework is applied to compressing videos in surveillance and video phone domains. The proposed method is evaluated on videos containing a variety of scenarios such as multiple objects undergoing occlusion, splitting, merging, entering and exiting, as well as a changing background. Results on standard MPEG-7 videos are also presented. For all the videos, the proposed method displays higher Peak Signal to Noise Ratio (PSNR) compared to MPEG-2 and MPEG-4 methods, and provides comparable or better compression.

References

1. Y. Altunbasak and A. M. Tekalp, "Occlusion-adaptive content-based 2-D mesh design and tracking for object-based coding," In IEEE Transactions on Image Processing, vol. 6, no. 9, pp.1270--1280, 1997.
2. K. Aizawa and T. Huang, "Model Based Image Coding: Advanced Video Coding Techniques for Very Low Bit-Rate Applications," In Proceedings of the IEEE, vol. 83, no. 2, pp.259--271, 1995.
3. J. Bergen, P. Anandan, K. Hanna, and R. Hingorani. "Hierarchical model-based motion estimation". In European Conference on Computer Vision, pp.237--252, 1992.
4. Buck, "Segmentation of moving head-and-shoulder shapes," In Picture Coding Symposium, Boston, 1990.
5. C. S. Choi and T. Tekebe, "Analysis and synthesis of facial image sequences in model-based image coding," In IEEE Transactions on Video Technology, pp.257--275, 1994.
6. D. DeCarlo, D. Metaxas, and M.Stone, "An Anthropometric Face Model using Variational Techniques," In Proc. SIGGRAPH, pp.67--74, 1998.
7. O. Javed, K. Shafique, and M. Shah. "A hierarchical approach to robust background subtraction using color and gradient Information". In Workshop on Motion and Video Computing, pp.22--27, 2002.
8. O. Javed, Z. Rasheed, O. Alatas, and M. Shah, "KNIGHTM:A Real Time Surveillance System for Multiple Overlapping and Non-Overlapping Cameras". In The Fourth International Conference on Multimedia and Expo, Baltimore, Maryland, 2003.
9. O. Javed, and M. Shah. "Tracking and Object Classification for Automated Surveillance". In European Conference on Computer Vision, pp.343--357, 2002.
10. I. T. Jolliffe. "Principal component analysis". New York: Springer-Verlag, 1986.
11. R. Koch, "Dynamic 3D scene analysis through synthesis feedback control", In IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 15(6), June, 1993.
12. I. Martins and L. Corte-Real, "A Video-coder using 3D model-based background for video surveillance applications,". In IEEE Internation Conference on Image Processing, pp.919--923, 1998.
13. G. Menegaz and J. Thiran, "Lossy to lossless object-based coding of 3D MRI data". In IEEE Transactions on Image Processing, Vol.11(9), pp.1053--1061, 2002.
14. Y. Nakaya, K. Aizawa, and H. Harashima, "Texture updating methods in model-based coding of facial images,". In Picture Coding Symposium, Boston, 1990.
15. C. Staffer, and W. E. L. Grimson. "Learning patterns of actvity using real-time tracking." In IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.22(8), pp.747--757, 2000.
16. G. Strang. "Linear algebra and its applications". New York: Academic, 1980.
17. M. G. Strinzis, "Object-based coding of stereospic and 3D image sequences,". In IEEE Signal Processing Magazine, pp.14--28, 1999.
18. M. J. Turk, and A. Pentland. "Eigenfaces for recognition". In Workshop on Human Computer Interaction, Vol.3, pp.71--86, 1991.
19. A. Vetro, T Haga, K. Sumi, and H. Sun, "Object based coding for long term archive of surveillance video". In Technical Report, TR-2003-98, MERL, July 2003.
20. W. J. Welsh "Model-based coding of videophone images," In Electronic and Communication Engineering Journal, Vol.3(1), pp.29--36, 1991.
21. J. Weng, Y. Zhang, and W. Hwang. "Candid covariance-free incremental principal component analysis". In IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.25(8), pp.1034--1040, 2003.
22. Y. Zhang, and J. Weng. "Convergence analysis of complementary candid incremental principal component analysis". Technical Report MSU-CSE-01-23, Michigan State University, East Lansing, 2001.
23. A. Yilmaz, X. Li, and M. Shah. "Contour-based object tracking with occlusion handling in video acquired using mobile cameras". In IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.26(11), 2004.
24. A. L. Yuille, P. W. Hallina, and D. S. Cohen, "Feature extraction from faces using deformable templates," In International Journal of Computer Vision, Vol.8(2), pp.99--111, 1992.
25. "CAVIAR: Context Aware Vision using Image-based Active Recognition", Downloadable from http://homepages.inf.ed.ac.uk/rbf/CAVIAR/.
26. "PETS: Performance Evaluation of Tracking and Surveillance", Downloadable from http://pets2002.visualsurveillance.org/.

Index Terms

• Computing methodologies

  • Computer graphics

    • Image compression

• Information systems

  • Data management systems

    • Data structures

      • Data layout

        • Data compression

  • World Wide Web

    • Web applications

      • Internet communications tools