A motion-flow-based fast video retrieval system

ABSTRACT

In this paper, we propose the use of motion vectors embedded in MPEG bitstreams to generate so-called "motion-flows", which are applied them to perform quick video retrieval. By using the motion vectors directly, we do not need to consider the shape of a moving object and its corresponding trajectory. Instead, we simply "link" the local motion vectors across consecutive video frames to form motion flows, which are then annotated and stored in a video database. In the video retrieval phase, we propose a coarse-to-fine strategy to execute the video retrieval task. Motions that do not belong to the main-stream motion flows are filtered out by our proposed algorithm. The retrieval process can be triggered by query-by-sketch (QBS) or query-by-example (QBE). The experiment results show that our method is indeed efficient and accurate in the video retrieval process.

REFERENCES

• 1: Myron Flickner , Harpreet Sawhney , Wayne Niblack , Jonathan Ashley , Qian Huang , Byron Dom , Monika Gorkani , Jim Hafner , Denis Lee , Dragutin Petkovic , David Steele , Peter Yanker, Query by Image and Video Content: The QBIC System, Computer, v.28 n.9, p.23-32, September 1995 [doi>10.1109/2.410146 ]
• 2: S. F. Chang, W. Chen, H. J. Meng, H. Sundaram, D. Zhong, "A fully automated content-based video search engine supporting spatiotemporal queries," IEEE Transactions on Circuits and Systems for Video Technology, Vol.8, No.5, pp.602--615, 1998.
• 3: D. H. Douglas and T. K. Peucker, "Algorithms for the reduction of the number of points required to represent a digitized line or its caricature," The Canadian Cartographer, vol.10, No.2, pp.112--122, 1973.
• 4: R. Wang and T. Huang, "Fast camera motion analysis in MPEG domain," ICIP, vol.3, pp.691--694, Oct. 1999.
• 5: S. Dagtas, W. Al-Khatib, A. Ghafoor, R.L. Kashyap, "Models for motion-based video indexing and retrieval," IEEE Transactions on Image Processing, vol.9, No.1, pp.88--101, 2000.
• 6: R. Fablet, P. Bouthemy, P. Perez, "Nonparametric motion characterization using causal probabilistic models for video indexing and retrieval," IEEE Transactions on Image Processing, vol.11, No.4, pp.393--407, 2002.
• 7: A. Pentland , R. W. Picard , S. Sclaroff, Photobook: content-based manipulation of image databases, International Journal of Computer Vision, v.18 n.3, p.233-254, June 1996 [doi>10.1007/BF00123143]
• 8: John R. Smith , Shih-Fu Chang, VisualSEEk: a fully automated content-based image query system, Proceedings of the fourth ACM international conference on Multimedia, p.87-98, November 18-22, 1996, Boston, Massachusetts, United States [doi>10.1145/244130.244151]
• 9: A. Hamrapur, A. Gupta, B. Horowitz, C.F. Shu, C. Fuller, J. Bach, M. Gorkani, R. Jain, "Virage Video Engine," SPIE Proceedings on Storage and Retrieval for Image and Video Databases V, pp.188--197, San Jose, Feb. 1997.
• 10: Y. Deng and B. S. Manjunath, "NeTra-V: Toward an Object-based Video Representation," IEEE Transactions on Circuits and Systems for Video Technology, Vol. 8, No.5, pp.616--627, Sept. 1998.
• 11: Yu-Fei Ma, Hong-Jiang Zhang, "Motion texture: a new motion based video representation," 16th International Conference on Pattern Recognition, vol. 2, pp.548--551, 11-15 Aug. 2002.
• 12: Dong-Jun Lan, Yu-Fei Ma, Hong-Jiang Zhang, "A novel motion-based representation for video mining," International Conference on Multimedia and Expo, vol. 3, pp.469--472, 6-9 July. 2003.
• 13: B.S. Manjunath , Phillipe Salembier , Thomas Sikora, Introduction to MPEG-7: Multimedia Content Description Interface, John Wiley & Sons, Inc., New York, NY, 2002
• 14: Y. Tsaig, A. Averbuch, "Automatic segmentation of moving objects in video sequences: a region labeling approach," IEEE Transactions on Circuits and Systems for Video Technology, vol. 12, no.7, pp.597--612, July. 2002.
• 15: V. Mezaris, I. Kompatsiaris, M.G. Strintzis, "Video object segmentation using Bayes-based temporal tracking and trajectory-based region merging," IEEE Transactions on Circuits and Systems for Video Technology, vol.14, no.6, pp.782--795, June. 2004.
• 16: R. Cucchiara, C. Grana, M. Piccardi, A. Prati, "Detecting moving objects, ghosts, and shadows in video streams," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.25, no.10, pp.1337--1342, Oct. 2003.
• 17: Cen Rao , Alexei Gritai , Mubarak Shah , Tanveer Syeda-Mahmood, View-invariant Alignment and Matching of Video Sequences, Proceedings of the Ninth IEEE International Conference on Computer Vision, p.939, October 13-16, 2003
• 18: Dimitrios Gunopoulos, Discovering Similar Multidimensional Trajectories, Proceedings of the 18th International Conference on Data Engineering, p.673, February 26-March 01, 2002
• 19: http://video.google.com/
• 20: http://search.yahoo.com/

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing

General Terms:
Experimentation

Keywords:
motion analysis, video retrieval