Recognizing pair-activities by causality analysis

Abstract

In this article, beyond solo-activity analysis for single object, we study the more complicated pair-activity recognition problem by exploring the relationship between two active objects based on their trajectory clues obtained from video sensor. Our contributions are three-fold. First, we design two sets of features for representing the pair-activities encoded as length-variable trajectory pairs. One set characterizes the strength of causality between two trajectories, for example, the causality ratio and feedback ratio based on the Granger Causality Test (GCT), and another set describes the style of causality between two trajectories, for example, the sampled frequency responses of the digital filter with these two trajectories as the input and output discrete signals respectively. These features along with conventional velocity and position features of a trajectory-pair are essentially of multi-modalities, and may be greatly different in scales and importance. To make full use of them, we then develop a novel feature fusing procedure to learn the coefficients for weighting these features by maximizing the discriminating power measured by weighted correlation. Finally, we collected a pair-activity database of five popular categories, each of which consists of about 170 instances. The extensive experiments on this database validate the effectiveness of the designed features for pair-activity representation, and also demonstrate that the proposed feature fusing procedure significantly boosts the pair-activity classification accuracy.

References

1. Belhumeur, P., Hespanha, J., and Kriegman, D. 1997. Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Trans. Patt. Anal. Mach. Intell. 19, 711--720.
2. Boiman, O. and Irani, M. 2005. Detecting irregularities in images and in video. IEEE Trans. Patt. Anal. Mach. Intell. 462--469.
3. Brand, M., Oliver, N., and Pentland, A. 1997. Coupled hidden markov models for complex action recognition. IEEE Conf. Comput. Vis. Patt. Recog. 994--999.
4. Chang, C. and Lin, C. 2001. LIBSVM: a library for support vector machines, http://www.csie.ntu.edu.tw/~cjlin/libsvm
5. Collins, R. 2003. Mean-shift blob tracking through scale space. In Procedings of the IEEE Conference on Computer Vision and Pattern Recognition. 234--240.
6. Comaniciu, D., Ramesh, V., and Meer, P. 2003. Kernel-based object tracking. IEEE Trans. Patt. Anal. Mach. Intell. 25, 564--574.
7. Granger, C. 1969. Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37, 424--38.
8. Haritaoglu, I., Harwood, D., and David, L. 2000. W4: Real-time surveillance of people and their activities. IEEE Trans. Patt. Anal. Mach. Intell. 22, 809--830.
9. Ivanov, Y. and Bobick, A. 2000. Recognition of visual activities and interactions by stochastic parsing. IEEE Trans. Patt. Anal. Mach. Intell. 22, 852--872.
10. Lin, D., Yan, S., and Tang, X. 2005. Comparative study: face recognition on unspecific persons using linear subspace methods. In Proceedings of the IEEE International Conference on Image Process. 3, 764--767.
11. Medioni, G., Cohen, I., Bremond, F., Hongeng, S., and Nevatia, R. 2001. Event detection and analysis from video streams. IEEE Trans. Patt. Anal. Mach. Intell. 23, 873--889.
12. Mitra, S. 2005. Digital Signal Processing: A Computer-Based Approach. McGraw-Hill.
13. Moore, D., Essa, I., and Haye, M. 1999. Exploiting human actions and object context for recognition tasks. In Proceedings of the IEEE International Conference on Computer Vision. 80--86.
14. Osuna, E., Freund, R., and Girosi, F. 1997. Training support vector machine: An application to face detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern. 130--136.
15. Park, S., and Aggarwal, J. 2004. A hierarchical Bayesian network for event recognition of human actions and interactions. Multimed. Syst. 10, 164--179.
16. Sims, C. 1972. Money, income, and causality. Amer. Econ. Rev. 540--552.
17. Stauffer, C. and Grimson, W. 2000. Learning patterns of activity using real-time tracking. IEEE Trans. Patt. Anal. Mach. Intell. 22, 747--757.
18. Turaga, P., Veeraraghavan, A., and Chellappa, R. 2007. From videos to verbs: Mining videos for activities using a cascade of dynamical systems. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
19. Viola, P. and Jones, M. 2004. Robust real-time face detection. Int. J. Comput. Vis. 137--154.
20. Wren, C., Azarbayejani, A., Darrell, T., and Pentland, A. 1997. Pfinder: Real-time tracking of the human body. IEEE Trans. Patt. Anal. Mach. Intell. 780--785.
21. Wu, J., Osuntogun, A., Choudhury, T., Philipose, M., and Rehg, J. 2007. A scalable approach to activity recognition based on object use. In Proceedings of the International Conference on Computer Vision.
22. Yan, S., Xu, D., Zhang, B., Zhang, H., Yang, Q., and Lin, S. 2007. Graph embedding and extensions: A general framework for dimensionality reduction. IEEE Trans. Patt. Anal. Mach. Intell. 29, 40-51.
23. Zhong, H., Shi, J., and Visontai, M. 2004. Detecting unusual activity in video. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, 819--826.
24. Zhou, Y., Yan, S., and Huang, T. 2007. Detecting Anomaly in Videos from Trajectory Similarity Analysis. In Proceedings of the IEEE International Conference on Multimedia and Expo.

Index Terms

• Computing methodologies

  • Machine learning

• Information systems

  • Information retrieval