Xiao Wu
ABSTRACT
This paper presents a novel motion localization approach for recognizing actions and events in real videos. Examples include StandUp and Kiss in Hollywood movies. The challenge can be attributed to the large visual and motion variations imposed by realistic action poses. Previous works mainly focus on learning from descriptors of cuboids around space time interest points (STIP) to characterize actions. The size, shape and space-time position of cuboids are fixed without considering the underlying motion dynamics. This often results in large set of fragmentized cuboids which fail to capture long-term dynamic properties of realistic actions. This paper proposes the detection of spatio-temporal motion volumes (namely Volume of Interest, VOI) of scale and position adaptive to localize actions. First, motions are described as bags of point trajectories by tracking keypoints along the time dimension. VOIs are then adaptively extracted by clustering trajectory on the motion mainfold. The resulting VOIs, of varying scales and centering at arbitrary positions depending on motion dynamics, are eventually described by SIFT and 3D gradient features for action recognition. Comparing with fixed-size cuboids, VOI allows comprehensive modeling of long-term motion and shows better capability in capturing contextual information associated with motion dynamics. Experiments on a realistic Hollywood movie dataset show that the proposed approach can achieve 20\% relative improvement compared to the state-of-the-art STIP based algorithm.
- J. Sun, X. Wu, SC. Yan, LF. Cheong, TS. Chua and J. Li. Hierarchical spatio-temporal context modeling for action recognition. CVPR, 2009.Google Scholar
- F. Wang, Y. Jiang, and C. Ngo. Video event detection using motion relativity and visual relatedness. ACM Multimedia, 2008. Google ScholarDigital Library
- J. Liu, J. Luo, et al. Recognizing realistic actions from videos in the Wild. CVPR, 2009.Google ScholarCross Ref
- B. Morris, et al. A survey of vision-based trajectory learning and analysis for surveillance. TCSVT, 2008. Google ScholarDigital Library
- I. Laptev, M. Marsza lek, C. Schmid, et al. Learning realistic human actions from movies. CVPR, 2008.Google ScholarCross Ref
- D. Batra, T. Chen and R. Sukthankar. Space-Time shapelets for action recognition. IEEE WMVC, 2008. Google ScholarDigital Library
- L. Gorelick, M. Blank, E. Shechtman, et al. Actions as space-time shapes. TPAMI, 2007. Google ScholarDigital Library
- R. Tron, et al. A benchmark for the comparison of 3D motion segmentation algorithms. CVPR, 2008.Google Scholar
- Y. Cheng, et al. Mean shift, mode seeking, and clustering. TPAMI, 1995. Google ScholarDigital Library
- P. Dollar, V. Rabaud, et al. Behavior recognition via sparse spatio-temporal features. In VS-PETS, 2005.Google ScholarCross Ref
- X. Wu, Y. Zhang, Y. Wu, J. Guo and J. Li. Invariant visual patterns for video copy detection. ICPR, 2008.Google Scholar
- OpenCV: sourceforge.net/projects/opencvlibrary.Google Scholar
Index Terms
- Localizing volumetric motion for action recognition in realistic videos
Recommendations
Motion keypoint trajectory and covariance descriptor for human action recognition
Human action recognition from videos is a challenging task in computer vision. In recent years, histogram-based descriptors that are calculated along dense trajectories have shown promising results for human action recognition, but they usually ignore ...
Hierarchical Filtered Motion for Action Recognition in Crowded Videos
Action recognition with cluttered and moving background is a challenging problem. One main difficulty lies in the fact that the motion field in an action region is contaminated by the background motions. We propose a hierarchical filtered motion (HFM) ...
Human action segmentation and recognition via motion and shape analysis
In this paper, we present an automated video analysis system which addresses segmentation and detection of human actions in an indoor environment, such as a gym. The system aims at segmenting different movements from the input video and recognizing the ...
Comments