Enabling technologies on hybrid camera networks for behavioral analysis of unattended indoor environments and their surroundings

ABSTRACT

This paper presents a layered network architecture and the enabling technologies for accomplishing vision-based behavioral analysis of unattended environments. Specifically the vision network covers both the attended environment and its surroundings by means of hybrid cameras. The layer overlooking at the surroundings is laid outdoor and tracks people, monitoring entrance/exit points. It recovers the geometry of the site under surveillance and communicates people positions to a higher level layer. The layer monitoring the unattended environment undertakes similar goals, with the addition of maintaining a global mosaic of the observed scene for further understanding. Moreover, it merges information coming from sensors beyond the vision to deepen the understanding or increase the reliability of the system. The behavioral analysis is demanded to a third layer that merges the information received from the two other layers and infers knowledge about what happened, happens and will be likely happening in the environment. The paper also describes a case study that was implemented in the Engineering Campus of the University of Modena and Reggio Emilia, where our surveillance system has been deployed in a computer laboratory which was often unaccessible due to lack of attendance.

References

1. I. Akyildiz, T. Melodia, and K. Chowdury. Wireless multimedia sensor networks:A survey. Wireless Communications, IEEE {see also IEEE Personal Communications}, 14(6): 32--39, December 2007.
2. A. Albiol, I. Mora, and V. Naranjo. Real-time high density people counter using morphological tools. Intelligent Transportation Systems, IEEE Transactions on, 2(4): 204--218, Dec 2001.
3. S. Apewokin, B. Valentine, L. Wills, S. Wills, and A. Gentile. Midground object detection in real world video scenes. IEEE Conference on Advanced Video and Signal based Surveillance AVSS'07., 2007.
4. M. Balcells, D. DeMenthon, and D. Doermann. An appearance-based approach for consistent labeling of humans and objects in video. Pattern Anal. Appl., 7(4): 373--385, 2004.
5. N. Banerjee, M. Corner, and B. Levine. An energy-effcient architecture for dtn throwboxes. INFOCOM 2007. 26th IEEE International Conference on Computer Communications. IEEE, pages 776--784, May 2007.
6. D. Biliotti, G. Antonini, and J. P. Thiran. Multi-layer hierarchical clustering of pedestrian trajectories for automatic counting of people in video sequences. In WACV-MOTION '05:Proceedings of the IEEE Workshop on Motion and Video Computing (WACV/MOTION '05)-Volume 2, pages 50--57, 2005.
7. A. D. Bimbo and F. Pernici. Uncalibrated 3d human tracking with a ptz-camera viewing a plane. In Proc. 3DTV International Conference:Capture, Transmission and Display of 3D Video (3DTV-CON 08), 2008.
8. G. Bradski. Real time face and object tracking as a component of a perceptual user interface. In Proc. of WACV, pages 214--219, 1998.
9. S. Calderara, R. Cucchiara, and A. Prati. Bayesian-competitive consistent labeling for people surveillance. IEEE Trans. Pattern Anal. Mach. Intell., 30(2):354--360, 2008
10. T. P. Chen, H. Haussecker, A. Bovyrin, R. Belenov, K. Rodyushkin, A. Kuranov, and V. Eruhimov. Computer vision workload analysis:Case study of video surveillance systems. Intel Technology Journal - Compute-intensive, highly parallel applications and uses, 9:109--118, 2005
11. M. Chu, J. Reich, and F. Zhao. Distributed attention in large scale video sensor networks. Intelligent Distributed Surveilliance Systems, IEE, pages 61--65, Feb. 2004
12. R. Collins, A. Lipton, H. Fujiyoshi, and T. Kanade. Algorithms for cooperative multisensor surveillance. Proceedings of the IEEE, 89:1456--1477, 2001.
13. D. Comaniciu, V. Ramesh, and P. Meer. Real-time tracking of non-rigid objects using mean shift. Computer Vision and Pattern Recognition, 2000. Proceedings. IEEE Conference on, 2:142--149 vol. 2, 2000
14. R. Cucchiara. Multimedia surveillance systems. In VSSN '05:Proceedings of the third ACM international workshop on Video surveillance & sensor networks, pages 3--10, New York, NY, USA, 2005. ACM.
15. R. Cucchiara, C. Grana, M. Piccardi, and A. Prati. Detecting moving objects, ghosts and shadows in video streams. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(10):1337--1342, Oct. 2003.
16. R. Cucchiara, C. Grana, A. Prati, and R. Vezzani. Computer vision system for in-house video surveillance. Vision, Image and Signal Processing, IEE Proceedings -, 152(2):242--249, April 2005.
17. R. Cucchiara, C. Grana, G. Tardini, and R. Vezzani. Probabilistic people tracking for occlusion handling. In Proceedings of IAPR International Conference on Pattern Recognition (ICPR 2004), pages 132--135, Aug. 2004.
18. R. Cucchiara, A. Prati, R. Vezzani, L. Benini, E. Farella, and P. Zappi. Using a wireless sensor network to enhance video surveillance. Journal of Ubiquitous Computing and Intelligence (JUCI), 1:1--11, 2006.
19. G. Foresti. A real-time system for video surveillance of unattended outdoor environments. Circuits and Systems for Video Technology, IEEE Transactions on, 8(6):697--704, Oct 1998.
20. I. Haritaoglu, D. Harwood, and L. S. Davis. W4: Real-time surveillance of people and their activities. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22:809--830, 2000.
21. S. Khan and M. Shah. Consistent labeling of tracked objects in multiple cameras with overlapping fields of view. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 25(10):1355--1360, Oct. 2003.
22. K. K. Kim, S. H. Cho, H. J. Kim, and J. Y. Lee. Detecting and tracking moving object using an active camera. Advanced Communication Technology, 2005, ICACT 2005. The 7th International Conference on, 2:817--820, 2005.
23. X. Li. Contour-based object tracking with occlusion handling in video acquired using mobile cameras. IEEE Trans. Pattern Anal. Mach. Intell., 26:1531--1536, 2004. Member-Alper Yilmaz and Fellow-Mubarak Shah.
24. L. lo Presti and M. L. Cascia. Real-time estimation of geometrical transformation between views in distributed smart-cameras systems. International Conference on Distributed Smart Cameras -ICDSC08, in press.
25. D. G. Lowe. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision, 60(2):91--110, 2004.
26. A. Mainwaring, D. Culler, J. Polastre, R. Szewczyk, and J. Anderson. Wireless sensor networks for habitat monitoring. In WSNA '02:Proceedings of the 1st ACM international workshop on Wireless sensor networks and applications, pages 88--97. ACM, 2002.
27. A. Maki, P. Nordlund, and J. -O. Eklundh. Attentional scene segmentation:integrating depth and motion. Comput. Vis. ImageUnderst., 78(3):351--373, 2000.
28. L. Marcenaro, F. Oberti, G. Foresti, and C. Regazzoni. Distributed architectures and logical-task decomposition in multimedia surveillance systems. Proceedings of the IEEE, 89(10):1419--1440, Oct 2001.
29. O. Masoud and N. Papanikolopoulos. A novel method for tracking and counting pedestrians in real-time using a single camera. Vehicular Technology, IEEE Transactions on, 50(5):1267--1278, Sep 2001.
30. A. Mittal and N. Paragios. Motion-based background subtraction using adaptive kernel density estimation. Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on, 2:II-302-II-309 Vol. 2, June-2 July 2004.
31. T. B. Moeslund, A. Hilton, and V. Krüger. A survey of advances in vision-based human motion capture and analysis. Comput. Vis. ImageUnderst., 104(2):90--126, 2006.
32. D. Murray and A. Basu. Motion tracking with an active camera. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 16(5):449--459, May 1994.
33. D. Ostheimer, S. Lemay, M. Ghazal, D. Mayisela, A. Amer, and P. F. Dagba. A modular distributed video surveillance system over ip. In CCECE, pages 518--521. IEEE, 2006.
34. P. Pérez, C. Hue, J. Vermaak, andM. Gangnet. Color-based probabilistic tracking. In ECCV '02: Proceedings of the 7th European Conference on Computer Vision-Part I , pages 661--675, London, UK, 2002.
35. M. Piccardi. Background subtraction techniques: a review. IEEE International Conference on Systems, Man and Cybernetics, 4:3099--3104, 2004.
36. M. Piccardi and E. D. Cheng. Multi-frame moving object track matching based on an incremental major color spectrum histogram matching algorithm. In CVPR '05:Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '05)-Workshops, 2005.
37. A. Prati, F. Seghedoni, and R. Cucchiara. Fast dynamic mosaicing and person following. In Proc. of International Conference on Pattern Recognition (ICPR 2006), volume 4, pages 920--923, Aug. 2006.
38. L. L. Presti and M. L. Cascia. Real-time object detection in embedded video surveillance systems. 9th International Workshop on Image Analysis for Multimedia Interactive Services, IEEE Proceedings, 2008.
39. L. Snidaro, C. Micheloni, and C. Chiavedale. Video security for ambient intelligence. Systems, Man and Cybernetics, Part A, IEEE Transactions on, 35:133--144, Jan. 2005.
40. C. Stauffer and W. Grimson. Adaptive background mixture models for real-time tracking. Conference on Computer Vision and Pattern Recognition '99, IEEE Computer Society, 2:246--252, 1999.
41. M. Valera and S. A. Velastin. Intelligent distributed surveillance systems:a review. Vision, Image and Signal Processing, IEEE Proceedings, 152:192--204, 2005.
42. T. Yang, F. Chen, D. Kimber, and J. Vaughan. Robust people detection and tracking in a multi-camera indoor visual surveillance system. Multimedia and Expo, 2007 IEEE International Conference on, pages 675--678, July 2007.
43. X. Yuan, Z. Sun, Y. Varol, and G. Bebis. A distributed visual surveillance system. In AVSS '03: Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, page 199, Washington, DC, USA, 2003. IEEE Computer Society.
44. X. Zhou, R. T. Collins, T. Kanade, and P. Metes. A master-slave system to acquire biometric imagery of humans at distance. In IWVS '03: First ACM SIGMM international workshop on Video surveillance, pages 113--120, 2003.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

      • Computer vision tasks

        • Scene understanding