Video Surveillance

Summary

Increasingly, large networks of surveillance cameras are employed to monitor public and private facilities. This continuous collection of imagery has the potential for tremendous impact on public safety and security. Unfortunately, this potential is often unrealized since manual monitoring of growing numbers of video feeds is not feasible. As a consequence, surveillance video is mostly stored without being viewed and is only used for data-mining and forensic needs. However, the ability to perform computer-based video analytics is now becoming possible, enabling a proactive approach where security personnel can be continually appraised of who is on site, where they are, and what they are doing. Under this new paradigm, a significantly higher level of security can be achieved through the increased productivity of security officers. The ultimate goal of intelligent video for security and surveillance is to automatically detect events and situations that require the attention of security personnel. Augmenting security staff with automatic processing will increase their efficiency and effectiveness. This is a difficult problem since events of interest are complicated and diverse. (Tu et al., 2007; Krahnstoever et al., 2009) discuss some of the challenges of developing surveillance systems, and present an overview of some solutions concerning with people detection, crowd analysis, multi-camera multi-target tracking, event detection, indexing, and search.

Dynamic Background Modeling

In order to extract the desired higher-level information, as an intermediate step, several video analysis tasks rely on modeling the background in order to detect the presence of foreground objects of interest. While several methods are available for simple scenarios, the case of a moving camera, observing objects moving in a scene with severe motion clutter, is still considered a challenge. (Kim et al., 2009) addresses this issue by providing a model for the background that takes into account the camera motion, as well as the motion clutter. Detecting a foreground object is equivalent to detecting a model change. This is done optimally online by exploiting the sequential generalized likelihood ratio test, applied to the sufficient test statistic that describes the motion clutter.

References

1. AVSS

   A model change detection approach to dynamic scene modeling Kim, S. J., Doretto, G., Rittscher, J., Tu, P., Krahnstoever, N., and Pollefeys, M. In IEEE International Conference on Advanced Video and Signal Based Surveillance, 2009. Oral abstract bibTeX pdf

   In this work we propose a dynamic scene model to provide information about the presence of salient motion in the scene, and that could be used for focusing the attention of a pan/tilt/zoom camera, or for background modeling purposes. Rather than proposing a set of saliency detectors, we define what we mean by salient motion, and propose a precise model for it. Detecting salient motion becomes equivalent to detecting a model change. We derive optimal online procedures to solve this problem, which enable a very fast implementation. Promising results show that our model can effectively detect salient motion even in severely cluttered scenes, and while a camera is panning and tilting.

   @inproceedings{kimDRTKP09avss,
     abbr = {AVSS},
     author = {Kim, S. J. and Doretto, G. and Rittscher, J. and Tu, P. and Krahnstoever, N. and Pollefeys, M.},
     title = {A model change detection approach to dynamic scene modeling},
     booktitle = {IEEE International Conference on Advanced Video and Signal Based
     	Surveillance},
     year = {2009},
     pages = {490--495},
     month = sep,
     bib2html_pubtype = {Conferences},
     bib2html_rescat = {Video Analysis, Dynamic Textures, Visual Motion Detection},
     file = {kimDRTKP09avss.pdf:doretto\\conference\\kimDRTKP09avss.pdf:PDF},
     keywords = {dynamic scene modeling, video analysis, PTZ camera, focus-ofattention, background
     	modeling, sequential generalized likelihood ratio, model change detection,
     	linear dynamical systems, dynamic textures},
     owner = {doretto},
     timestamp = {2009.09.28},
     wwwnote = {Oral}
   }
   

2. AVSS

   Intelligent video for protecting crowded sports venues Krahnstoever, N., Tu, P., Yu, T., Patwardhan, K., Hamilton, D., Yu, B., Greco, C., and Doretto, G. In IEEE International Conference on Advanced Video and Signal Based Surveillance, 2009. (Invited paper) Oral bibTeX pdf

   @inproceedings{krahnstoeverTYPHYGD09avss,
     abbr = {AVSS},
     author = {Krahnstoever, N. and Tu, P. and Yu, T. and Patwardhan, K. and Hamilton, D. and Yu, B. and Greco, C. and Doretto, G.},
     title = {Intelligent video for protecting crowded sports venues},
     booktitle = {IEEE International Conference on Advanced Video and Signal Based
     	Surveillance},
     year = {2009},
     pages = {116--121},
     month = sep,
     note = {(Invited paper)},
     bib2html_pubtype = {Conferences},
     bib2html_rescat = {Video Analysis},
     file = {krahnstoeverTYPHYGD09avss.pdf:doretto\\conference\\krahnstoeverTYPHYGD09avss.pdf:PDF},
     owner = {doretto},
     timestamp = {2009.09.28},
     wwwnote = {Oral}
   }
   

3. SPIE DSS

   An intelligent video framework for homeland protection Tu, P. H., Doretto, G., Krahnstoever, N. O., Perera, A. A. G., Wheeler, F. W., Liu, X., Rittscher, J., Sebastian, T. B., Yu, T., and Harding, K. G. In Proceedings of SPIE Defence and Security Symposium - Unattended Ground, Sea, and Air Sensor Technologies and Applications IX, 2007. (Invited paper) abstract bibTeX pdf

   This paper presents an overview of Intelligent Video work currently under development at the GE Global Research Center and other research institutes. The image formation process is discussed in terms of illumination, methods for automatic camera calibration and lessons learned from machine vision. A variety of approaches for person detection are presented. Crowd segmentation methods enabling the tracking of individuals through dense environments such as retail and mass transit sites are discussed. It is shown how signature generation based on gross appearance can be used to reacquire targets as they leave and enter disjoint fields of view. Camera calibration information is used to further constrain the detection of people and to synthesize a top-view, which fuses all camera views into a composite representation. It is shown how site-wide tracking can be performed in this unified framework. Human faces are an important feature as both a biometric identifier and as a method for determining the focus of attention via head pose estimation. It is shown how automatic pantilt-zoom control; active shape/appearance models and super-resolution methods can be used to enhance the face capture and analysis problem. A discussion of additional features that can be used for inferring intent is given. These include body-part motion cues and physiological phenomena such as thermal images of the face.

   @conference{tuDKPWLRSYH07SPIEdss,
     abbr = {SPIE DSS},
     author = {Tu, P. H. and Doretto, G. and Krahnstoever, N. O. and Perera, A. A. G. and Wheeler, F. W. and Liu, X. and Rittscher, J. and Sebastian, T. B. and Yu, T. and Harding, K. G.},
     title = {An intelligent video framework for homeland protection},
     booktitle = {Proceedings of SPIE Defence and Security Symposium - Unattended Ground,
     	Sea, and Air Sensor Technologies and Applications IX},
     year = {2007},
     editor = {Carapezza, E. M.},
     volume = {6562},
     address = {Orlando, FL, USA},
     month = apr,
     note = {(Invited paper)},
     bib2html_pubtype = {Conferences},
     bib2html_rescat = {Video Analysis},
     file = {tuDKPWLRSYH07SPIEdss.pdf:doretto\\conference\\tuDKPWLRSYH07SPIEdss.pdf:PDF;tuDKPWLRSYH07SPIEdss.pdf:doretto\\conference\\tuDKPWLRSYH07SPIEdss.pdf:PDF},
     keywords = {intelligent video, surveillance, camera calibration, person detection,
     	crowd segmentation, site-wide tracking, person reidentification,
     	face modeling, face super-resolution, deception detection},
     owner = {doretto},
     timestamp = {2007.01.20}
   }