Computer Vision and Pattern Recognition in Homeland Security Applications 1 Giovanni B. Garibotto Elsag spa Genova, Italy giovanni.garibotto@elsagdatamat.com Abstract. The tutorial will summarize the status of research and innovation in the field of Security of Computer Vision and Pattern Recognition Technology. Two main research areas are considered: intelligent scene analysis in videosurveillance, and mobile Automatic Number Plate recognition ANPR, for investigation and crime prevention. The lecture will refer the most recent advances of mobile ANPR solutions on board of patrol car as well as portable hand-held devices to improve mobility and flexibility. From the patrol car it is possible to collect vehicle information within the traffic flow with a performance that far exceeds human detection and recognition capabilities in all weather conditions and 24 h operation. Such a solution is currently used by most advanced police departments in the world. Keywords: Computer Vision, Pattern Recognition, Video Surveillance & Security Applications. 1 Intelligent Video Analysis Systems A common objective of new advanced Video Analysis System is the possibility to describe in a synthetic high-level what happens in the framed scene. Basic components of such description are the objects that are found in the scene and their movement (trajectories) during time. Another essential requirement is the possibility of reasoning in a 3D reference coordinate system and using a volumetric representation of objects (size and shape) beside radiometric color features. Such a representation should be quite similar to the human description in terms of basic primitives like the appearance of a target (when and where) its trajectory (in a reference plane), its persistence in the scene coordinates (how long). Based on such low-level primitives it is possible to build any higher level interpretation of the overall behavior and sequence of the main events in the scene, depending on the specific content and application objectives. The accuracy of people/object detection and tracking is particularly important for a variety of situations like people flow or density estimation, detection of objects (of a certain size and shape), and their condition, to be left or removed from the scene. In this research field, people modeling is a key issue 1 Proceedings of the 7 th International Workshop on Fuzzy Logic and Applications, WILF 2007, Camogli, Italy, July 7-10, 2007.
for the description of flexible and complex objects to be detected and tracked. There is a wide literature dealing with such problem. A recent interesting discussion of the problem and the most advance proposed solution can be found in the special issue [1]. Among other important initiatives in this area, it is worth to remark the Performance Evaluation workshop PETS that is organized on a regular basis (the last event has been hold in New York last June 18, 2006 [2]), where most advanced research labs in the world are involved to present the last achievements of their studies. Moreover, this topic of people detection and tracking has a great interest also in the industrial domain [3] for a variety of applications from security and crime prevention, to statistical commercial analysis, up to entertainment and sports applications. 2 3D model-based target tracking The main processing steps of a vision system for target detection and tracking are briefly summarized in the following. 1. Calibration of the involved cameras (possibly multi-camera systems) w.r.t the 3D world coordinates, with strong requirements to be robust, fast and easy to use, in practical installations. It is also required to provide the system with a sufficient level of self-awareness to immediately detect and possibly correct any deviation from the initial installation conditions. 2. Foreground/background segmentation and background updating with adaptive control to compensate the variability of light and environmental noise (shadows, highlights, etc.). A highly sensitive event detection system is of the required, even in very low contrast conditions (this is particularly important for intruder detection). 3. Target tracking using a variety of geometric and radiometric features with a suitable 3D target model and an appropriate state model including position and motion parameters 4. New target detection process for the localization of potential new objects and targets in the scene at any stage of the process (including the boot-strap situation). 5. Low-level data fusion to remove local ambiguities and minimize the number of false alarms and missed targets. 6. High-level data integration and event detection; this processing step is highly dependent on the current application with some specific constraints and contextual information. The proposed solution is based on a simple volumetric representation of the targets and their detection, prediction and tracking in the reference 3D system, by using multiple camera views. An essential component of the process is camera calibration including both intrinsic as well as extrinsic parameters. The area to be inspected may be quite large and it may require the installation and activation of multiple cameras to cover the scene from different multi-view positions to guarantee the correct detection and tracking of multiple objects (of different classes) in the 3D scene. The considered application is wide-field target detection and tracking, where the targets are supposed to be quite small with poor details. In high resolution
applications with close-up video recording, human body representations require a much complex model scheme to describe their non-rigid properties. In the proposed approach new candidate objects are detected at each new time step in the different camera views according to their projected silhouettes of the foreground map (color segmentation is performed, with shadow detection and removal). A local search is performed around the predicted position to refine the object estimate using color feature representation [6], of the different class of objects (from the larger to the smaller and from the closer to the farthest). Figure1 vanishing geometry on the camera view Figure 2. People tracking from outdoor scene with strong highlight Some of the targets are temporarily lost during the video sequence The automatic detection of abnormal behavior represents one of the main objectives of any new intelligent video surveillance system to drive the human attention only when relevant actions take place. An essential component of an advance video analysis system is the detection and tracking of people in a video sequence in all environmental conditions.
Fig. 2 shows an example of people detection and tracking by using a 3D model based approach. An essential component of the process is the accurate calibration of the scene and the video camera system since the tracking process is performed in the 3D scene. The target models are then re-projected into the image plane to verify their correct position in the image view. The use of a very simple symmetric 3D target model has proved to be appropriate for people counting with a sufficient precision even in very crowded and complex situations. The accuracy of target position in the 3D reference coordinate system is highly dependent on the image resolution in wide-field cameras and it may be definitely improved by multi-camera configuration. Further research efforts will be devoted to improve the robustness of the full processing chain. The final goal is the design of an easy to use, self-consistent system to be managed by security personnel, in real applications, in both indoor and out-door environments. 3 License Plate Recognition in Security Applications Increasing traffic density and congestion represents one of the main problems of everyday life. Motorists lose time and money in the cues, and safety and security are often compromised. License-plate recognition (LPR) technology, often represents a solution to save time and alleviate congestion by allowing motorists to pass toll plazas or weigh stations without stopping. It can improve safety and security by helping control access to secured areas or assisting patrol in law enforcement. LPR is a consolidated technology (see the tutorial in (1)), widely used in a variety of ITS applications since License plates are the only universal identification device from vehicles to roadside. From the long list of suppliers it is possible to identify two different types of products, i.e. software packages of Optical Character Recognition (OCR), mainly oriented to system developers, and integrated systems, where OCR is a component of complete solutions, including image sensors and lighting, and special image processors, for Traffic Control or Police Security applications. Most referred applications are based on fixed installations for access control, electronic pay-toll collection and law-enforcement (red-light violation, average speed control) and security control (stolen-car detection and crime investigation). Currently, the new challenge in ITS applications is based on mobile LPR systems, installed on standard patrol vehicles, to increase data-collection capabilities and extend the inspection field during normal patrol missions. The mobile system may identify particular vehicles of interest which may be hidden amongst the huge volume of traffic using our roads today. The recognition task of a moving platform is much more complex than for fixed urban or highway traffic control installations, and represents a real challenge for Computer Vision. In fact all critical situations are concurrently present, like sudden, unpredictable changes of lighting conditions during patrols (transition from sunlit to shadow areas, tunnels, night patrols, etc.), arbitrary license plate 3D orientation, fast real-time processing requirements.
4 Mobile ANPR system Auto-Detector [8] is a new mobile Automatic Number Plate Recognition system, installed on board of any kind of patrol vehicle (car, motor-vehicle), to automatically detect and read the license plates of the vehicles falling in its field of view. Figure.3 Example of Auto-Detector installation for the Italian Carabinieri-Arm, the Italian Road Police, and USA State Highway Patrol forces. The innovation content of Auto-Detector is twofold: it is a new service solution for security and surveillance monitoring; moreover it represents a significant
innovation of integrated Computer Vision solutions by continuous high-rate number plate recognition from a moving sensor in all possible environmental conditions. Auto-Detector is actually an independent intelligent sensor that is continuously inspecting what happens around the patrol vehicle and is able to detect automatically the presence of a license plate irrespective of its orientation in the field of view. As such the proposed system represents a revolutionary contribution to patrol crew, working in background and without affecting normal patrol duties. The continuous recognition of plates in the scene is a performance far exceeding any practical possibility by the human eye and the on-line response feature of Auto- Detector (by on-board real-time checking the recognized plate against a search list size of more than millions of plates) provide a great economical value for all security and surveillance applications. Technology innovation is provided also in the Computer Vision process and Optical Character Recognition to achieve a detection performance better than 90% of all license plates in the field of view and a correct recognition rate greater than 99% among all detected license plates. Moreover the system provides a great flexibility in terms of learning tools, to achieve successful performance for all international number plates in all countries. Another important innovation contribution is the miniaturization of the imaging sensor case, that must be installed on-board within a very small space (lighting bar, roof of the car, interior, etc.). To achieve such goals the proposed solution is an effective optimization of existing technology components in the area of digital cameras (LVDS and Camera-Link standard) and infrared illumination sources (using a very innovative LED-on-chip technology that was originally developed and used for automation inspection and Machine Vision). Finally, the selected Auto-Detector onboard processing unit is an effective network processor using a local LAN connection between dedicated processors devoted to each camera and data storage and search, with low-power consumption, and automotive constraints. The decision to adopt fixed camera positions is a compromise to acquire as much data as possible (license plate strings and images) during patrol. This capillary data collection system can rapidly alert patrols (through the onboard navigation system) and the operations centre when blacklisted numbers are detected. The system has been designed for low consumption to fit with existing power supply onboard. It integrates perfectly with existing onboard systems without adding any extra effort to the workload of patrol personnel, who is free to operate as usual in his/her patrol missions. It can be easily defined as an auxiliary officer on-board of the car, with a reading and recognition performance that highly exceeds human capabilities, being able to detect immediately any appearance of selected number plates in the neighbourhood of the patrol car. As such it strongly enhances the capabilities of the patrol crew. 5 Conclusions The paper provides a summary of advanced research and innovation in the field of Security. The automatic detection of abnormal behavior represents one of the main objectives of any new intelligent video surveillance system to drive the human attention only when relevant actions take place. An effective video surveillance
system should be an active tool towards crime prevention rather the most common after the fact investigation. People detection and tracking is one of the most representative computer vision tasks, using multi-camera integration and fusion of information. Another relevant application area is considered. It is the exploitation of License Plate Recognition technology in mobile applications for Homeland Security, to provide security forces the most efficient tools for a wide and thorough vehicle data collection and the possibility to early detect potential criminal behavior. Recent studies confirm that most relevant crime events are closely connected with the use of a car and the identification and tracking of all vehicles along the road and in downtown has proved to be an essential tool for investigation and crime prevention. References 1. Special Issue on Modeling People: Vision based Understanding of a Person s shape, appearance, movement and behavior, Computer Vision and Image Understanding, vol 104, N.2-3, Nov/Dec. 2006. 2. www.cvg.cs.reading.ac.uk, PETS2006, (th Int. Conf. on Performance Evaluation of Tracking and Surveillance, June 18, 2006, New York 3. D.Lowe, The computer vision Industry, http://www.cs.ubc.ca/spider/lowe/vision.html 4. G. Garibotto, M. Corvi, Landmark-based Stereo Vision, Proceedings of the International Conf. On Image Processing and Analysis, ICIAP 05, Cagliari, 6-8 Sep. 2005. 5. G.Garibotto, C.Cibei, 3D Scene Analysis by Real-Time Stereovision, Proc of the IEEE Int. Conf. On Image Processing, ICIP-05, Genova, 12-14 Sep. 2005. 6. T. Horpraset, D. Harwood, L.S. Davis, A Statistical Approach for Real-Time Robust Background Subtractiuon and Shadow Detection, Proc. IEEE Frame Rate Workshop, pp. 1-19, Kerkya, Greece, 1999. 7. G.Garibotto et al., Detection of Abnormal Behavior of people and vehicles in a parking lot; an outcome of the ISCAPS project on Security, submitted to the Int. Conference on Image Analysis and Processing, ICIAP07, Modena, Sep. 2007 8. G.Garibotto, Auto-Detector: Mobile Automatic Number Plate Recognition, Handbook of Pattern Recognition and Computer Vision, ed. C.H.Chen & P.S.P. Wang, chapter 5.6, 2005, pp. 601-618