International Journal of Computer Systems (ISSN: 2394-1065), Volume 03 Issue 06, June, 2016 Available at http://www.ijcsonline.com/ Aniket A. Patil, Prof. S. A. Shinde Department of Computer Engineering, Savitribai Phule Pune University, Pune, India Abstract The need to detect behavior of crowd is to address social issues and have safer and more secure society. The surveillance videos can be used for detecting abnormal behavior of crowd. People naturally run away from the place where unexpected event takes place. Based on this observation we propose a system which detects behavior of crowd automatically when unusual events happen. The proposed system performs appropriate detection of unexpected events. The proposed system is based on optical flow and detects the pattern in crowd motion. The system takes sequence of images i.e. video as input. Preprocessing of video is done. Then frames are extracted from video. Every frame is processed to remove the background and to extract foreground patches. Then features are extracted on the basis of position, magnitude and direction. These extracted features are given to proposed system for further processing. Optical flow is then calculated to find the pattern between frames. The system then detects behavior of crowd as normal or abnormal. The proposed system gives more accurate results then the existing system and identifies crowd behavior through real time videos. Keywords: Crowd behavior, crowd motion, video survelliance, escape and non-escape activity, crowded scence analysis. I. INTRODUCTION In recent years number of surveillance cameras is installed in public places and on streets to monitor people. Places such as malls, airport, railway stations and crowded streets have more number of surveillance cameras. These cameras are installed for public safety and secure society. The crowd behavior analysis faces more challenges like complex interactions and various factors and expressions. The large crowd is difficult to handle which also increases computational complexity. Increase in population has made crowded scenes frequent in day to day life. Therefore, there is variety of human activity. Massive challenges such as public safety, management, or security has attracted researchers in the computer vision community to develop automatic system. In case of crowded scenes, the challenge is that it cannot be handled well due to large number of individual participation. Because of large number of individuals tracking and detection of crowd behavior fails. Due to lack of communication in large crowd if disaster takes place then number of deaths are more. In a crowd individual can only communicate with their neighboring person and have very limited view of whole crowd. This leads to false perception of movement when people are falling and people who are behind perceive a movement and move forward thereby increasing the pressure even more. Crowd detection is mainly required in dense urban areas where pedestrians often move in groups. These scenarios are extremely difficult to analyze because no individual pedestrian can be properly segmented out for detection, the path people take can be quite chaotic and the background is not homogenous and hence it is difficult to distinguish humans from man-made objects. The study of crowd behavior analysis is based on public safety and transportation. Existing models have been developed for detecting individual and group behaviors in crowded scenes. Crowded scene analysis could lead to lot of major applications. In case of video surveillance, there are many places of security interest such as railway station and shopping mall. Crowd scenes often contain many pedestrians that move in opposite directions, or many pedestrians that move at a distance. Such scenes pose a serious challenge to pedestrian detection algorithm as the pedestrians are not clearly visible in the image. However, these scenes are rich in unique motion patterns. The system can be used to develop crowd management strategies during festivals and in sports stadium. When abnormal events affecting public safety happen, such as fires, explosions, road accidents and so on, people naturally escape from such places since it is natural act of avoiding something dangerous, unpleasant or desirable. This paper presents an approach for crowd behavior detection by modeling optical flow in both normal and abnormal situations. We consider that when unexpected event occurs people naturally escape from such places. So the focus of this paper is on crowd behavior detection during such situations. For video containing crowd activities, the frames are represented using corresponding optical flow field. To detect behavior of crowd each optical flow field is represented as collection of patches. These patches are foreground patches extracted from frame and are considered as moving objects. To get the motion patterns of 450 International Journal of Computer Systems, ISSN-(2394-1065), Vol. 03, Issue 06, June, 2016
crowd we introduce the concept of potential destinations. When people move away from those destinations we use set of motion patterns to detect crowd escape behavior. Thus the main objectives of the system are: 1) To extract the foreground patches so that optical flow for crowd can be calculated using dataset when unexpected event occur. 2) The model is proposed to combine different attributes to capture motion pattern for crowd behavior detection. 3) The feature vector is given to the system which will classify the crowd behavior into normal and abnormal. The remainder of the paper is organized as follows: Section 2 describes the previous work related to crowd motion pattern extraction and anomaly detection. In section 3 the implementation details of proposed framework are provided. Experimental evaluation and details of dataset is described in section 4. Finally in section 5 our paper concludes. II. RELATED WORK In recent years, several methods have been developed to deal with the problem of crowd analysis, especially in public as well as private sectors for public safety. R. Mehran, A. Oyama, and M. Shah, [2] author proposed a new method to detect and localize abnormal behaviors in crowd scenes on the basis of social force model. The scheme proposed for abnormal behavior detection in the crowd videos consists of four fold approach. The particles in grid are placed over the image and then they compute the social force between moving particles to extract interaction forces. After this, change of interaction forces in time determines the ongoing behavior of the crowd. Interaction forces of individuals are not need to perform segmentation and used as base for proposed method and method is capable to capture the dynamic of crowd behavior. Finally, classify frames as normal and abnormal by using a bag of words approach. This method captures the dynamic behavior of crowd on the basis of interaction forces of individuals without the objects tracked individually. Abnormal behaviors in the crowd is effectively detected and localized by this method. Barbara Krausz et. al [3], has developed a system that detects automatically the motion behavior of crowd that is based on the optical flow fields to determine motion patterns and motion directions in the crowded area. The polynomial technique is used to calculate the optical flow fields. The dense optical flow fields computation using the method proposed by Farenback, that is quadratic polynomials is used to estimate translations of a local neighborhood and motion vectors are determined from polynomial expansion coefficients. E. Andrade, S. Blunsden, and R. Fisher [4], author presented the detection technique for detection of abnormal events in crowd scheme. Crowd behavior is hard to know and to get normal crowd behavior author used unsupervised feature extraction. To discover the optimal number of models to represent normal motion patterns the spectral clustering is applied on feature extraction. Proposed approach is effective for detecting emergency in crowd scenarios. Anomaly detection in crowded areas is shown with new approach by authors[5]. The study is mainly aimed at abnormal movements of crowd. Anomaly detection is done by using the new approach called optical flow of textures. The normal crowd behavior is detected on basis of mixtures of dynamic textures. 3D images of textures are considered instead of 2D images. A vector is extracted from 3D images to get information about motion and spatial and texture information. Optical flow algorithm is used for anomaly detection in crowd. The algorithm has enough accuracy on datasets taken. Images in low resolution are processed. The system is trained on large datasets for anomaly detection. This approach accurately detects anomalous objects like vehicles. The author [6] presented developed social force model. It has been shown that pedestrian motion can be described as if they would be subject to social forces. These forces are a measure for the internal motivations of the individuals to perform certain actions. In this system computer simulation of pedestrian groups is demonstrated, 1.the development of lanes consisting of pedestrians who walk into the same direction, 2.oscillatory changes of the walking direction at narrow passages. These spatiotemporal patterns arise due to the nonlinear interactions of pedestrians. Author combines the spatio-temporal contextual information with a novel deformable template matching procedure. Spatio-temporal contextual information are evaluated on several complex datasets and proposed method is better in comparison to other algorithms on the basis of the evaluation results. Proposed algorithm is not flexible enough for moveable visual surveillance system. Author proposed [7] a new technique sparse reconstruction cost to detect abnormal event detection on normal bases. A abnormal behavior is determined by its sparse reconstruction cost, through a weighted linear reconstruction of the over-complete normal basis set. The proposed method easily handles both local abnormal events and global abnormal events. It also support detection of online events. The method is robust. To determine behavior of the crowd author has introduced certain methods [8]. They are namely objectbased and holistic approach. Object-based method does the crowd behavior detection through the segmentation or by detecting the individuals to analyze the group behaviors. This approach considers a crowd as a collection of individuals. In high density scenario it is difficult to track the individual activity in the crowd. Therefore the second approach that is holistic approach is well suited for the proposed system. This approach try to obtain the 451 International Journal of Computer Systems, ISSN-(2394-1065), Vol. 03, Issue 06, June, 2016
global information like main crowd flows and they discard the local information such as a person moving against the flow. Antoni B. Chan, and Nuno Vasconcelos[9] proposed a solution for the problem of pedestrians traveling in different directions. In this method first crowd segmentation is done. The goal is to count people those moving in different directions or in different speeds. The collection of spatiotemporal patches is used for representation of video. Scanning is done sequentially of video locations. Features are extracted to capture properties of segment like shape and size. Features are also extracted from the segment perimeter. Automated visual surveillance of crowds is a rapidly growing area of research. In this paper [10] we focus on motion representation for the purpose of abnormality detection in crowded scenes. We propose a novel visual representation called textures of optical flow. The proposed representation measures the uniformity of a flow field in order to detect anomalous objects such as bicycles, vehicles and skateboarders; and can be combined with spatial information to detect other forms of abnormality. We demonstrate that the proposed approach outperforms state-of-the-art anomaly detection algorithms on a large, publicly-available dataset. III. DETAILED DESIGN OF SYSTEM Here in Fig. 1. input is taken as video of any format. As video contains more number of frames, these frames are extracted from the input video. After extracting each frame from input video convert each frame into eight by eight block. Each block contains 64 pixels. Frames are divided for easy and efficient processing of video sequences. Next step is to remove background and extract foreground features. Then proposed system extracts features from each frame and optical flow is calculated by comparing previous frame and current frame. The system classifies the data and thus detects normal and abnormal behavior of crowd. A. Background Subtraction This model has very important role in video analysis system. Background subtraction method is used to fragment the object in a frame. When static cameras are used, a popular approach is background subtraction, which consists of obtaining a mathematical representation of the static background and comparing it with each new frame from the video sequence. A moving object can be detected easily by identifying parts of the image that do not match with the model. This process is known as background subtraction. It is groundwork for various post-processing modules such as object tracking, recognition, and counting. B. Feature Extraction Fig.1. System Architecture Feature construction is one of the key steps in the data analysis process, largely conditioning the success of any subsequent statistics or machine learning endeavor. In particular, one should beware of not losing information at the feature construction stage. It may be a good idea to add the raw features to the preprocessed data or at least to compare the performances obtained with either representation. As the frames in video are more and are suspected to be redundant then it can be transformed into reduced set of features. This process is also called feature selection. The selected features contain the required and relevant information so that the desired output is obtained by using reduced set of features instead of initial complete data. C. Proposed System The proposed system compares the magnitude of each vector, speed factor, and the distance between them. Distance is calculated by Euclidean distance formula. According to these parameters system classifies the vector as normal or abnormal. Depending on the distance to the other vector the chosen variation can vary from ±Π/16 when distance is large to ±Π/2 when distance is short. Optical flow works on several assumptions. The pixel intensities of an object do not change between consecutive frames. Neighboring pixels have similar motion. Optical flow method is used to detect moving human being or objects based on their velocities from surveillance cameras. The motion of pixels in image is produced due to motion of objects represented in pixels in image. Optical flow also depends on relative distance between the object and the camera. The crowd velocity is calculated by assuming the magnitude of optical flow vectors. Magnitude value will be zero if there is no movement otherwise it will be nonzero. D. Algorithm Input: video files of different formats. Output: detection of crowd behavior. 1: Input frames of a video {V1,V2 Vn} 2: for v=1 to V do 452 International Journal of Computer Systems, ISSN-(2394-1065), Vol. 03, Issue 06, June, 2016
3: Calculate optical flow field. 4: Extract foreground patches. 5: Assume k as vector. 6: Calculate magnitude of vector k 7: if (bins==4) 8: Take four quadrants to get the value of k 9: elseif(bins==8) 10: Divide the quadrant into eight. 11: else(bins==16) 12: Divide the quadrant into sixteen. 13: Classify optical flow field 13: end for 14: Train system with k vector features 16: Output classification result {O1, O2}. The algorithm shows step by step process of system. The video files of.mp4,.mov,.flv, and.avi format are taken as input. The frames are extracted. Each frame is divided into eight by eight block and from those frames foreground patches are detected. The magnitude of vector from each frame is obtained. The value of vector from four quadrants is calculated. The quadrants are again divided into eight parts and further again it is divided into sixteen parts. This improves the accuracy of system. Then the optical flow is calculated by considering flow pattern between previous and current frame. The system is trained by vector features and the behavior of crowd is detected. E. Mathematical Model Let S be the system S = {U, I, O, Fea, F, P} Where, U = set of users Hc = H F H F 1 Hc = Histogram change in consecutive frames IV. EXPERIMENTAL RESULT A. Dataset We have focused on real data for the evaluation of proposed model. All videos are mainly collected from the public UMN dataset which are widely used for performance evaluation. B. Outcome We are taking videos as input to the system and performing the evaluation. Fig. 2. shows that crowd movement is abnormal. System detects the abnormal movement of crowd. Fig.3. shows the accuracy comparison of existing system and proposed system. Fig.4 shows the accuracy graph of proposed system when quadrant is divided into sixteen parts. Highest accuracy is achieved at this point. Fig. 5. shows the accuracy graph of proposed system when quadrant is divided into eight parts. Fig.6. shows accuracy graph of proposed system when only four quadrant considered and not divided into further parts. Accuracy is less at this point. Fig.7. shows the comparison of time required by existing system and proposed system. Less time is taken by our system. I = set of video files I = {V 1, V 2, V 3, V n } O = set of outputs O = {O 1, O 2 } Framing Fn = V v / t p (1) Fn = number of frames in video Vγ = video length in second T P = time period between two consecutive frames in (second) t p = F w *F h / w*w (2) t P = total patches in frames F w, F h = width and height of frame respectively W = size of each patch (width and height) R = {r i r i ϵ A = (r 1 r tp ), mr i > ϵ}...(3) R = set of foreground patches A = set of all patches in frames mr i = magnitude of optical vector in for ground = threshold magnitude for patch. Fig.2. H F = m1, m2,..., mn (4) H F = Histogram of collective motion of crowd m1= magnitude of motion vector with angle Θ 1. Fig.3. 453 International Journal of Computer Systems, ISSN-(2394-1065), Vol. 03, Issue 06, June, 2016
V. CONCLUSION Automatically crowd behavior is identified from surveillance videos. The existing method does not give the accurate results for the identifying the crowd behavior in the videos. Therefore proposed system by using the classification gives the more accurate result. The proposed method is able to update the density of optical flow and calculate the magnitude of feature vector when unexpected events take place. The proposed method detects crowd behavior which involves changes in position, speed and direction. In future we plan to study the different characteristics of crowd in high density situations. Fig.5. Fig.6. Fig.4. REFERENCES [1] Si Wu, Hau-San Wong, and Zhiwen Yu, "A Bayesian Model for Crowd Escape Behavior Detection ", IEEE Transactions on circuits and systems for video technology, VOL. 24, NO. 1, JAN 2014. [2] R. Mehran, A. Oyama, and M. Shah, Abnormal crowd behavior detection using social force model, in Proc. IEEE Conf. Comput. Vision Pattern Recognit., 2009, pp. 935 942.. [3] Barbara Krausz, Christian Bauckhage, Fraunhofer IAIS, Automatic Detection of Dangerous Motion Behavior in Human Crowds, in 8th IEEE International Conference on Advanced Video and Signal-Based Surveillance,2011, pp. 224-229. [4] E. Andrade, S. Blunsden, and R. Fisher, Modelling crowd scenes for event detection, in Proc. Int. Conf. Pattern Recognit., 2006, pp. 175 178.. [5] David Ryan, Simon Denman, Clinton Fookes, Sridha Sridharan, Textures of Optical Flow for Real-Time Anomaly Detection in Crowds, 8th IEEE International Conference on Advanced Video and Signal-Based Surveillance, 2011 [6] Drik Helbing and Peter Molnar, Social Force Model for Pedestrian Dynamics, in Physical Review E vol. 51 no. 5,May 1995, pp. 4282. [7] Y. Cong, J. Yuan, and J. Liu, Sparse reconstruction cost for abnormal event detection, in Proc. IEEE Conf. Comput. Vision Pattern Recognit., 2011, pp. 3449 3456. [8] J. Jacques Junior, S. Musse, and C. Jung, Crowd analysis using computer vision techniques, IEEE Signal Process. Mag., vol. 27, no. 5, pp. 66 77, Sep. 2010.. [9] Antoni B. Chan,and Nuno Vasconcelos, Counting People With Low-Level Features and Bayesian Regression, IEEE Transactions on image processing, VOL. 21, NO. 4, April 2012. [10] D. Ryan, S. Denman, C. Fookes, and S. Sridharan, Textures of optical flow for real-time anomaly detection in crowds, in Proc. IEEE Int. Conf. Adv. Video Signal-Based Surveill., 2011, pp. 230 235. [11] Weina Ge, Robert T. Collins, Senior Member, IEEE, and R. Barry Ruback, Vision-Based Analysis of Small Groups in Pedestrian Crowds IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 34, NO. 5, MAY 2012 [12] B. Zhou, X. Wang, and X. Tang, Understanding collective crowd behaviors: Learning a mixture model of dynamic pedestrianagents, in Proc. IEEE Conf. Comput. Vision Pattern Recognit., 2012, pp. 2871 2878. [13] L. Lin, H. Gong, L. Li, and L. Wang, Semantic event representation and recognition using syntactic attribute graph grammar, Pattern Recognit. Lett., vol. 30, no. 2, pp. 180 186, 2009. [14] A. Chan and N. Vasconcelos, Modeling, clustering, and segmenting video with mixtures of dynamic textures, IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 5, pp. 909 926, May 2008. Fig.7. 454 International Journal of Computer Systems, ISSN-(2394-1065), Vol. 03, Issue 06, June, 2016