IDENTIFYING BEHAVIORS IN CROWDED SCENES THROUGH STABILITY ANALYSIS FOR DYNAMICAL SYSTEMS

Transcription

1 IDENTIFYING BEHAVIORS IN CROWDED SCENES THROUGH STABILITY ANALYSIS FOR DYNAMICAL SYSTEMS Berkan Solmaz, Dr. Brian Moore and Dr. Mubarak Shah 1/21

2 Outline Behaviors to be Identified Theoretical Concepts Lagrangian Particle Dynamics, Numerical Integration Linearization of Dynamical Systems, Jacobian Matrix Proposed Framework Experimental Results 2

3 Behaviors in Crowded Scenes Bottlenecks Departure Lanes Arch/Rings Blockings 3

4 Identifying Behaviors in Crowded Scenes Challenges : Various real-world conditions High-densities of moving objects Difficult to analyze behaviors of individuals 4

5 Conventional Approaches for Behavior Analysis Object Detection Tracking Training Behavior Recognition Fail for high density of objects Computationally expensive Require training, hence the manual isolation of behaviors 5

6 Theory Behind Our Method (Lagrangian Particle Dynamics) Sequence Optical Flow F(y(t)) using numerical integration, Euler s Method y(t+1) = y(t) + F(y(t)) Particle Trajectories Φ x (x-flowmap) Φ y (y-flowmap) positions of particles 6

7 Theory Behind Our Method (Linearization of a Dynamical System) particle velocities fixed points (1) (2) By Taylor s theorem where f x and f y are partial derivatives (3) (4) new coordinates (5) (6) (7) (8) Jacobian matrix 7

8 Eigenvalues of Jacobian Matrix Jacobian matrix Eigenvalues λ 1, λ 2 Sink Source Line of Saddle Center fixed points λ 1 <0 λ 2 <0 λ 1 >0 λ 2 >0 λ 1 =0 or λ 2 =0 λ 1 <0<λ 2 λ 1 and λ 2 complex conj. Bottleneck Departure Lane Blocking Arch/Ring (Fountainhead) 8

9 Proposed Framework Task 2 Computing Eigenvalue Map Input Video Computing Optical Flow Particle Advection Accumulation of Particles u(x,y,t) v(x,y,t) u(x,y,t) v(x,y,t) Flow maps φ x φ y Computing Jacobian in ROIs Defining ROIs Clustering Particle Trajectory Angles J F (x,y) Candidate Points, Paths, Precincts Check Eigenvalue Conditions Ratios Detecting Behaviors Behavior Accumulation points Task 1 Finding ROIs 9

10 Task 1 Finding ROIs Sequence Optical Flow Particle Trajectories Density Map Sequence Optical Flow Particle Trajectories Density Map 10

11 Task 1 Finding ROIs Density Map Accumulation Points Particle Trajectories Angles of Trajectories Density Map Accumulation Points Particle Trajectories Particle Trajectories Various directions Candidate point for Bottleneck One direction Candidate paths for Lanes or Arches/Rings Angles of Trajectories Angles of Trajectories 11

12 Task 2 Computing Eigenvalue Map Average Optical Flow Eigenvalue Map Around Candidate Point λ 1,2 real λ 1 <0 λ 2 <0 λ 1 >0 λ 2 >0 λ 1 =0 or λ 2 =0 λ 1, λ 2 λ 1 <0<λ 2 λ 1,2 =α±jβ, α<0 λ 1,2 complex λ 1,2 =α±jβ, α>0 λ 1,2 =±jβ No motion Average Optical Flow Eigenvalue Map Around Candidate Path 12

13 Ratios & Identified Behaviors Eigenvalue Map Around Candidate Point Bottleneck Identified Behavior Bottleneck Lane Arch/Ring Ratio Condition Red / Total > Thr Blue / Total > Thr (White+Cyan+Magenta) / Total > Thr Eigenvalue Map Around Candidate Path Lane and Arch 13

14 Detecting Departure Particle Trajectories Density Map Accumulation Points Particle Trajectories Angles of Trajectories Particle Trajectories Angles of Trajectories Various directions Candidate points for Departure 14

15 Detecting Departure Average Optical Flow Eigenvalue Map Around Candidate Point λ 1 <0 λ 2 <0 λ 1 >0 λ 2 >0 Departure λ 1 =0 or λ 2 =0 λ 1 <0<λ 2 λ 1,2 =α±jβ, α<0 λ 1,2 =α±jβ, α>0 λ 1,2 =±jβ No motion Identified Behavior Departure Ratio Condition Yellow / Total > Thr 15

16 Experiments on Real Videos YouTube, Getty-Images, BBC Motion Gallery, Though Equity 60 videos including Crowd and Traffic Scenes Ground Truth -Positions of bottlenecks/departures/blockings -Paths of lanes/arches/rings Evaluation Criteria -Distance test for bottlenecks/departures/blockings -Path overlap ratio test for lanes/arches/ring 16

17 Identified Behaviors on Real Videos 17

18 Quantitative Results (I) Detected Missed False Detections 0 18

19 Advantages Integrates of low-level local motion features and high-level information of the scene Performs well in various types of crowded scenes No training, or detection and tracking No isolation of activities Fast and robust to occlusions Multiple behavior identification 19

20 Thank You