Intelligent Video Analytics for Urban Management

Transcription

1 Smart-I Gabriele Randelli Founder & CTO Intelligent Video Analytics for Urban Management Gabriele Randelli Founder & CTO 1

2 Gabriele Randelli Founder & CTO Smart- Feel Interactive I Short Company Overview Growth Plan 1st yeartargets Activity & Results Towards the 2nd year Company Overview Enel Lab Review 2 1

3 3 3

4 4 4

5 SmartEye Platform NVIDIA Tegra T30 with Quad ARM CORTEX -A9 Embedded NVIDIA ULP GeForce GPU (OpenGL) 64-bit ARM A57 CPUs 1 TFLOP/s 256-core with NVIDIA Maxwell Architecture 5

6 Adaptive traffic lights go with the flow by measuring vehicle inflow and outflow through each intersection Traffic lights usually are controlled according to an optimal cycle that maximizes the expected flow of traffic U.S. drivers lost $124 billion in 2013 due to the cost of fuel and the value of time wasted in traffic!!! (CEBR) 6 6

7 More than 200 traffic lights currently optimized and controlled with SmartEye! (1200 in May 2017) 7

8 Traffic Lights Control - Overall Architecture & T30 Algorithms Raw Images Intelligent Video Analytics Traffic Statistics Routing Planner Traffic Light Phase Profile Traffic Light Control Profile Application ROI Crop Median Blur Equalization Background Subtraction Legend CPU Only Blob Association & Matching Extended Kalman Filter Lucas Kanade Tracking CPU with NEON Vehicle Classification Homography Video Streaming On-board 8 GPU (OpenGL)

9 Major Issues Computational Power vs Analytics Power 5 fps implies problems for blob tracking and Kalman filter Optimizing with NEON takes a long time! (Inline Assembly) Maximum number of concurrent moving objects set to 32 Weak classifiers only 9

10 Traffic Lights Control - GPU-based Architecture Raw Images Intelligent Video Analytics Traffic Statistics Routing Planner Traffic Light Phase Profile Traffic Light Control Profile Application Legend ROI Crop Median Blur Equalization Background Subtraction HCM CPU Only Blob Association & Matching Extended Kalman Filter Lucas Kanade Tracking Traffic Light Synchronization CPU with NEON GPU Vehicle Classification Homography Video Streaming On-board Webster Cycle Length 10 On-board

11 SmartEye + Jetson TX1 - Computer Vision Optimizations General Considerations: We rely on VisionWorks Computer Vision pipeline becomes graph based execution Manifold VisionWorks vision functions adopted (LUT, arithmetic operations, color convert,...) ROI Crop vxaccessimagepatch Median Blur Equalization Lucas Kanade Tracking Homography vxequalizehistnode vxmedian3x3node vxopticalflowpyrlknode nvxfasttracknode vxwarpperspectivenode - 4 new algs on GPU - 4 implementations already available - About 30% code re-engineered - 15 fps! - Routing planning on-board! Extended Kalman Filter Ad-hoc implementation 11

12 12 12

13 13

14 Real-time In the loop control No Cloud 14

15 15

16 Smart Lighting - Overall Architecture & T30 Algorithms Raw Images Intelligent Video Analytics Traffic Statistics LED Dimming Prediction Plant Profile Prediction LED Plant Control Profile Application Same as Traffic Light Nonlinear Autoregressive Exogenous Model (NARX) Traffic Volume Time Series Traffic Volume Prediction Lighting Category Prediction Plant Profile Prediction On Cloud (Training & Prediction) 16

17 Major Issues Machine learning takes time No closed-loop control (we need cloud) Adaptive, but no lighting on demand 17

18 Smart Lighting - GPU-based Architecture Raw Images Intelligent Video Analytics Traffic Statistics LED Dimming Prediction Plant Profile Prediction LED Plant Control Profile Application Same as Traffic Light Nonlinear Autoregressive Exogenous Model (NARX) On board prediction enables lighting on demand profile Traffic Volume Time Series Traffic Volume Prediction Lighting Category Prediction Plant Profile Prediction On Board (Prediction) Model Training On Cloud (Training) 18

19 SmartEye + Jetson TX1 - Time Series Prediction Optimization for Closed-loop Lamp Control General Considerations: cudnn does not support NARX models :( Ad-hoc implementation with CUDA Toolkit Only network execution, training still on cloud Design Considerations: neurons on a same network level are completely isolated (parallelization) no writing sync access during NARX prediction every network level computation is a CUDA function - 37% faster - Closed-loop control - Lighting on demand 19

20 Conclusions In Smart Cities real-time closed loop control is a significant business advantage Cloud is fine, but most of processing has to be locally deployed Unloading algs on GPUs takes less time than using NEON/SSE or relying on FPGAs VisionWorks on Jetson TX1 already supports many algs and easily interacts with OpenCV (very low dev effort) cudnn still needs to support more advanced NN models More than 2x speed-up on two relevant application fields 20

21 Gabriele Randelli Founder & CTO Smart-I We live in the big data world it is now time to reason on top of these data to integrate smart control of the environment where we live Gabriele Randelli Founder & CTO 21