Smart Ultra-Low Power Visual Sensing

Transcription

1 Smart Ultra-Low Power Visual Sensing Manuele Rusci*, Francesco Conti * manuele.rusci@unibo.it f.conti@unibo.it Energy-Efficient Embedded Systems Laboratory Dipartimento di Ingegneria dell Energia Elettrica e dell Informazione Guglielmo Marconi DEI Università di Bologna

2 Internet of Smart Things "a worldwide network of interconnected and smart objects uniquely addressable, based on standard communication protocols" How about the design process of IoT end-nodes for the emerging network infrastructure? 2

3 Node Lifetime Energy Issues for IoT devices Power-consumption is a big issue for smart always-on systems and IoT devices. Need to reduce the consumption to the μw range to extend the battery lifetime. Button Cell Smartwatch Battery μw range 10 years 1 year 1 week 1 hour 1.E-09 1.E-07 1.E-05 1.E-03 1.E-01 Average Power Consumption (W) [Alioto, Massimo. "IoT: Bird s Eye View, Megatrends and Perspectives." Enabling the Internet of Things. Springer International Publishing, ] 3

4 Smart Camera Sensors Several smart cameras for always-on monitoring applications feature a power consumption of hundreds of mw 2MB SRAM Status LEDs (additional 2MB on bottom) OV5642Sensor Module FTDI USB to Serial SWD Connector Where does this consumption comes from? Cortex M4 CPU STM32F417(168MHz) LiPo Battery Connector 2x15pin Extension Headers T r u s t E Y E. M 4 C o r e C o m p o n e n t s High processing Ext. SRAM 0 SWD/SWV (high workload) Ext. SRAM 1 Debug + Trace Image Cortex M 4 Sensor Serial to USB External Task memory STM32F417because Runtime of the Micro high USB Connector Power amount JPEG/YUV422 of data parsing Management 9.2ms - programming Trusted Platform - debugging Multimodal Mean (4 41.8ms Module - power supply - Lipo Charging Remarkable sensor power consumption Cells) Bounding Box Calculation SHA1 Computation AES256 Encryption T r u s t E Y E. M 4 Total C o n n e c t i v i t y 4.8ms RaspberryPI Transmission Single -Cell LiPo Connector 2x 15pin power extension headers to (I2C, be SPI, ) reduced thanks Battery (SPI) to the on-board processing 1.2ms 1.7ms 58.7ms Component Input 3.3V STM32F417 CPU 98mA SRAM 23mA OV5642 Sensor 156mA WiFi (TX) 140mA TPM 21mA Total 438mA Necessity of optimized sensing and processing! 4

5 Event-Based Visual Sensing Silicon Retina to mimic the human eye! Every pixel generate a significant events corresponding to spatial and temporal changes Focal Plane Processing Per-pixel circitut for filtering and binarization Event-based sensing output frame data bandwidth depends on the external context-activity PO V res to pixel PN to pixel PE comp1 V Q PN PE PO QO QN QE Contrast Block V EDGE V th comp2 Frame-based {x 0,y 0 } {x 1,y 1 } {x 2,y 2 } {x 3,y 3 } {x n-1,y n-1 } Eventbased 5

6 Event-Driven Visual Sensing Available Thesis Works : Development of new HW functionalities for the event-based camera interface (suggested if HW background) Measuring and optimizing smart camera performance 6

7 Deep Neural Networks Convolutional Neural Networks are state-of-the art for visual recognition, detection and classification tasks 7

8 CNNs on Always-on devices: Issues How to exploit CNNs on always-on devices with a power envelope of few mws or sub-mw? Multi-Dimensional Image Data Inference Engine Output Class Label bike Issues: Large memory footprint to store weights (the program ) and intermediate results (up to hundreds of MBs), greater than memory footprint available on ultra-low power engines (100 s kbs) Requires billions of operations in a few milliwatts 8

9 Deep Learning on MCUs? Autonomous-nano UAV Smart Surveillance Camera 9

10 Available Thesis Works Ultra-low-power deep learning and AI Development CNN-based applications on commercial ULP MCU by means of optimized libs (CMSIS-NN), possibly using commercial sensors (vision, audio, biomedical) Optimization of computing kernels for CNNs, focusing on compression tecniques such as quantization Optimizazion of CNN-based applications on parallel ultra low power computing devices (e.g PULP), featuring dedicated accelerators Training of CNN networks (only if ML background and strongly motivated...) What you will learn: Writing Optimized C-code for Microcontroller Building Sensory-Based systems optimized for low-power Learning Advanced Machine learning tecniques Advanced processing architectures (MCUs, FPGA, ASICs) Suggested background (at least one): C programming Linear algebra basics and scripting (MATLAB, python) Digital Electronics Design 10