Methodology and Implementation of Early Vision Tasks for the Viola Jones Processing Framework

Size: px

Start display at page:

Download "Methodology and Implementation of Early Vision Tasks for the Viola Jones Processing Framework"

Beverley Norton
5 years ago
Views:

Rodríguez-Vázquez University of Seville - SPAIN Institute of

1 Methodology and Implementation of Early Vision Tasks for the Viola Jones Processing Framework Jorge Fernández-Berni, Ricardo Carmona-Galán, Ángel Rodríguez-Vázquez University of Seville - SPAIN Institute of Microelectronics of Seville - SPAIN berni@imse-cnm.csic.es WASC 2013 Seville (Spain) ANAFOCUS 2007

Resolution Focal-Plane Array for Low-Power

2 FLIP-Q: Focal-Plane Low-power Image processor J. Fernández Berni, R. Carmona Galán and L. Carranza González, FLIP-Q: A QCIF Resolution Focal-Plane Array for Low-Power Image Processing, in IEEE J. Solid-State Circuits, vol. 46, no. 3, pp , March 2011

containing 2135 features in total for the Viola-Jones baseline face detection algorithm

3 Viola-Jones Processing Flow Haar-like features: encode differences in average intensities between rectangular regions Large number of Haar-like features to compute: 22 classifiers containing 2135 features in total for the Viola-Jones baseline face detection algorithm provided by the OpenCV library Sequence by Adam Harvey

Integral and Squared Integral Images Intermediate image representations to speed up the execution of the processing flow The integral image enables the computation of the sum at any rectangular

4 Integral and Squared Integral Images Intermediate image representations to speed up the execution of the processing flow The integral image enables the computation of the sum at any rectangular region of the input image by accessing only four pixels adequately chosen The squared integral image enables the variance normalization of the Haarlike features in order to minimize the effect of different lighting and contrast conditions

implementation of reconfigurable focal-plane circuitry delivering integral images at different scales

5 Massively parallel focal-plane realization The computation of and can clearly benefit from the concurrent operation and distributed memory provided by focal-plane processing architectures Our objective: physical implementation of reconfigurable focal-plane circuitry delivering integral images at different scales original image and successive versions after pixel binning Original image and successive downsampled versions equivalent to

Massively parallel focal-plane realization General floorplan of the QVGA (320 240-px) prototype smart imager 4-connected mixed-signal sensingprocessing elementary cells providing

6 Massively parallel focal-plane realization General floorplan of the QVGA ( px) prototype smart imager 4-connected mixed-signal sensingprocessing elementary cells providing computational and memory resources Peripheral circuitry endowing the array with enormous flexibility for reconfiguration Four full-custom 8-bit SAR A/D converters, each providing a throughput of 4MSa/s

7 Massively parallel focal-plane realization Pixel copy Sensing-processing elementary cell Squarer Block-wise square addition Standard 0.18µm UMC CMOS process 19.59µm Block-wise HDR 17µm Block-wise addition Output buffers mm 2 mixed-signal core

8 Block-wise HDR: emulation with Wi-FLIP 3.9ms 24.2ms 992ms 11ms 21ms 45.1ms The algorithm only looks for specific local patterns, dismissing the artifacts

9 Integral images: focal-plane reconfiguration Reconfiguration patterns Peripheral circuitry per column

feasibility of our approach Mismatch between sensing

10 Functional feasibility: sources of deviation Four primary sources of deviation have been analyzed to confirm the feasibility of our approach Mismatch between sensing capacitances Error in pixel copy Error in charge redistribution Error in squaring

11 Functional feasibility: sources of deviation Pixel copy and pixel squaring constitute the major sources of deviation Pixel copy simulation results 30 post-layout Monte-Carlo simulations Deviation model:

12 Functional feasibility: sources of deviation Pixel copy and pixel squaring constitute the major sources of deviation Pixel squaring simulation results 30 post-layout Monte-Carlo simulations Deviation model:

13 Functional feasibility: models in OpenCV The deviation models are introduced into the Viola-Jones baseline face detection algorithm provided by the OpenCV library

Functional feasibility: models in OpenCV Finally, the algorithm

realization was tested by making use of the Caltech Frontal Face

people under different lighting, expressions and background

14 Functional feasibility: models in OpenCV Finally, the algorithm including the sources of deviation derived from physical realization was tested by making use of the Caltech Frontal Face Dataset: 450 face images px resolution JPEG format 27 people under different lighting, expressions and background Examples: Image #0008 Image #0216 Image #0422

15 Functional feasibility: results The input images are previously scaled to QVGA to keep the test realistic The successive scales are obtained by subsampling both dimensions by two, just as the chip will provide through pixel binning Ideal case: no deviation considered 373 faces successfully detected out of 450: 83% hit rate 1 false positive On-chip case: all sources of deviation considered 356 faces successfully detected out of 450: 79% hit rate 2 false positives The proposed physical realization of early visions tasks based on focal-plane mixed-signal processing will hardly impact the performance of the Viola-Jones algorithm at high-level

Advantages of Focal-plane Approach Ultra low-power consumption: < 100mW expected Significant reduction of memory accesses Flexibility for reconfiguration: skip integral images by exploiting

16 Advantages of Focal-plane Approach Ultra low-power consumption: < 100mW expected Significant reduction of memory accesses Flexibility for reconfiguration: skip integral images by exploiting concurrent rectangular area sum Implementation of additional low-level processing tasks: Block-wise HDR imaging Gaussian pyramid generation Image pre-warping for subsequent reduced kernel filtering Image pre-processing for subsequent saliency estimation Some throughput figures expected (assuming an integration time of 10ms): 5 scales of integral images at 10fps 5 scales of Gaussian pyramid at 25 fps 35fps when working as an imager

17 the end THANK YOU VERY MUCH FOR YOUR ATTENTION

Smart imaging for power-efficient extraction of Viola-Jones local descriptors

Best Paper Award Smart imaging for power-efficient extraction of Viola-Jones local descriptors J. Fernández-Berni a, R. Carmona-Galán a, R. del Río a, Juan A. Leñero-Bardallo a, M. Suárez-Cambre b and