Embedded many core sensor-processor system

Transcription

1 Efficiency of our computational infrastructure Embedded systems (P-ITEEA_0033) Embedded many core sensor-processor system Lecture 4 2, March, nm technology 1.2 billion transistors 3.4 GHz clock frequency Up to 1500 pins High-end processor: 6 x 3.5 GHz 100 GFlops (32 bit) ~ 1000GOps(8bit) 90W Intel 8080 processor MHz 0.5MOps It powered a simple computer at that time ~ 6000 transistors 200,000 pieces of 8080 can be placed to one single chip!!! 1700 times clock frequency today!!! The technology offers 340,000,000 times highest capacity However, a Pentium has only ~2,000,000 times larger capacity!!! 170x inefficiency = 6 years scaling down!!! Why do we need many-core? The single core microprocessors reached the power ceiling 11 years ago The performance/power figures are better for many-cores The many-core processors does not need new technology Vertical integration requires some technology changes Next generation computer technology (nanotechnology?) is still in research phase W 100W 2005 year Why does many-core consumes less power? Digital circuits power is proportional with the frequency on constant DC level DC level drops with frequency drop Complexity of circuits drops with frequency drop Simpler, smaller, shared auxiliary circuits Scheduler Fetcher Instruction reordering # proc. f P Pc High-speed IO 1 4 GHz 100 W 20 GOps SIMD architectures 4 2 GHz 100 W 40 GOps One instruction control unit One IO unit Local IOs during operation 16 1 GHz 100 W 80 GOps Biomotivated early-vision architectures Embedded sensor and processor arrays; Local interconnections; Single Instruction-Multiple Data (SIMD); Local memories; Adaptivity through masking; Different data representations (analog, logic); Sensor Processor Memory Implementations Analog CNN architecture (fine-grain) ACE16k Analog fine-grain RISC SCAMP Q-Eye (Homework will be on this architecture!) Biological sensor-processor Multiple layers, specialized cells (non-programmable) Silicon based sensor-processor Single programmable layer 1

2 Eye-RIS system AnaFocus Ltd, Seville Q-Eye chip is a SIMD architecture Single Instruction Multiple Data Based on the Q-Eye sensor-processor chip 176x144 array size ~30x30 micron cell size analog SIMD processor array 100mW An array of 176x144 locally interconnected uniform processing elements (called cells) Each processor have own physical address Efficient local communication Same instruction is executed in each proc. at a time instant Local memories are in the processing cells There is a sensor in each processing cell One control unit is shared by the array 1:1 pixel-processor mapping (fine-grain) Ctrl. Unit Digital Microprocessor Universal Turing machine Time-multiplexed hardware resources Functionality determined by software data storage program processor x1,x2,x3 xi x x = F(pn,s,x) x s = G(pn,s,x) p1 p2 p3 p4. pn pn instruction Analogue Microprocessor Universal Turing machine Time-multiplexed hardware resources Functionality determined by software Discrete-time continuous-value ANALOGUE MICROPROCESSORS data storage program processor x1,x2,x3 xi x x = F(pn,s,x) x s = G(pn,s,x) p1 p2 p3 p4. pn pn instruction How the processor array works? Memories: Entire 176x144 sized 1 bit or analog images Operations: Shift, add/sub, copy, conversion, logic, diffusion Capture LLM1 Shift_Down LLM1 LLM2 XOR LLM1 LLM2 LLM3 LLM1 LLM3 LLM2 Sensing images through the photodiodes of the sensor layer; Storing up to 6 grayscale and 4 binary images; Performing grayscale image additions, subtraction and scaling; Performing diffusion operator; Performing grayscale to binary conversions; Performing logic and morphologic operations; Up to 1000 FPS loading and downloading of images (176x144); Q-Eye Chip Architecture Analog processing components 2

3 Cell Architecture Basic operations Grayscale Diffusion (Gaussian, directional, masked); (~12ms) Multiple-add (MAC); (~2ms) Shift; (~0,5ms) Threshold; (~2ms) Mean ; (~2ms) Difference (positive, negative, absolute, signed); (~4ms) Image capture Linear or HDR; Morphologic Arbitrary 3x3 morphologic operations; (~0,2ms) Logic AND, OR, EQU, XOR, NOT, etc. (~0,1ms) Image I/O Full, (gray: 153ms, binary: 27ms) part, Address event. Image acquisition with different integration times Sobel operation 24ms 200ms 1.6ms 6.4ms 12.8ms 25.6ms 51.2ms 102.4ms Result Mean calculation Thresholdings Threshold global Threshold global adaptive Final result Evaluation time: 200ns The final result is a completely smooth image, which preserved the charge. Diffused image Threshold local adaptive 3

4 Diffusion operation Conditional operation Conditional copy 80 ns 600 ns 900 ns 1600 ns s mask conditional copy Conditional diffusion mask diffusion result Algorithmic processing Analog transient for a non propagating instruction: 600ns (typical) Output Bottleneck Issue Calibration between intermediate processing steps: 3 ms (typical) B&W processing tasks are executed 4 times faster 176x144 image Median Filter Sobel Edge 25 kb 6500 frames/sec (grayscale readout) Input Input 2 kb Sobel Low-Pass Erosion + XOR Shadowing Binary Map frames/sec binary readout How many black pixels? 2 Bytes Any black pixels? 1 bit Conventional Vision System Eye-RIS Vision System Be aware of the data size of F! In this case: f << F Pixel-wise local processing is more efficient in the analog domain in an array computer 4

5 Eye-RIS system Software Eye-RIS ADK Nios II: ANSI C/C++ Q-EYE: CFPP Programming Programming Q-Eye chip The CFPP Image Processing Library (IPL) Spatio-temporal filters Arithmetic operation between images Thresholding Morphological and logic binary operations The Extended Image Processing Library (EIPL) Blob management Classifying functions Linear and non-linear digital processing of grey-level images Geometrical transformations Programming Altera Nios II C,C++, Assembly The Eye-RIS Basic Library (EBL): Control the execution of the CFPP code I/O management Sending and/or receiving images to/from the PC either to be displayed or saved to disk Printing information messages to a console Timer management Programming Eye-RIS (sync) 5

6 Programming Eye-RIS (async) CFPP global memories // Declaration extern fpp_int value; extern fpp_bool flag; extern fpp_time texp; // Access void foo() value = 123; int foo = value; if(flag) FPPTime_write(texp, 500); int readtexp = FPPTime_read(texp); Acess Exercises (Everyone should select one) FJf4KLwZkz1YxSjg/edit?usp=sharing Documentations You will need Eye-RIS v1.3 IPL Reference.pdf mostly Please do not share the link on public places! Getting started with the Eye-RIS system Login: Admin (no password) Make sure that the camera is plugged both to the 220V and the laptop via Ethernet cable Start Eye-RIS ADK 10.1 File New Eye-RIS project Add your project name Set Eye-RIS1.3 as system version Finish Edit the two codes Main.cpp CFPPCode.fpp Compile and Run with the green triangle Select Eye-RIS application If neverending cycle, stop with red square If it the camera is not found, unplug, plug, and/or reset the laptop (Windows) Make sure that the optics is set properly Programming the Eye-RIS system First task Capture an image Apply thresholding Find edges Display images Main.cpp program code #include "eyerisbl.h" extern fpp_int img0, img1, img2, img3, img4; int main() setnumberofwindows(5,5); while (1) Section_execute(Sample1); Image_display(img0, WINDOW_0, GREY); Image_display(img1, WINDOW_1, BINARY); Image_display(img2, WINDOW_2, BINARY); Image_display(img3, WINDOW_3, BINARY); 6

7 FPP program 1 #include "ipl.hfpp" int img0=0, img1=1, img2=2, img3=3, img4=4; void section Sample1() time exptime = 15; int gain = 2; Sense_acquire(LAM_0, exptime, gain); Move_downloadImage(LAM_0, img0, GREY); Thresh_global(LAM_0, LDM_0, 128); Move_downloadImage(LDM_0, img1, BINARY); Morph_erode(LDM_0, LDM_1, CONNECT_8, 1, BINARY_BLACK); Move_downloadImage(LDM_1, img2, BINARY); Logic_xor(LDM_0, LDM_1, LDM_2); Move_downloadImage(LDM_2, img3, BINARY); Problems The illumination changes dislocate the edges There are many single white points on the results Improved second version Adaptive thresholding is applied The single white points are deleted void section Sample2() time exptime = 15; int gain = 2; long ncycles=300; FPP program 2 Sense_acquire(LAM_0, exptime, gain); Move_downloadImage(LAM_0, img0, GREY); Move_moveImage(LAM_0, LAM_1); Filter_diffusion(LAM_1, ncycles); Move_downloadImage(LAM_1, img1, GREY); Thresh_local(LAM_0, LDM_0, LAM_1); Morph_removeSinglePoints(LDM_0, LDM_0, BINARY_BLACK); Move_downloadImage(LDM_0, img2, BINARY); Morph_erode(LDM_0, LDM_1, CONNECT_8, 1, BINARY_BLACK); Move_downloadImage(LDM_1, img3, BINARY); Logic_xor(LDM_0, LDM_1, LDM_2); Move_downloadImage(LDM_2, img4, BINARY); 7