Design challenges for wireless smart cameras

Size: px

Start display at page:

Download "Design challenges for wireless smart cameras"

Allan Anthony
5 years ago
Views:

1 Design challenges for wireless smart cameras Marc Heijligers Richard Kleihorst, Anteheh Abbo, Vishal Choudhary, Leo Sevat, Ben Schueler Philips Research, Eindhoven Contents Context Vision Platform Architecture Vision IC technology WiCa: Wireless Camera Future Challenges Conclusions Marc Heijligers, Philips Research MPSoC

2 Context Consumer faces overflow of Control equipment Complex Events Marc Heijligers, Philips Research MPSoC

3 Smart Vision for Comfort, Ease of Use and Safety Marc Heijligers, Philips Research MPSoC Smart Vision for Comfort, Ease of Use and Safety Marc Heijligers, Philips Research MPSoC

And many more Learning Play instruments Safety for automotive Lane

detection Security Guarding, babysitting Calibration Control Professional

Healthcare Patient monitoring Marc Heijligers, Philips Research MPSoC 2006

devices. Small mobile devices instead of large super computers.

4 And many more Learning Play instruments Safety for automotive Lane tracking, pedestrian detection, traffic signs, airbag control, sleep detection Security Guarding, babysitting Calibration Control Professional appliances, 3D-TV rendering Autonomous vacuum cleaning, robotics Healthcare Patient monitoring Marc Heijligers, Philips Research MPSoC Smart Vision Application Domain The eyes and intelligence for consumer devices. Small mobile devices instead of large super computers. Consumer pricepoint. Image from Scientific American, May 2005 Marc Heijligers, Philips Research MPSoC

5 Computational Requirements Mobile Multimedia 0.3 GOPS 1 GOPS Home-Dialog/Robotics ~ 1 GOPS Automotive Vision ~ 5 GOPS 3D-Video ~ 20 GOPS High-Performance Video Restoration ~ 1 TOPS Marc Heijligers, Philips Research MPSoC Example: Mobile 3D Experience Specification Resolution : ~ VGA (640 pixels x 480 lines) Frame Rate: 30 frames/sec Tasks : 2D-3D conversion 500 Ops/pixel 3D rendering 500 Ops/pixel 10 Mpixel/s 10 GOPS State of the art Xetal CMOS18 : 5 GOPS/Watt Battery mobile : 3 Watt-hour 1.5 hours!!! 1 TOPS 200 Watt Marc Heijligers, Philips Research MPSoC

6 Vision Platform Architecture The Sense-Think-Act cycle Embedded ICs Computer science Smart Sensors THINK System Control SENSE ACT CMOS sensors Physics world / scene Engineering Floris Mantz, TUDelft Marc Heijligers, Philips Research MPSoC

7 Video Analysis Partitioning Video Low Level Intermediate Level Pixel Processing - Image improvement - Edge Enhancement - Convolution kernels Object processing - Shape analysis/coding - Segmentation Data High Level Decision-making Real Time OS Networking Marc Heijligers, Philips Research MPSoC Video Analysis Mapping -> heterogeneous Video Low Level Pixel Processing M Pixels/sec - Similar processing per pixel SIMD Intermediate Level Object processing K Objects/sec - Similar processing per object MIMD FPGA Data High Level Decision-making Real Time OS Networking RISC VLIW Marc Heijligers, Philips Research MPSoC

8 Vision IC technology SIMD Scaling: 1PE - 320PEs 0.02mm mm 2 Data Memory 0.6mm 2 1mm 2 1 MHz Control Program Memory 25MHz Performance Size Perf/area Bandwidth 25MOPS 11.4 mm MOP/mm 2 500Mb/sec 9.8mm mm mm 2 Data Memory 0.02mm 2 1mm 2 0.6mm 2 Program Memory Performance Size Perf/area Bandwidth 8 GOPS 18.0 mm MOP/mm Gb/sec MHz Control 25MHz Marc Heijligers, Philips Research MPSoC

9 Locality of reference A B C = A + B; C = A > B? A : B; PE instruction C Typical DSP instructions need 4 accesses to memory Locality of reference saves power! Marc Heijligers, Philips Research MPSoC Programmable SIMD core: XeTaL 320 PEs 64K 2x16X320=10k wires Lines(*) Global Global Control Processor Control Processor GPI GPO I2C 2x16X320=10k wires 2 Lines 2048 Video In DIP 3x2560 DOP 3x2560 DOP Video/Data Out (*) depending on Xetal-version Marc Heijligers, Philips Research MPSoC

Xetal - 3D 320 processors 50 GOPS max pixel performance 300mW power consumption 21 mm 2 area, 150nm Marc Heijligers, Philips Research MPSoC 2006 19 Xetal family Xetal-I Xetal-3D Xetal-II (**) Typical

0Tbit/s Data-path Resolution 10-bit 10-bit 16-bit On-Chip Memory 100 kbit: 16 VGA lines 200 kbit: 64 ½ VGA lines 10Mbit: 4 VGA frames Input Video Port 1x10bit 4x8bit 3x10bit Output Video/Data Port

10 Xetal - 3D 320 processors 50 GOPS max pixel performance 300mW power consumption 21 mm 2 area, 150nm Marc Heijligers, Philips Research MPSoC Xetal family Xetal-I Xetal-3D Xetal-II (**) Typical Clock Speed 25MHz (max) 80MHz (max) 200 MHz Max. Performance (+, *) 16 GOPS 50 GOPS 200 GOPS Internal Memory Bandwidth 150Gbit/s 450Gbit/s 3.0Tbit/s Data-path Resolution 10-bit 10-bit 16-bit On-Chip Memory 100 kbit: 16 VGA lines 200 kbit: 64 ½ VGA lines 10Mbit: 4 VGA frames Input Video Port 1x10bit 4x8bit 3x10bit Output Video/Data Port 3x10bit 4x8bit 3x10bit Maximum input rate 380Mbit/s 1920Mbit/s 2400Mbit/s Silicon Technology 180nm 150nm 90 nm Chip Area 22 mm 2 18 mm 2 72 mm 2 Number of Gate Equivalents 700 kgates Power Consumption 20mW/GOP 15mW/GOP 10mW/GOP ** Recent results on the tester Marc Heijligers, Philips Research MPSoC

Consumption[nJ/pixel] 15 10 W=13 W=11 W=9 5 W=7 W=5 W=3 0 0 100 200 300 400 500 600 Number of PEs - Energy saving through

11 Proposed Chip Layout 128b x 2048 TILE3 TILE0 DOP PDP ISR OSR 8 PEs DIP GCP Xetal-II (8mm X 9.5 mm) Xetal-I C090 (2.9mm X 1.9 mm) TILE39 Marc Heijligers, Philips Research MPSoC SIMD power consumption 25 SIMD Energy Consumption W 20 Enery Consumption[nJ/pixel] W=13 W=11 W=9 5 W=7 W=5 W= Number of PEs - Energy saving through parallelism is linked to the computational complexity - Without voltage scaling, energy saving levels off Marc Heijligers, Philips Research MPSoC

12 Xetal-II Power management DIP Line-Memories DOP LUT LPA GCP Vdd Vddr Vddg Power Management IC Xetal-II Voltage Domains. The supply voltage of the LPA is controlled to follow the performance demands of the processing. The GCP supply is the only remaining in hibernation mode. Marc Heijligers, Philips Research MPSoC SIMD with Voltage Scaling (Xetal-1 1 case) Scaling Energy Consumption in SIMD Architectures 30 GOPS 0.8 Energy Scaling Factor GOPS 10 GOPS GOPS Number of PEs Marc Heijligers, Philips Research MPSoC

13 Estimated State of the Art Processors "Peak" Performance in 32-bit GOPS Normalized to 130nm Normalized Processors 1.2V, 32-bit Ops Including memories PicoChip CELL Stream BF561 TI C64x EVP SiHive Sandbridge SiliconHive Visconoti Xtensa SC140 ondsp VI1 Macgic GP DSP Vector Energy Efficiency in 32-bit MOPS/mW Courtesy: Raghavan IMEC Marc Heijligers, Philips Research MPSoC Coolfux XeTaL-II full performance supply voltage scaling XeTaL-II V T scaling limit IPE WiCa: Wireless Camera 13

14 Wireless power consumption Power mains 1W UTP 100m a battery 10m Bluetooth 1m Zigbee Autonomous 100µ PicoRadio power/bit does not scale with Moore s Law Data Rate b/s 100 1k 10k 100k 1M 10M 100M ~sensor ~ speech, audio, hifi ~ moving pictures (R.Roovers) Marc Heijligers, Philips Research MPSoC Power consumption of a wireless camera A typical wireless digital VGA camera needs 200 mwatt to transmit life video over a short distance. 1 year of daily use means 1750 Wh. Using a good fuel cell it consumes about 2.2 liters of methanol fuel per year. This takes about 400X the total camera volume. Marc Heijligers, Philips Research MPSoC

15 Intrinsic power consumption in the radio path DSP DA RF PA TRANSMITTER E T transmit signal processing energy per bit E L Link energy per bit E T [nj/bit] Bluetooth E L [nj/bit] Solution/system partition: Perform video processing locally Marc Heijligers, Philips Research MPSoC 2006 Raf Roovers 29 WiCa architecture XeTaL 8051 SIMD CPU Single threaded Heterogeneous Marc Heijligers, Philips Research MPSoC

WiCa implementation Best-in-class networked smart vision platform: 5 high-end PCs pixel

Quick prototyping. Allows application innovation with partners.

16 WiCa implementation Best-in-class networked smart vision platform: 5 high-end PCs pixel performance Mobile, battery powered. Standardized interfaces and components. Easily programmable. Quick prototyping. Allows application innovation with partners. Marc Heijligers, Philips Research MPSoC Power consumption Corner features A unique list of corners describes the object. Corner detection on SIMD, list matching on DSP/CPU 30fps VGA SIFT (Scale Invariant Feature Transform) Size independent characterization. 30fps VGA Marc Heijligers, Philips Research MPSoC

17 Face detection video Marc Heijligers, Philips Research MPSoC Depth estimation from stereo Marc Heijligers, Philips Research MPSoC

18 Future Challenges Architectural Challenges XeTaL ARM SIMD CPU Memory, memory, memory - What memory hierarchy is needed to enhance locality of reference? - Which ratios of size? - Which interfaces? - What type of feedback loops? - Do we need to partition the external memory in different banks? - Standalone vs. system integration. - What memory model for programming? (cache/scratchpad/mix) - How can we share the memory of other cameras? - Should support dynamic extension of the network Marc Heijligers, Philips Research MPSoC

19 Architectural Challenges XeTaL ARM SIMD CPU Performance increase - More performance?=>? More Xetals and/or ARMS? - How to distribute workload over multiple WiCas? Programming model - How to sync SIMD/CPU? - Memory map? - Where to put databases for recognition? Marc Heijligers, Philips Research MPSoC Architectural Challenges XeTaL ARM SIMD CPU Application - How to calibrate? How to perform arbitration? - How to share/decide about multiple view geometries? Marc Heijligers, Philips Research MPSoC

20 Integration challenges: 3D? More performance, less power Marc Heijligers, Philips Research MPSoC Conclusion 20

Conclusions Smart vision architecture with Low power SIMD architecture Power management State-of-the-art performance Programmable, enabling a wide scope of applications Only

21 Conclusions Smart vision architecture with Low power SIMD architecture Power management State-of-the-art performance Programmable, enabling a wide scope of applications Only possible with experts ranging from application domain via compilers to deep-submicron Challenging SoC design and integration problems Marc Heijligers, Philips Research MPSoC

SMART CAMERA MOTE WITH HIGH PERFORMANCE VISION SYSTEM. Richard Kleihorst, Ben Schueler, Alexander Danilin, Marc Heijligers

SMART CAMERA MOTE WITH HIGH PERFORMANCE VISION SYSTEM Richard Kleihorst, Ben Schueler, Alexander Danilin, Marc Heijligers Philips Research Laboratories, WAY4.1, Prof. Holstlaan 4, NL5656AA Eindhoven, The