Multimedia in Every Mobile. Peter Chiang Nov, 04

Size: px

Start display at page:

Download "Multimedia in Every Mobile. Peter Chiang Nov, 04"

Dominic Berry
5 years ago
Views:

1 Multimedia in Every Mobile Peter Chiang Nov, 04

2 A statement about the launching of SMedia Color mobile-phone occupy over 60% mobile-phone market in 2004 Camera phone occupy over 50% of color mobile-phone in 2004 Multimedia become default function of mainstream handheld device in 2005 Enhanced capability drives data bandwidth and carrier revenue Multimedia offers handset differentiation opportunity Consumers will pay for compelling games and content SMedia delivers key enabling technologies for multimedia handheld devices

3 SMedia at a Glance Founded: 2004 Aug Initial Capital: USD 6 Million (approximately) Major Shareholder: UMC Employees: 60 (approximately) Headquarters: Operating Sites: Taiwan/ Hsin-Chu S&M, Service: Taiwan/ Taipei

4 Core Competence R&D - 100% own IP for core multimedia features - First full hardwired solution of low-power graphics, streaming, imaging integration in the world - Initial/Endorse industry standard via participating these committee/organization Service and Support - Long-term relationship and timely support to customers with time-tomarket product support - Close relationship with alliance partners to synchronize in latest technology development and industry specifications.

Core Value Integrated core technologies into real innovation Deliver the valuable innovation to customers completely Audio up Graphics Innovation

5 Core Value Integrated core technologies into real innovation Deliver the valuable innovation to customers completely Audio up Graphics Innovation Imaging Video Low Power >Real-time feedback > >Efficient logistics > Deep service > Knowledge transfer> > Total solution > Customer Satisfaction Integraion

6 About SiS Multimedia Score SiS 16% Nvidia 3% ATI <1% VIA/S3 9% SiS Matrox 1% 10% ATI 13% Nvidia 31% Others 1% Intel 35% VIA/S3 17% SiS 4% Matrox 3% Intel 64% IGC Desktop Market Share Q2'03 Desktop Graphics Market Share Q2'03 ATI 28% Nvidia 65% Source: Jon Peddie Research Note: Discrete Desktop Market Share Q2'03 SiS GPU department spun off at Q2 03. So there is no correct statistic data after Q3 03. SiS GPU department combined Trident Inc. as a new company XGi, which became the top 3 GPU suppliers, and SiS is still the biggest shareholder.

7 Glamo 3360

8 Mobile Phone (Handheld device) Trend 4 Development Directions for Feature Phone: Multimedia, Interactive, User-friendly, Personalization Multimedia Polyphonic Native 2D GamingNative 3D Gaming WAP Broadcast 3D MMI Still Camera Video Camcorder Video Broadcast Interactive SMS EMS Smart message MMS On-line Instant Gaming Messaging Video Conference

9 Inverse Screen Size Law The need for advanced graphics Smaller screens need more advanced graphics processing per pixel To make best use of restricted screen real-estate Advanced graphics techniques provided by state-of-the-art APIs Anti-aliasing, multi-level compositing, rich media, 3D Video Processing and Interactive synchronization 3D Gaming Content Perspective windows maximize screen real-estate Translucent Menus don t obscure lower layers Anti-aliased text to make small text extremely legible

10 SMedia Solution SMedia is integrating the interactive multimedia visual solution for handheld devices to enable new applications that enrich more enjoyment and quality to life. Asynchronous SRAM Memory Controller CCD/CMOS COMS Sensor Video Capture, Image Processor JPEG, MPEG4 Codec 2D/3D Graphics LCD Controller LCD Host Bus Controller MMC Controller MMC Host CPU

11 Multimedia Mobile Phone Evolution Stacked SRAM Lens Sensor Camera Interface 2D Engine 3D Engine JPEG decode JPEG encode MPEG decode MPEG encode LCD Interface LCD Melody & Audio Amplifier (MP3/MIDI ) Speaker & Vibrator Antenna RF BB Power Management Memory Battery

12 Possible Architecture 1 Traditional multimedia phone architecture Lens Sensor Imaging DSP YUV Camera sensor interface JPEG codec MPEG4 codec 2D engine 3D engine RAM LCD interface (CPU interface) LCD Mem B.B.

13 Possible Architecture 2 Newer multimedia phone architecture Lens Sensor Raw RGB Imaging DSP CMOS sensor interface Advantage: Save total system cost, theoretically Disadvantage: Need extra effort for calibration Lack flexibility JPEG codec MPEG4 codec 2D engine 3D engine RAM B.B. LCD interface LCD Advantage: RGB Save Memtotal system cost, theoretically From graphic view, it can perform much better quality Disadvantage: More bandwidth usage of stack psram for LCD display

and Systems Paper : 2490 Rendering performance with texturing Glamo 3D Hardware : 5M pixels/sec (80 times of the StrongARM 200Mhz with

14 Why H/W Graphic Accelerator For Performance Geometry performance with lighting Glamo 3D Hardware : minimum 100k polygons/sec (15 times of the StrongARM 200Mhz with 160Mhz FPU) Max 1280k polygons/sec Max 33M pixels/sec ISCAS 2002 IEEE International Symposium on Circuits and Systems Paper : 2490 Rendering performance with texturing Glamo 3D Hardware : 5M pixels/sec (80 times of the StrongARM 200Mhz with 160Mhz FPU) Reference data: a) QVGA 320x240x2 overdraw x30fps/(5x5/2pixels/triangle)=369kpoly/s b) QVGA 320x240x2 overdraw x30fps=4.6mpixels/s

Why H/W Graphic Accelerator For Power Consumption Software solution (executed by CPU) Define every picture Clean & Redraw every movement on display Execute complex instructions

15 Why H/W Graphic Accelerator For Power Consumption Software solution (executed by CPU) Define every picture Clean & Redraw every movement on display Execute complex instructions to compute simple movement Saturation accessing memory Hardware solution Only define some components: Object oriented operation to execute a simple instruction to do complex action

16 Comparison of Performance & Power Performance: 4.6 times speed (200x200pixel BLTs x 30 times) PDA with PXA250 + Graphic Accelerator PDA with PXA250 only 25ms 115ms Power consumption: 85% less power (see cyan-blue portion) Only initial (around 10ms) need higher power

17 Outstanding MPEG4 Codec Full hard-wired solution better performance + less power consumption vs E company s half H/W + half SW Full support MPEG4 simple profile better quality & efficiency Based on the VM 18 & MPEG-4 reference code: A modified mode decision algorithm for deciding the MB mode. Can achieve over 0.2dB improvement for complex sequence and 1.0dB in scene change. A new rate control algorithm can remain high quality when scene changed. Full search in search range I & P frame support P frame will have over 5X performance/quality than I frame Special Scaling-down process as a partial low-pass filter to reduce the blocking & ringing Rate-control support to meet HVS(Human Visual System) behavior Human eyes are more sensitive on low-complexity MB(Macro Block), rate control can reduce blocking impact vs A company s H.263 only Just one of MPEG-4 mode (short-head mode) vs A company s No rate control vs S & A companies I frame support only vs S & A companies blocking & ringing result Ringing & Blocking Example

18 Why H/W Camera/Camcoder Dramatics Power Saving: around 10X~

19 Technology of Power-saving Clock Low power PLL for 32KHz clock input (standard RTC clock for the standby usage of mobile-phone) Active Graphics Drawing and rendering more power efficient on dedicated hardware than on Baseband SMedia solution spends less time drawing power by her decades of graphic experience(sis experience) Image capture & video recording Page-mode processing to minimize memory accessing times which is the major power consume Screen Refresh Display refresh isolated from system & Baseband Partial Display make the efficient power saving for whole system ASIC Design for Power Stack psram, not SDRAM Auto switch-off non-active function Low-power process

20 Glamo Measured Core Power Summary Average Power can be < 1mW per 1MHz

21 Best Solution (1) Economical system solution Add ISP to reduce camera module cost save around $0.5 for VGA, around $0.5~ $1 for 1.3M model Add LCD controller w/ frame buffer to reduce LCM cost save around $4+ for 176x220 model; save around $10+ for 240x320 model Short developing period Strong FAE support with completed documents Sample code supply for application support Strategy alliance with Game/Middleware/LCD/CMOS/BB vendors to give a total solution choice Small power-consumption system solution With power-saving design and process (~85% less power than one processor solution) The most advanced process support by UMC group Follow main stream standard Compatible with most of I/F of BB, LCD, and CMOS/CCD sensor Follow 3GPP video stream standard Follow Java & OpenGL_ES graphics standard

22 Best Solution (2) Fancy features Full 2D & 3D graphic acceleration Newest MPEG4 (up to CIF 30fps), JPEG codec for imaging & video application Fancy imaging processing of digital camera 16X Continuous Zoom, Flash synchronization control, Emboss/Sepia/Mono/Negative/Color-Filer effect Best performance Best performance of 2D/3D (80X than strong-arm simulation) Full hardwired architecture make the most efficient imaging processing Full hardware MPEG4 acceleration make the fastest video compression Flexible architecture Support most of CMOS sensor w/ programmable compensation YUV & Bayer Pattern input Support most of TFT LCD CPU I/F, RGB I/F Brightness Statistics Table output to reserve the flexibility for manufacturers to make own AE/AF Support the rotation/mirror of Camera & LCD & 2D object to make your design more varied Support extra Java H/W accelerator SIP solution

23 Strategy Partner U company Graphics AP Game G company B company I company CMOS/CCD S company O company B company P company A company M(F) company M company T company S company N company S2 company SRAM Middleware & authoring tool SMedia API Driver Glamo coprocessor Baseband I company M company A company OS I company H company X company F company S company Java accelerator N company LCD S company S2 company C company T company A company S3 company

24 Features & Specification

25 Feature List (1) Host Interface & Clock Support 16bits SRAM-like bus Support most of B.B./CPU chip (16 bit 80/68 series) Input Clock: KHz or 13MHz JPEG Engine Fully Compliant with JPEG ISO/IEC Baseline standard JPEG encoding and decoding Display Interface Display Resolution: Max. 512x512 75Hz 16/18bpp (RGB565/RB666) Support CPU I/F, RGB I/F, LCD I/F Support C-STN and TFT of most of LCD module venders Display 90 o /180 o /270 o Rotate Support 8/9/16/18 bit Output Interface Gamma correction Partial Display at Power Saving Mode. (LCM should support)

26 Feature List (2) MPEG-4 Engine Full compliance with ISO/IEC (MPEG4) video simple profile level 0,1,2,3 Full H/W acceleration MPEG4/H.263 Encoder and Decoder CIF/QCIF 30 FPS Provide audio and video synchronization scheme High Performance 2D Accelerator Full compliant with J2ME MIDP 2.0 2D requirement Built-in an 1T pipelined 16bit BitBlt graphics engine ROP3, Rectangle fill, Font expansion, Line-drawing Transparent BitBlt with source and destination keys Alpha blended Bitblt, Clipping Support maximum fill rate: minimum 33M pixels per second

27 Feature List (3) High Performance 3D Accelerator Full compliance with OpenGL ES v 1.0 and OpenGL ES v 1.1(proposal7) Full compliance with JSR Mobile 3D Graphics API for J2ME TM Built-in pipelined 3D graphics engine Nearest/Bi-linear Perspective Corrected Texture Mapping Texture transparency, blending, wrapping, mirroring and clamping 1/2/4/8/16-bit texture format Support palette texture and compressed texture 16-bit Z-buffer, triangle setup engine Fog, alpha blending and specula effects Alpha blended Bitblt, Clipping, dithering Extra 8 lighting support Support maximum fill rate: 33M pixels per second Support polygon rate: 150~1280K polygons per second

28 Feature List (4) Video Interface Max. Image Resolution : 3 M Sensor Interface supports : programmable image size up to 3 Million-pixel CCD/CMOS sensors. Supports Input Formats Bayer (Raw Data) YUV 4:2:2 Gamma Correction by programmable look-up table. Lens Shading Correction. Bad Pixel Compensation. Black Level Compensation. Support Flashlight function. Preview/MPEG Frame Rate Control. Independent Size Scale-down of Vertical or Horizontal direction. Support 180 o Rotate & Mirror Software Support Nucleus, Smartphone, Linux, Symbian Process and Package UMC 0.15um low power process 3.3~1.8V I/O VDD and 1.5V core VDD supply BGA 8mm x 8mm 140 ball 0.6mm ball-pitch

29 S/W Architecture 2D/3D Midlet 2D/3D Native Application (C/C++) MIDP 2.0 JSR184 Game-related module KVM (CLDC 1.1) 3D Graphics Middleware OS-related Graphics API SMedia 2D/3D API (OpenGL ES) RTOS Core SMedia Driver Suit Suit Hardware (Processor + Glamo coprocessor)

30 Glamo Product Roadmap ES CS MP

31 Glamo 3360 Schedule First customer sample Nucleus driver Application-note release EV kit (1) SDK (Drivers, APIs, Tools, Sample programs) EV kit (2) Smartphone 2003 driver (1) Smartphone 2003 driver (2) Linux driver Mass production 3360 Symbian driver 8/E/2004 9/M/2004 9/M/2004 9/M/2004 9/E/ /E/ /E/ /E/ /M/ /M/2004 2/E/ J : CS: 12/E MP: 2/E, : CS: 12/M MP: 1/E, : CS: 12/E MP: 2/M, 2005

32 Thank you

Multimedia in Mobile Phones. Architectures and Trends Lund

Multimedia in Mobile Phones. Architectures and Trends Lund Multimedia in Mobile Phones Architectures and Trends Lund 091124 Presentation Henrik Ohlsson Contact: henrik.h.ohlsson@stericsson.com Working with multimedia hardware (graphics and displays) at ST- Ericsson