Mobile Multimedia Application over DOrA How to Ensure Positive End-user Experience? Page 1
Agenda Why DOrA? Multimedia Applications over DOrA Multimedia Applications testing requirements Overview of current video analysis methodologies and algorithms Spirent s unique approach to mobile multimedia applications testing Page 2
Drivers for EV-DO Rev A Faster 3500 3000 Cheaper Better Data Speed? 2500 2000 1500 1000 500 0 IS-95 1xRTT EV-DO Rev. 0 EV-DO Rev. A (1995) (2000) (2002) (2007) Commercial Service? Page 3
Applications Enabled by EV-DO Rev A Higher Data Rates Lower Latency End-to to-end QoS Enables Video Streaming Video Telephony Mobil IPTV Voice-over over-ip (VoIP) Push To Talk Instant Messaging Page 4
Mobile Multimedia Applications What Makes Them Feasible Now Transport Technologies Higher data rates, lower latency, higher QoS Devices Higher processing power, more memory, high-end displays Improved Content Mobile Applications Higher sources of content quantity and quality SMS MMS File Sharing Image Sharing 384 kbps 153 kbps Short Video burst Streaming Video 1.8 Mbps 2.4 Mbps 3.1 Mbps Layer 1 Throughput Rates CDMA2000 UMTS Release 99 HSDPA with QPSK only EVDO Release 0 EVDO Release A Air Interface Page 5
Key Wireless Applications Market Growth Indicators Many operators offering high speed data services More multimedia capable devices Better content Increased Purchase of New Applications Increased Number of Repeat Applications Purchases Mobile Applications present a significant opportunity for higher ARPU Repeat Purchase & Increased Usage are key measures of Operators Success Page 6
Mobile Multimedia Applications Market Drivers Growing Demand for New Services Video On-Demand Video Streaming Video Telephony Mobile TV New Areas of Opportunities Potential for being the next killer app Service Differentiation Increased Revenue ~4.3 million to 14 million Mobile TV subscribers by 2009 ~$700 million to $2.3 billion in revenue Page 7
Mobile Multimedia Applications Testing Challenges Ensuring Positive Quality of End User Experience QoS Perceptual Quality of Content Overall satisfaction Mobile Video Testing Standards unavailable Mid-late 2007 (VQEG, IT) Current Testing methodology limited No content-based objective methodology available testing limited to subjective testing Rapidly Changing Requirements New applications, technologies & physical design changes Page 8
Video/Voice Quality Assessment Testing Methodologies Quality Assessment Subjective Objective Capture MOS Objective methodology can provide more detail data in addition to the quality MOS Analysis Full Reference MOS + Cause Analysis Non-Reference MOS + Cause Analysis Page 9
Quality Assessment Tools Subjective Testing Benefits and Limitations Mean Opinion Score (MOS) Listening or Video Quality are defined as a measure of listener/viewer s satisfaction and are generally expressed as a Quality MOS Score Excellent 5 Good 4 Fair 3 Poor 2 Bad 1 Advantages Reflects the true end user experience based upon their Psycho visual quality experience Disadvantages Usually involves organizing large study groups Time consuming Cost prohibitive Studies are not repeatable Objective methodology offers cost-effective, repeatable tests in a much shorter time Page 10
Objective Multimedia Analysis Methodologies Challenges of Capturing Video from End User s Perspective Invasive methods Raw data stream Captured with device and decoded on PC with media player Doesn t account for device codec and display Decoded video extracted from device No standard video output interface available Device-resident capturing application impacts performance Invasive Methods do not capture the exact end-user experience Non-invasive methods Camera solutions allows replicating true end use experience No impact on device performance Device-independent Page 11
Multimedia Analysis Methodologies & Tools Full Reference & Non-reference Models q Full-Reference objective models collect the information from both sides of the network Source Network Output under test q Non-Reference model makes use only of the output signal q Advantages of using both models Experience Expectation Semantics Having the two models will provide a coverage for all testing scenarios - in the absence and the presence of a reference test signal Subjective assessment Intrusive (full reference) model VQuad SQuad-LQ Learn from Subjects Non-Intrusive (no reference) model VMon P.563 Page 12
APEX Wireless Multimedia Analysis (WMA) Module Offering Unique Value Propositions Uniquely flexible video capture & device fixture with High resolution camera & universal fixture Powerful objective & technology independent multimedia analysis tool Industry leading quality analysis video/audio algorithms In-house subject matter expertise to quickly respond to changing requirements & help drive standards Page 13
APEX Wireless Multimedia Analysis Module System Architecture Video Test Clips Algorithms Media Server Video Streaming Video Telephony Cellular Network Mobile TV Broadcast Wireless Multimedia Fixture Captured Video Controller PC APEX Test Manager Page 14
Quality Video/Voice Analysis Requires Comprehensive Tools Industry Leading SwissQual Video/Voice Quality Algorithms Full Reference Video model VQuad Non-Reference Video model VMon Full Reference Voice model SQUAD LQ/NS/AEC/RTT Non-Reference Voice model NiNA+/P.563 Advanced Video testing solutions Lip-Sync (audio & video synch) Video Telephony Video Streaming Mobile TV Page 15
Quality Video Analysis Requires Comprehensive Tools VQuad A Full Reference Quality Assessment Model Full Reference Test (VQuad) Intrusive Algorithm Human vision Results MOS PSNR Block distortion Blurring Packet Loss Jitter Jerkiness Level Audio Video synch Reference Video Network Format Conversion Contrast and Gain Alignment Perceptual Modeling Distortion Evaluation Video Quality Measure Captured Video Page 16
Quality Video Analysis Requires Comprehensive Tools VMon A Non-Reference Quality Assessment Model Non-Reference Test (VMon) A Non-Intrusive Algorithm Human vision Results MOS Repeated Frames Fps Calculated Inter-frame Difference Colorfulness Black Frames Skipping with Pausing Blurring Blocking Source Video Network Format Conversion Perceptual Modeling Distortion Evaluation Captured Video Temporal & Spatial Analysis Video Quality Measure Page 17
How to optimize the video quality assessment Process? Providing root causes analysis of video helps Reduce development time Improve efficiency of testing process Accelerate analysis and improvement of positive end user experience Root Cause Examples Blurriness Block Distortion Tiling & Slicing Colorfulness Frame Freezing Page 18
Spirent-SwissQual Leading the Way in Standards ITU-D ITU ITU-R ITU-T Head Perceptual Quality Measures Group in ITU-T T SG12 Universities Industries ETSI STQ-mobile Reporter JRG-MMQA VQEG Editor ITU International Telecommunication Union ETSI European Telecommunications Standards Institute VQEG Video Quality Expert Group STQ Speech Quality Group JRG-MM Joint Raporteurs Group for MultiMedia MMQA MultiMedia Quality Assessment MMQA Standard Recommendations Provides the actual Audio/Video quality measurement methods Provides framework for drive testing and measuring Audio/Video Quality Page 19
Customers Demand for More Flexibility and Control APEX Test Manager is an open-architecture, scalable test system designed to meet the rapidly evolving needs of Applications testing. Easy-to-use Customizable Automated APEX Platform Open Tests Modular Data Performance WMA App x App y Reporting Customizable Multiple Tests Single Environment Field Tests & Lab-based based tests all in One Environment! Echo Test creation Intelligent, real-time One-step recording of test scores used for customized test creation! Page 20
Wireless Multimedia Analysis Module Feature Summary Objective, repeatable and reliable multimedia analysis Quality video and audio algorithm MOS with cause analysis (Blurring, Blocking, Noise, etc.) Lip-Sync measurements with full reference Enhanced customizable, automated software management through Spirent Test Manager Open & flexible architecture - with ability to create and change test cases with ease & efficiency with APEX platform Bearer-independent solution works with EV-DO RevA, MediaFLO, Mobile TV, VCAST etc. Page 21
Summary Delivering Value to Customers & End-users Multimedia technology enablers are already here Success will be measured by the incremental revenue they generate The end user s positive experience is critical to attracting new customers and keeping repeat customers Spirent provides the necessary solutions to verify a positive experience throughout the multimedia application testing process Spirent: Helping you get it right the first time! Page 22
Page 23 Thank You!