OptoFidelity Measurement Services High-end smartphone measurement report Q1/2013

Transcription

1 OptoFidelity Measurement Services High-end smartphone measurement report Q1/2013 OptoFidelity Oy

2 Contents 1 Introduction Measurement devices OptoFidelity Touch & Test PDT OptoFidelity WatchDog with Multimodal option OptoFidelity Scroll Performance Analyzer OptoFidelity Frame Rate Meter Devices under test Test case description Touch performance test Latency measurements Viewfinder launch time Shot to shot time Calendar launch time Calculator launch time Gallery open Music player launch time Music song start (visual) Music song start (audio) Boot up time Transition smoothness Music player scroll Video playback performance YouTube video playback Results Touch performance tests Apple iphone Nokia Lumia Samsung Galaxy SIII Latency measurements Transition smoothness Video playback performance Conclusion Page 2 of 10

3 1 Introduction This document shows the results of OptoFidelity Measurement Services for three mobile devices. Document includes introduction of the measurement devices which were utilized for device testing, a list of devices under test and a presentation of the test results. All measurements have been carried out non intrusively for three off the shelf products. The only modification to products was the addition of content such as music files to device memory. 2 Measurement devices This chapter presents the measurement devices utilized in this study. Products are standard OptoFidelity products without modification. 2.1 OptoFidelity Touch & Test PDT OptoFidelity Touch & Test is a test platform for touch device testing. PDT stands for panel performance device level testing. OptoFidelity Touch & Test measures the touch panel accuracy. Tester includes a robot with artificial finger. Tester compares the coordinates of robot to the coordinates captured from touch device. 2.2 OptoFidelity WatchDog with Multimodal option OptoFidelity WatchDog is a camera based measurement tool for measuring latencies from user interaction to visual feedback. Multimodal option adds capability to measure also tactile and audio latencies. Figure 1: OptoFidelity WatchDog mechanics 2.3 OptoFidelity Scroll Performance Analyzer OptoFidelity Scroll Performance Analyzer is a camera based measurement tool for detecting content updating frequency from display. To detect changes content should be moving on display. Following image illustrates the content update from frame to frame. Page 3 of 10

4 Figure 2: Scroll performance definition Scroll Performance Meter utilizes the WatchDog system hardware. 2.4 OptoFidelity Frame Rate Meter OptoFidelity Frame Rate Meter is a stand alone device for detecting frame changes from display. Frame change detection is based on the synchronization markers of test video. Marker insertion can be done with OptoFidelity Test video generator or other video editing tools. Figure 3: OptoFidelity Frame rate meter 3 Devices under test Tested devices were selected to present the state of the art products of different operating systems. Devices under test (DUT) were Apple iphone 5, Nokia Lumia 920 and Samsung Galaxy SIII. Apple iphone 5 has the operating system ios 6, Nokia Lumia has Windows 8 and Galaxy SIII has Android v4.0.4 (Ice Cream Sandwich). 4 Test case description This chapter presents the test case descriptions for this study. Test cases were designed to be as comparable as possible. 4.1 Touch performance test Touch performance measurable was touch accuracy. Measurement was carried out as point to point measurement. Robots artificial finger touched panel surface every 2 mm. Test covered the whole display area from left top corner to right bottom corner with 2 mm spacing. Page 4 of 10

5 4.2 Latency measurements Latency measurements included multiple test cases, and were carried out with OptoFidelity WatchDog with Multimodal option. In this study two visual latencies were studied. First one was reaction time, which is the time from user interaction to transition start. Second one was response time, which is the time from user interaction to complete the transition Viewfinder launch time DUT was placed so that the camera was pointed towards the same test target which was positioned 20 cm from DUT. Viewfinder was launched from DUT s menu view. Response time is the time when camera view and finder brackets are stabilized Shot to shot time DUT was placed so that the camera was pointed towards the same test target which was positioned 20 cm from DUT. Image capture was triggered from touch screen. Response time is the time after image is reviewed and camera view and finder brackets are stabilized on the screen Calendar launch time Calendar application was launched from DUT s menu view by touching the calendar icon Calculator launch time Calculator application was launched from DUT s menu view by touching the calculator icon Gallery open Gallery application was launched from DUT s menu view by touching the gallery icon. 100 pictures were taken with DUT s own camera to produce more realistic use case Music player launch time Music player application was launched from DUT s menu view by touching the music player icon. 500 songs were loaded to DUT to produce more realistic use case Music song start (visual) Audio playback was started by pressing audio file name from the music library. Same audio file was used in each DUT. This test case studied the display response time Music song start (audio) Audio playback was started by pressing the audio file name from the music library. Same audio file was used in each DUT. This test case studied the time from user interaction to time when signal was detected from the DUT s speakers Boot up time DUT s were started from mechanical key. Reaction time presents the first visual feedback latency and response time presents time to the pin code dial view or network connection view. All devices have their own boot up behavior. Page 5 of 10

6 4.3 Transition smoothness Transition smoothness was tested in this study with single test case Music player scroll Music player song list was scrolled from top to bottom. The updating frequency of icons and song names was captured. 4.4 Video playback performance Video playback performance was tested with a single test case by using OptoFidelity Frame Rate Meter YouTube video playback Video which contained synchronization markers was uploaded to YouTube video streaming service. Video playback frame rate was captured from display. 5 Results This chapter shows the results of the all test cases. 5.1 Touch performance tests Results of the touch performance tests are presented with two images. First image is an overview plot, which shows the point to point measurement of all points. Green color indicates that detected coordinate is within 1 mm of actual coordinates. Red color indicates that point distance between robot coordinate and panel coordinate is more than 1 mm. The second image is a scatter plot, which presents the variation of all points. Perfect result point position would be in the origin Apple iphone 5 Results for Apple iphone 5 in Figure 4 show that touches on the edge area of panel are ignored. Figure 4: Touch accuracy results of Apple iphone 5 On the active area Apple iphone performs well. Only one single point is just outside of target circle. Page 6 of 10

7 5.1.2 Nokia Lumia 920 Figure 5 shows the touch accuracy results of Nokia Lumia 920. Figure 5: Touch accuracy results of Nokia Lumia 920 Nokia Lumia 920 detected multiple points outside the 1 mm target cycle and variation of detected points is larger than in other tested devices Samsung Galaxy SIII Figure 6 shows the touch accuracy results for Samsung Galaxy SIII. Figure 6: Touch accuracy results of Samsung Galaxy SIII Samsung Galaxy SIII gives the best touch panel accuracy of measured samples. All detected coordinates are within 1 mm from actual location. 5.2 Latency measurements The results of system latencies are presented in two images. First one presents the reaction time and second one presents the respond time. In both graphs the smaller the value is, the better it is. Page 7 of 10

8 Time [s] 0,9 0,8 0,7 0,6 0,5 0,4 0,3 0,2 0,1 0 Reaction times Apple iphone 5 Samsung Galaxy S III Nokia Lumia 920 Figure 7: Reaction time chart. Lower value is better. 3,5 3 Response times 2,5 Time [s] 2 1,5 1 0,5 0 Apple iphone 5 Samsung Galaxy S III Nokia Lumia 920 Figure 8: Response time chart. Smaller value is better. Page 8 of 10

9 5.3 Transition smoothness All devices in this study have 60 fps content refresh rate. Optimal value for average would be close to 60 fps, and lower fps value means freezes during playback. Lower standard deviation is better. Figures 9 and 10 show the transition smoothness results. Average 61 Framerate [fps] Apple iphone 5 Samsung Galaxy S III Nokia Lumia 920 Figure 9: Transition performance chart. A value closer to 60 fps is better. 12 Standard deviation 11,5 Frame rate [fps] 11 10,5 10 9,5 9 Apple iphone 5 Samsung Galaxy S III Nokia Lumia 920 Figure 10: Standard deviation of transition frame rate. Lower value is better. Samsung playback freezes few times during scrolling. Samsung was the only device which updated music album icon in the fly, which may cause the frame rate drops during scroll measurement. Page 9 of 10

10 5.4 Video playback performance Source video frame rate is 30 fps. Optimal average value would be 30 fps and a value closer to 30 fps is better. Average Framerate [fps] 32 31, , , ,5 28 Apple iphone 5 Samsung Galaxy S III Nokia Lumia 920 Figure 11: Video playback frame rate. A value closer to 30 fps is better. 6 Conclusion Comparison to earlier studies shows that all devices present the state of the art of nowadays mobile industry. However, product characters can be ranked on base of the test results. Samsung Galaxy SIII has the best touch accuracy, completing the test without errors. Apple iphone 5 is the best in latency and video playback measurements. Nokia Lumia 920 has the best transition playback performance in music library scroll. Page 10 of 10