VISUAL QUALITY ASSESSMENT CHALLENGES FOR ARCHITECTURE DESIGN EXPLORATIONS. Wen-Fu Kao and Durgaprasad Bilagi. Intel Corporation Folsom, CA 95630

Transcription

1 Proceedings of Seventh International Workshop on Video Processing and Quality Metrics for Consumer Electronics January 30-February 1, 2013, Scottsdale, Arizona VISUAL QUALITY ASSESSMENT CHALLENGES FOR ARCHITECTURE DESIGN EXPLORATIONS Wen-Fu Kao and Durgaprasad Bilagi Intel Corporation Folsom, CA ABSTRACT Typical computer architecture design primarily addresses performance and power tradeoff. In modern GPU design, more and more video and imaging features are integrated as part of the GPU. Media architecture design requires image/video quality as the third domain and the tradeoff among them. Assessment of video quality plays a crucial role in design space exploration. Although many popular evaluation methods and various quality metrics have been proposed, conducting cross-platform comparison as well as architecture optimization and tuning continues to pose great challenge during product design. We present realworld case studies in designing Intel processor graphics architecture on how we apply known quality metrics in the experiments and studies to form appropriate interpretation and guide us on the optimization on the actual silicon. 1. INTRODUCTION Video quality assessment is fundamental to evaluate any codec or imaging algorithm design. Many research works investigated and proposed subjective and objective metrics and methodologies in order to provide better understanding about the visual quality. In typical computer architecture design, substantial amount of effort spent on the tradeoff of performance and power. When more and more video and imaging features are integrated as part of the graphics processor, it is essential to consider performance, power, and visual quality during the design exploration to achieve the optimal decision under the design constraints. Typically, the visual quality evaluation is completed in early algorithmic exploration, and the performance/power tradeoff is evaluated during the design implementation. However, the need to bring all three elements together to optimize the design is crucial to deliver the right product. There are many challenges to overcome for a better and fair assessment in order to achieve this goal. In this paper, we visit these challenges through real case studies we conducted in the lab to understand the implications of these challenges when designing processor graphics. 2. RELATED WORKS Visual quality evaluation is a highly complex problem due to two reasons as mentioned in [3]. One is the video system that includes capture & display hardware, converters, networks, codecs all affect the quality. The second one is solutions that can match how human visual system perceives video & its quality. Many papers describe metrics to assess video quality. [1][2] reviews many metrics and classify them as data metrics, picture metrics, packet metrics, and hybrid metrics. They can also be classified as No-Reference (NR), Reduced-Reference (RR), and Full-Reference (FR). The FR metric evaluates the content respect to the original reference. The NR metrics only analyze the test content without any reference available. The RR uses an alternative channel between senders and receivers to transmit the parameters of the delivered content in a reduced way or use watermarking schemes. In [4], it is discussed that traditional data metrics such as PSNR, SSIM, MSE, MOS etc are still widely used, but significant improvements in prediction performance and versatility can only be achieved by perceptual quality metrics. In our studies, these difficulties directly challenge us on how to better combine various metrics and approaches in order to integrate perceptual quality experience and quantifiable target. 3. OVERVIEW OF INTEL PGFX First, Figure 1 shows Intel s latest processor graphics (pgfx) design, code name Ivy Bridge. It depicts a highlevel function diagram. As the graphics is part of the CPU/GPU integration on the same die, there is an interface to communicate with shared last-level cache (LLC) and memory. Media functions include a highperformance multi-format codec engine, and a media sampler that handles heavy computation tasks and high quality imaging enhancement. The array of execution 27 VPQM2013

2 units (EU) is the primary interface to control and work with media sampler. All media tasks can benefit from local L3 cache and the shared LLC. Another key component in design exploration is to understand other competitive products on the market and their design choice. The information greatly contributes to shape the design strategy. Although there are many software solutions that provide excellent quality, it might not be feasible for realworld application if the computation cost is too high. For example, HD video calling application on mobile devices would benefit from using power efficient hardware solution. However, simply implementing the solution directly in the hardware is often not enough. It requires careful design to balance between performance and power. Having the right quality assessment is the key to understand the necessary quality level under the design constraints. Enabling these new applications also guide us on deciding between fixed function unit design and generic shader unit computation. provide improved video and imaging experience on the PC and mobile devices. More and more users directly watch videos and process images on these devices. Enabling these features come with a cost. Many of these features are compute-intensive operations. For high performance PC with power source, turning on these features is not a problem. When PC is operated on battery or users are using personal mobile devices that have very limited thermal budget, it is not trivial to decide when to turn these features on. We explore each feature s performance and power efficiency under various configurations from high performance product to ultra low power product in order to define the best possible combination to enable or disable features. The screenshots in Figure 2 are both Intel pgfx and competition product operate with AC power source. The second screenshots in Figure 3 are both machines operate on battery. Apparently, the competition product on the right hand side will need to choose to disable important image enhancement features such as noise reduction in exchange for smooth video playback under the thermal and power budget. Consequently, the image quality will need to be compromised. Figure 2. Screenshots on AC power (Right: Competition, Left:Intel) Figure 1. High-level function diagram of Intel pgfx 4. CASE STUDIES We present several case studies to help understand how we explore the design space and optimize HW/SW solution for multiple market segments. Objective metrics such as SSIM and PSNR are still our primary metrics whenever we can measure and subjective quality assessment is still required in many of the studies when measuring objective metric is not feasible. Figure 3. Screenshots on battery (Right: Competition, Left: Intel) 4.1. Image enhancement quality on battery There are many image enhancement features included in Intel s pgfx such as noise reduction, color enhancement, skin tone correction, etc. These features are targeted to 4.2. H.264 encode tuning 28

3 This is a major task that is fundamentally challenging in delivering a good hardware encode solution. In early phase of the codec design, we greatly rely on C-based model to implement the algorithm and evaluate the quality. The computation cost in the C model is generally based on analytical model. The actual performance and power optimization will need to be conducted after the silicon comes back from the fab. There are many different aspects and goals in the optimization and tuning effort. We are choosing one of the scenarios as an example in this paper. The popular x264 is currently one of the high quality H.264 encoder available today. Generally, software encoder has the greatest flexibility to run complicated algorithm because the CPU is the most general-purpose processor. HW encoder has its design limitation in order to be small, fast, and power efficient. Studying the quality difference between x264 and Intel hardware encoder can still guide us on the areas where we should improve such as different rate control policy, different motion estimation behavior and mode decisions based on the contents, etc. In Figure 4, the chart from MSU s 2012 video codec comparison report shows that using a laptop PC with weaker CPU and basic integrated processor graphics, Intel s hardware encoder is better in terms of speed and quality tradeoff than best pure software encoder x264 at very high-speed encoding [5]. The green dot at the lower right corner represents baseline 1.0 from x264. The red dot at the upper left corner indicates that Intel hardware encoder takes about 5% less bits to achieve the same quality at nearly 14% faster speed. In order to conduct such quality study, immediately, we face the miscorrelation between highlevel C model and actual silicon driver implementation. Therefore, the quality obtained from the actual system does not always represent the real quality from our encoder design. To determine where to spend the tuning effort, we have to investigate what causes miscorrelation. If the quality gap truly comes from the algorithm design, it is reasonable to optimize the algorithm. However, if the goal of the tuning is to optimize performance, power, and quality, we can only rely on the real system for true performance and power measurement.. The tuning is time consuming since separate effort to validate and correlate between C model and the real driver needs to be made in parallel. The next challenge is what configurations of the encoder to use. Ideally, comparing the quality difference under total controlled environment and exact matching configuration gives us the best understanding on how the encode behavior is. However, even both x264 and Intel hardware encoder allow a wide range of parameters to reconfigure, it is not possible to have a perfect match naturally. We generally need to settle with best possible match. In addition, we have the need to compare the two encoders based on their default and stock settings in order to understand general users experience. Another challenge is to bring user experience to the tuning effort. PSNR continues to be used heavily during evaluation. The norm is also shifting to rely more on SSIM or similar metric to better reflect the visual quality. To evaluate a tuning option, we cannot just target to obtain higher PSNR or SSIM, We need to be able to realize the tuning through subjective user experience in addition to quantify with the objective metric. Courtesy: MSU Graphics & Media Lab video Group Figure 4. Intel HW encoder vs. x264 performance/quality tradeoff 4.3. Video calling In this case study, we face a different challenge. We are evaluating better hardware encode solution directly when customers have taken current hardware encoder to provide their own solution. Microsoft s Skype and Apple s FaceTime are two popular video calling applications. In the recent update, Apple has enabled hardware acceleration in their FaceTime HD on the Macs running on the latest Intel s pgfx. Comparing the two totally different applications with different approaches is difficult but it is essential to understand how other people utilize our hardware. We have to rely on subjective quality assessment with several indirect metrics such as network bandwidth, CPU and GPU utilization, and package and platform power consumption in order to form an appropriate study. There are additional considerations when we evaluate potential optimizations after we see the result from third-party applications. A work-in-progress option is compared with a software encoder. When we look at the screenshots in Figure 5 and Figure 6, right hand side is Intel s hardware encoder from reference C model and left hand side is a software encoder of interest. The reference C model tells us that our hardware encoder did a fairly good job in preserving some of the detail when we inspected these frames of interest. However, we cannot draw conclusion too early when we do not know what the quality would be on the real silicon. Another question is 29

4 the difference may be spotted when performing frame-byframe inspection, but how often users would spot this and be bothered during real-time video calling. Video calling is unique because it is not just about tuning the encode quality. It is a platform solution that requires us to carefully balance the performance, power, and quality. While market trend pushes further to enable high quality HD 1080p video calling, the market also pushes longer battery life at the same time. To find the balance of reasonable bit rate, better bit-rate control approach, and low power encode solution is the key to deliver a practical HD video calling platform solution. difference. In both Figure 7 and Figure 8, the right-most screenshot is Intel pgfx, and the other two are other hardware encoders. From the zoom-in screenshots, users can easily observe a better visual quality from Intel s hardware encoder. The chart in Figure 9 also shows superior performance advantage over other hardware solutions that Intel s hardware encoder delivers significantly faster than real-time transcode speed. Figure 6. Screenshots of HW1 (left), HW2 (middle), and Intel pgfx (right) Figure 7. Zoome-in screenshots of HW1 (left), HW2 (middle), and Intel pgfx (right) Figure 5. Exmaple screenshots (Left: SW, Right: Intel) Figure 6. Exmaple screenshots (Left: SW, Right: Intel) 4.4. Cross-platform quality comparison Figure 9. Transcode speed comparison A popular type of study that is often seen in technical review is product review. Most companies also conduct their own competitive studies in order to better understand and shape strategy. In one of our studies, we target other hardware encoder solution to compare with Intel pgfx. The primary challenge is each hardware encoder solution is tailor made and not all of them have the same level of freedom to reconfigure. The best-case scenario is finding a cross-platform application that supports many different hardware venders so we can minimize the difference. Nevertheless, each hardware encoder is based on its unique design and philosophy on balancing between fixed function unit and generic compute shader, we are still comparing total different configurations even using the same application. We again rely more on subjective quality assessment and inspect the video frame to understand the 5. FUTURE WORKS We presented quality assessment challenges during the architecture exploration. The gap between C model reference and real silicon driver itself will be a huge topic in the validation area. However, integrating C model directly into cycle-accurate architecture simulator with real-world workload will greatly help early architecture exploration. It can provide a better early indicator the tradeoff among performance, power, and quality. Next challenge is image enhancement feature evaluation such as noise reduction, image stabilization, frame rate conversion, skin tone correction, etc. These features aim to improve the visual quality of the original image. Existing metrics are designed to evaluate the match of the original. New approach and metric are needed to assess 30

5 the quality. Finally, addressing the real user visual experience is equally important to the objective metrics. Integrating subjective assessment into objective metrics to form a realistic benchmark to represent user experience will ultimately improve and guide media architecture design. 6. REFERENCES [1] R.Serral-Graci`a et al, An Overview of Quality of Experience Measurement - Challenges for Video Applications in IP Networks [2] Stefan Winkler, Video Quality Measurement Standards Current Status and Trends, ICICS 2009 [3] S. Winkler: Digital Video Quality Vision Models and Metrics. John Wiley & Sons, [4] Stefan Winkler, Video quality and beyond. Proc. European Signal Processing Conference, Poznan, Poland, Sept. 3-7, 2007 [5] Dmitriy Vatolin et al, MPEG-4 Video Codecs Comparison 31