Leveraging High Definition and Efficient IP Networking WHITE PAPER
Introduction Already dominant in traditional applications such as video conferencing and TV broadcasting, H.264 Advanced Video Coding (AVC) standards are expanding into applications that benefit from the use of high-definition (HD) content that can be stored and networked efficiently. These applications typically take advantage of today s highly integrated semiconductor solutions to minimize costs while achieving high-speed H.264 compression and decompression. The emerging H.264 applications include: Medical operating rooms, where video from HD cameras is streamed to other locations and/or stored for archival purposes Universities and other educational institutions offering distance learning as well as an effective way for students to replay lectures Rural telecommunication providers that want to broadcast content to small communities Video Compression History The H.264 AVC standards have their origins in the standards developed by the Motion Picture Experts Group (MPEG). MPEG is a part of the International Organization for Standardization/International Electrotechnical Commission (ISO/IEC), an organization that creates standards for audio and video compression. The MPEG-2 standard completed in the late 1990s is now widely used as the digital television format for terrestrial over-the-air broadcast, cable and satellite systems. After the success of MPEG-2, the ITU-T Video Coding experts group collaborated with ISO/IEC MPEG to develop a new audio and video compression standard with aggressive goals: Compression 2 to 3 times more efficient than MPEG- 2 Support for a wide range of applications from broadcast TV to mobile phones using a single compression standard Support these applications with the ability to scale from mobile phone data networks to full HD applications across a wide range of bandwidths Ensure efficient streaming using Internet Protocol (IP), allowing HD-quality video to be streamed over the Internet efficiently Make sure that the cost to implement the standard is not substantially higher than that of MPEG-2 so that low-cost products can proliferate in the market Through hard work and much deliberation, the group released its first draft of the standard known as H.264 AVC (also known as MPEG-4 Part 10) in 2003. This standard achieves a 50 percent efficiency improvement over MPEG-2 for full HD video. H.264 supports a wide variety of profiles as well as narrowing the compatibility requirements to target each specific application. Work on the standard continues through the addition of new profiles as new technology becomes available. In the meantime, H.264 is mandatory for Blu-Ray and HD-DVD standards. Many broadcasters are using H.264 in their high-end broadcast equipment. Note that H.264/AVC is a video compression standard developed by the ITU-T Video Coding Experts Group (VCEG) together with the ISO/IEC Moving Picture Experts Group (MPEG), and it was the product of a partnership effort known as the Joint Video Team (JVT). The ITU-T H.264 standard and the ISO/IEC MPEG-4 AVC standard (formally, ISO/IEC 14496-10 - MPEG-4 Part 10, Advanced Video Coding) are jointly maintained so that they have identical technical content. Emerging Markets In addition to high coding efficiency, H.264 AVC supports the ability to convert full HD video to IP digital video using the standard s network adaptation layer (NAL). This capability suits the current market trend to converge three kinds of applications: data, voice and video over IP. This powerful combination fuels growth in a number of emerging market areas, with H.264 AVC supporting applications ranging from mobile devices running low-resolution video to full HDTV. This white paper focuses on emerging applications that highlight the growing need for HD. Medical Applications The merits of capturing HD video in operating rooms is clear to everyone involved. The detail and clarity of HD have many uses for the surgeons completing the procedure as well as for many others both within and outside of the operating room. Given the benefits, it might seem that every operating room would already be equipped with HD cameras for even the most routine procedures. Unfortunately, HD video coverage is far from routine. The challenge lies primarily in the difficulty of storage and distribution of video for the system. Early cameras allowed for storage of video in DVD media. At full HD, however, conventional DVDs store only about 20 minutes of video. Blu-ray disks increase storage capacity by about 5X still barely adequate for many surgical procedures. Additionally, storing the video to disk for archival purposes is only part of the application Page 1
requirement. What about streaming the video to students and others outside the operating room? The huge volume of HD video overwhelms most networks. Adding H.264 encoding to the system easily overcomes these challenges. The H.264 compression enables a single-sided Blu-ray disk to store 4.5 hours of HD video. Moreover, since H.264 offers the ability to stream the video over an IP network, users can transfer the HD content to a remote video server thus eliminating the need for DVD or Blu-ray media storage in the operating room. With these capabilities, hospitals now have the ability to stream the video real-time to other locations so others can observe the operation. Additionally, large video servers can store and organize all of the video content. In the US, university hospitals have been the first to adopt such systems into their operating rooms. The HD video can be encoded and streamed real-time to numerous locations and shared by many people at the same time the ideal way for students and other surgeons to quickly learn about new techniques as they are introduced. Even if the observers do not watch live, the ability to replay these videos from a server eases distribution compared to the old practice of copying optical disks. As H.264 encoding and video distribution mature, this capability could be easily introduced into all hospitals. Looking into the future, this capability could be expanded to smaller medical clinics as well as home health care. Ill or disabled people could have a small video camera equipped with an H.264 encoder to stream information to doctors. This system could save both time and money, as well as providing higher levels of service for patients. Education Distance Learning The use of distance learning continues to grow in universities throughout the US and even in K-12 school districts. The primary reason for this growth is convenience, followed by cost as educational institutions try to do as much as they can with limited budgets. Distance learning can be implemented using two different models. In the synchronous model, students and teachers can interact in real time. In the asynchronous model, students accesses the content from a remote location at their own convenience. In the synchronous case, typically the professor is actively teaching the class. Cameras capture the video, which is encoded using H.264 and then streamed to a number of remote locations. Whether the video is HD or SD (standard definition), both benefit from H.264 compression since the speed of the links between the main and remote sites may vary. For lower-speed links, the system can be tuned to display the video at lower resolution levels to maintain a real-time connection. Because such connections provide opportunities for remote students to participate in class discussion, students and teachers often prefer to give up image resolution in favor of low latency. In the asynchronous case, video as well as other content can be stored on a server and then typically accessed over the Internet. Students get the greatest amount of flexibility in this situation by being able to take classes at any time and most likely within their own homes. Internet HD Camera H.264 Encoder Video Server PC with HD Monitor Remote HD Monitor Conference Room Remote Surgeon Figure 1 Operating Room Page 2
HD Camera, HD Monitor and Microphone Main Classroom H.264 Encoder / Decoder Video Server / Bridge Internet H.264 Encoder / Decoder H.264 Encoder / Decoder HD Camera, HD Monitor and Microphone HD Camera, HD Monitor and Microphone Remote Classroom Figure 2 Distance Learning Remote Classroom Even for students meeting in traditional classrooms, universities can encode lectures and make then available for the students to access at any time. Access to all lectures can help students brush up on concepts that they might have missed in their notes. In most cases, a university starts by purchasing a single H.264 encoder and video camera for use within a classroom. If they are pleased with the results, university administrators expand the number of video systems. As the cost of H.264 decreases, the systems become much more attractive for schools to set up the encoder and video camera in each classroom. Broadcast Applications Many broadcasters in the cable and satellite industries started with MPEG-2 encoding for standard-definition TV, and MPEG-2 is still the most dominant standard today. Industry trends require broadcast service providers to support ever greater bandwidth, however. As viewers watch more HD content, the bandwidth demand for a single channel can be roughly 4X the bandwidth of an SD channel. In addition to the migration to HDTV, a number of other services promised by the broadcasting industry will push the envelope on network bandwidth requirements. These services include video on demand, 3DTV, and gaming. To respond to these new requirements, broadcasters look to H.264 AVC as a workable option for expanding network bandwidth. Early users of H.264 tended to be telecom providers rolling out IPTV services. H.264 helped telecom providers make best use of the limited bandwidth in existing broadband networks. As the number of IPTV vendors grows, customers will have a fourth option in addition to the existing providers: terrestrial (over-the-air broadcast), cable and satellite. As HDTV became commonplace, satellite broadcasters were second to the H.264 migration because they need to transmit a large amount of content to their satellites. There is no question that the use of H.264 AVC will continue to migrate to the larger broadcast environments. In addition to the major markets, there is a good possibility of growth for rural Telcos using IPTV or satellite options. These smaller networks will demand more H.264 encoders but at more cost- Page 3
effective price points. Some of these smaller markets see promise with video on demand (VOD) services as well as the broadcast of private and regional contents. H.264 Encoder Implementation Trends A system vendor has several choices when implementing an H.264 encoder solution. The focus for these applications is to encode full HD (1920x1080p) at 60 frames per second. Numerous tradeoffs need to be made between performance, product costs and development costs. When developing an H.264 encoder for the emerging applications described here, developers have three basic options: 1. Encoder software running on a general-purpose system platform 2. FPGAs 3. SoCs Software Solution Software solutions typically run on a system platform with some kind of real-time OS, Linux or Windows. These solutions are among the first to be developed when a standard is going through the draft stage. The key advantage is the ability to quickly make changes to the solution so it can be customized to meet particular requirements. Options for this approach range from developing the encoding software to licensing code from vendors that support the application-specific customization requirements. The cost of this approach is moderate compared to FPGA-based or SoC-based solutions, but performance for the software solution tends to be on the lower end. In fact, typically it is difficult to get a software solution to encode in real time. The time required to encode, decode and display HD video (latency) is usually significant. To support full HD H.264, therefore, the software solutions typically run in the background to compress the HD video and store the content to a server. An additional issue is that software implementations tend to be housed in fairly large boxes, similar in size to small PCs. FPGA Solution FPGAs offer better performance than software-based solutions and thus support real-time applications. FPGA-based systems can become highly differentiated over other offerings since the system vendor can make changes to the encoder capabilities. This ability to change the hardware at any time is useful if customers request new features or changes are made in the standard (although the H.264 standard has been stable for some time). Products can be upgraded in the field by reprogramming the FPGAs. Such products are typically available in the early stages of standards development. Due to the high product costs associated with FPGAs, less-price-sensitive markets such as high-end broadcasting appear to be a reasonable match. In addition to the FPGA costs, the product costs are high due to the expense of acquiring the skills and resources necessary to customize the H.264 encoding algorithm. SoC Solution When the standard is stable as long as H.264 has been, semiconductor vendors develop highly integrated SoCs that typically incorporate a high-performance processor and functionality specific to the application. In the case of H.264 encoding solutions, vendors have integrated solutions typically around ARM cores that do the heavy lifting required for the encoding functions. Some SoCs have enough on-chip integration to handle the complete requirements of the application. Examples include set-top boxes (STBs), STBs with integrated DVR, and camcorders. SoCs offer numerous benefits. With the high level of integration comes the best performance for encoding along with the smallest form factor, lowest power consumption and lowest costs. Even if a vendor s early designs incorporated software solutions or FPGAs, the Feature Software FPGA SoC Performance in Encoding Low Good Excellent Form Factor Size Largest Moderate Smallest Cost to Implement Moderate Moderate Low Flexibility in Feature Changes Flexible Flexible Less Flexibility Power Consumption High Moderate Low Table 2 Implementation Tradeoffs of H.264 AVC in Software, FPGA and SoC Page 4
H.264 Encoder / Decoder The Fujitsu MB86H46 FPGA HD Camera Figure 3 The Fujitsu MB86H46 Single Chip Solution SoC is a natural migration to enable more products to proliferate in the market. The primary tradeoff of an SoC fixed functionality, although firmware changes can be used to upgrade functions that run on the on-chip processor. Implementation Summary Since the H.264 AVC first draft was completed in 2003, ample time has passed for chip vendors to develop a wide variety of SoCs for handling H.264 encoding in full HD. Given this level of maturity in the market, a software-based H.264 encoder for full HD quality can hardly compete. Similarly, FPGA-based solutions have a chance only in the most price-insensitive applications. If the currently available SoCs offer most but not all of the features desired for an application, then combining an SoC with a small FPGA may offer a way to implement all the functionality a solution that leverages the lower cost point of the SoC while allowing for customization in the FPGA. The Fujitsu MB86H46/MB86H56 is among the first SoC offerings that offer full HD (1920x1080x60p) H.264 AVC encoding and decoding in real time. The product supports the lowest power consumption in its class at just 600 mw for full HD encoding. Additionally, the device s footprint is the smallest in the industry at only 15mm x 15mm including all of the necessary memory in-package. The combination of low power consumption and small footprint make the product ideal for small battery-powered devices, yet the device delivers performance and functionality suitable for larger professional encoders. Conclusion The tremendous growth of the Internet has fueled the need to converge services such as voice and video onto IP networks. With a mature H.264 AVC standard as well as wide availability of low-cost SoCs for encoding, new applications are emerging such as medical operating rooms, distance learning and smaller broadcasters primarily supporting IPTV. For More Information For For more moreinformation informationon onthe thefujitsu Fujitsuvideo videoprocessing ICs, ICs, please pleasego goto to http://us.fujitsu.com/micro/h264 http://fujitsu-multimedia.com or address e-mail Email: to multimedia_info@fme.fujitsu.com inquiry@fma.fujitsu.com FUJITSU SEMICONDUCTOR MICROELECTRONICS EUROPE AMERICA, INC. Corporate Corporate Headquarters Headquarters Pittlerstrasse 1250 East Arques 47, D-63225, Avenue, M/S Langen, 333, Germany Sunnyvale, California 94085-5401 Tel: +49 (800) (0)6103 866-8608 690-0 Fax: Fax: (408) +49 737-5999 (0)6103 690 122 E-mail: info@fme.fujitsu.com inquiry@fma.fujitsu.com Web Web Site: Site: http://emea.fujitsu.com/semiconductor http://us.fujitsu.com/micro 8 2010 2010 Fujitsu Fujitsu Microelectronics Semiconductor America, EuropeInc. GmbH. All rights All rights reserved. reserved. All All company company and and product product names names are trademarks are trademarks or or registered registered trademarks trademarks of their of their respective respective owners. owners. Printed EPS-WP-21376-05/2010. in U.S.A. EPS-WP-21376-05/2010.