Online Webinar. June 16, Intel SDK for opencl applications. Intel Media SDK

Transcription

1 Online Webinar June 16, 2016 Intel Media SDK Intel SDK for opencl applications

2 Welcome! Meet our Experts Jeff McAllister Media Software Senior Technical Consulting Engineer Developer Products Division Robert Ioffe OpenCL Software Technical Consulting Engineer Developer Products Division Intel Software & Services Group 2 2

3 What We ll Cover Today Introduction Intel Processor Graphics -the Other Side of the Chip Programming Model Examples Intel OpenCL & Media SDKs Better Together Q&A 3 3

4 Take advantage of advanced media accelerators, graphics processing & programmable graphics

5 Intel CPUs + Intel Processor Graphics (GPUs): A Winning Combination Intel HD Graphics 6 th Gen Intel Core Processor If you re only targeting CPUs, you re not fully utilizing the chip See Technical Specifications for hardware requirements & other details. Not shown Intel Xeon Processor Graphics 5

6 Hardware Overview Graphics Technology Highlights Execution Units (EUs) = general purpose cores EUs, samplers, caches, etc. in slices Fixed function is in unslice edram adds cache, increases bandwidth Naming Convention Intel HD Graphics Intel Iris Graphics Intel Iris Pro Graphics 24 EUs 1x FF 48 EUs 2x FF edram 72 EUs 2x FF edram Other Names GT2 4+2 GT3 2+3e GT3e,GT4e 4+4e Summary Good Better Best Fixed Function (VDBox, VEBox) GT2 GT4 GT3 6 Look for Intel Processor Graphics at ark.intel.com 6

7 Intel SDK for OpenCL Applications & Intel Media SDK: Better together for video apps Codecs AVC/H264, HEVC/H265, MPEG2, MJPEG Filters Resize/crop, color conversion, deinterlace, composition/blend, FRC, telecine/interlace reversal Increasing GPGPU + fixed function = high performance custom pipelines 7 7

8 Takeaways Intel Media SDK & Intel SDK for OpenCL Applications used together allows programmers to exploit the full computational capabilities of Intel processors by utilizing CPU, GPU & fixed function hardware. Intel offers drivers, runtimes/compilers & a suite of tools allowing video application developers to create, build, debug, analyze & optimize. Intel tools & hardware help you create exciting new applications & usages for image/video processing, virtual & augmented reality, artificial intelligence, robotics, & machine learning (including deep learning). 8 8

9 Get the most out of Intel platforms with Intel Software Development Tools

10 Intel Software Development Tools Technical Computing Embedded Systems & IoT (CPU) Heterogeneous Tools (GPU) New Interface (future) Intel RealSense SDK Intel System Studio Intel IoT Developer Kit Intel Media SDK Intel SDK for OpenCL Applications 10 10

11 Developer Hardware Access GPU Components EUs Media Fixed Function VDbox/MFX VEbox/VQE Media Accelerator Model (GPU) Intel Media Server Studio Intel SDK for OpenCL Applications Intel Media SDK Core Core Ring System Agent LLC (edram) Core CPU Components LLC Core LLC Traditional CPU Model Intel Parallel Studio Intel System Studio CPU Compilers, performance libraries, etc

12 Media Accelerator vs. Traditional Model Comparison Media Application Decode Frame Processing Encode App MSDK Decode DXVA/ VAAPI MSDK VPP OpenCL GPU MSDK Encode VAAPI, OpenCL (+ VME extension) Library CPU or App CPU Driver GPU Driver Driver Driver CPU (Traditional Model) VDBOX EUs, VEBOX EUs, VDBOX GPU (Accelerator Model, includes CPU) 12 12

13 Main loop Initialization OpenCL & Media SDK Programming Models Application (written C, C++, ) Kernels to Accelerate (OpenCL C) Parameters (from header) Parameters (in & out) Parameters kernel void dot ( ) { int tid = get_global_id(0); c[tid] = dot(a[tid], b[tid]); } init init init OpenCL API Khronos ICD stream Decode frame VPP frame Encode stream Compiler Front End (Clang) SPIR Back End (LLVM) Runtime Intel Media SDK GPU Device CPU Device Coprocessor Device Intel SDK for OpenCL Applications 13

14 Intel Media SDK Value Propositions Acceleration: Utilize full capabilities of Intel processors. Easy optimizations: Simple high level asynchronous API no need for specialized developers to code to the metal. Multi-platform/OS support: Utilize Intel Xeon and Core in Windows* and Linux* Future proofed: Align with Intel hardware strategy for the longterm. Optimize for new hardware without code changes BenefitsHardware (TTM): Automatically utilize latest hardware capabilities on Linux & Windows without multiple implementations. Write in Windows, deploy on Linux or vice versa. Fully utilize Intel hardware capabilities for more powerful, cost efficient devices/platforms. Client: Better experience, longer battery life Server: Higher density = more streams per server & lower space/power costs ImpactsTime-to-Market Disruptive technology: more streams per server with lower space/power costs changes the rules of the game and makes new things possible. 14 One unified API for codecs & frame processing across hardware generations 14

15 3D FF Codecs + Frame Processing use Fixed Function + EUs Video Encoding ENC= EU+VDBox VME (MB type, motion vectors, bit budget/brc) PAK = VDBox (residue packing & entropy coding) VDENC = low power encode (6 th Generation Core & forward) VPP Video Decoding BSD=VDBox decode Media Fixed Function VDBOX VEBOX EU EU EU EU EU EU EU EU EU EU EU EU EU EU EU EU EU EU EU EU EU EU EU EU Sampler Sampler Sampler Caches VPHal Video Processing Hardware Acceleration Layer VEBox Deinterlacing Denoise (Luma/Chroma) Frame Rate Conversion Color space conversions Composition/alpha blending Scaling 15 15

16 Application Basic Structure of a Media SDK-optimized Application Initialize Session, set parameters Query + Allocate Main loop Find free surface Q stages: decode, VPP Encode Sync Retrieve output Drain loop Same as above Clean up, exit 16 16

17 SDKs More Efficient Together Problem Solution GPU CPU Decode X Process Encode X GPU CPU Decode Process Encode Expensive GPU<->CPU copy/sync Entire pipeline on GPU for greatest efficiency Key Intel Media SDK Component Intel SDK for OpenCL Applications Component 17 17

18 Hardware View (OpenCL perspective) Extensions: Built-in fixed function access Load function in app init clcreateprogramwithbuiltinkernels set parameters (src/ref images, outputs, ) enqueuendrangekernel as usual GPGPU for EUs Low level C subset/spir provides instructions 2x 128-bit SIMD FPUs (8 32 bit float or int operations per cycle) Images (I/O for MSDK) read through sampler Buffers read through data port On many processors EU capabilities higher than CPUs 18 18

19 Motion Estimation Extensions Two extensions for hardware video motion estimation Host APIs & built-in kernels & associated constant definitions cl_intel_motion_estimation (1.0) Defines an accelerator for cl_intel_accelerator extension Provides a built-in kernel for Intel Graphics Hardware motion estimation cl_intel_advanced_motion_estimation (2.0) New built-in kernel for shape decisions Intra/Inter, bi/uni-directional prediction, partitioning, partition size Available Today: Khronos Spec & Sample software.intel.com/en-us/articles/intro-to-advanced-motion-estimation-extension-for-opencl 19

20 Application Media SDK+OpenCL Application Initialize Session, set parameters Compile/load OpenCL kernels Find free surface Query + Allocate (add clcreatefromdx9mediasu rfacekhr) Q stages: decode, VPP, Main loop Sync clenqueueacquiredx9me diasurfaceskhr Launch kernel set params, clenqueuendrangekernel clenqueuereleasedx9me diasurfaceskhr 20 20

21

22 How-To 1. A minimal example (code walk through) 2. Use case: Color conversion filter 3. Use case: BRC Advantages: A few simple steps to keep your pipeline unified Constant workload boosts GPU to peak frequency Free your CPUs for other tasks Create new capabilities not possible otherwise in small space/power/cost constraints Fixed Function Performance Your innovation via GPGPU New possibilities! 22 22

23 A Simple Kernel kernel void Flip( read_only image2d_t src, write_only image2d_t dst, int h) { int2 coord_src = (int2)(get_global_id(0), get_global_id(1)); int2 coord_dst; coord_dst.x = coord_src.x; coord_dst.y = h - coord_src.y - 1; Init Decode Process } uint4 pixel = read_imageui(src, CLK_FILTER_NEAREST, coord_src); write_imageui(dst, coord_dst, pixel.xyzw); Inner loop, only 4 pixels 23 23

24 Kernel Components (parameters) kernel void Flip( read_only image2d_t src, write_only image2d_t dst, int h) { int2 coord_src = (int2)(get_global_id(0), get_global_id(1)); int2 coord_dst; kernel name + parameters coord_dst.x = coord_src.x; coord_dst.y = h - coord_src.y - 1; } uint4 pixel = read_imageui(src, CLK_FILTER_NEAREST, coord_src); write_imageui(dst, coord_dst, pixel.xyzw); 24 24

25 Kernel Components (coordinate calculations) kernel void Flip( read_only image2d_t src, write_only image2d_t dst, int h) { int2 coord_src = (int2)(get_global_id(0), get_global_id(1)); int2 coord_dst; coord_dst.x = coord_src.x; coord_dst.y = h - coord_src.y - 1; where to get data, where to put it } uint4 pixel = read_imageui(src, CLK_FILTER_NEAREST, coord_src); write_imageui(dst, coord_dst, pixel.xyzw); 25 25

26 Kernel Components (read data in) kernel void Flip( read_only image2d_t src, write_only image2d_t dst, int h) { int2 coord_src = (int2)(get_global_id(0), get_global_id(1)); int2 coord_dst; } coord_dst.x = coord_src.x; coord_dst.y = h - coord_src.y - 1; uint4 pixel = read_imageui(src, CLK_FILTER_NEAREST, coord_src); write_imageui(dst, coord_dst, pixel.xyzw); fn to read from sampler, not automatic like traditional CPU model 26 26

27 Kernel Components (operation + output) kernel void Flip( read_only image2d_t src, write_only image2d_t dst, int h) { int2 coord_src = (int2)(get_global_id(0), get_global_id(1)); int2 coord_dst; coord_dst.x = coord_src.x; coord_dst.y = h - coord_src.y - 1; Combine steps fn to write back } uint4 pixel = read_imageui(src, CLK_FILTER_NEAREST, coord_src); //do something more interesting here write_imageui(dst, coord_dst, pixel.xyzw); 27

28 Decode Steps for (;;) { sts = mfxdec.decodeframeasync(&mfxbsin, &pmfxsurfaces[nindex], &pmfxframesurfout, &syncp); Init Decode Process if (MFX_WRN_DEVICE_BUSY == sts) {MSDK_SLEEP(1); continue;} if (MFX_ERR_MORE_DATA == sts) {sts = ReadBitStreamData(&mfxBSin, fsource); continue;} if (MFX_ERR_MORE_SURFACE == sts) {nindex = getfreesurfaceindex(pmfxsurfaces, numsurfaces); continue;} if (MFX_WRN_VIDEO_PARAM_CHANGED == sts) continue; //todo: handle resolution changes, etc. if (MFX_ERR_NONE == sts) break; //new frame is ready after sync } ready sts = session.syncoperation(syncp, 60000); // Wait until decoded frame is Media SDK decode may loop through several states before completing 28 28

29 More input Input finished Simplified Decode Flow Initialize Main loop DecodeFrameAsync Drain loop DecodeFrameAsync Expected Return Codes for DecodeFrameAsync MFX_ERR_MORE_SURFACE A new surface is required to proceed this is where decode will write its output MFX_ERR_MORE_DATA (bitstream in) (null in) More input bitstream data is required to proceed MFX_WRN_DEVICE_BUSY MFX_ERR_MORE _SURFACE MFX_ERR_MORE _SURFACE Hardware device is unable to respond. This is an expected output for normal operation & should clear shortly. If this state persists more than a few milliseconds, this may indicate a problem. MFX_WRN_VIDEO_PARAM_CHANGED MFX_ERR_MORE _DATA The SDK decoder parsed a new sequence header. Decoding can continue with existing frame buffers. The app can optionally retrieve new video parameters by calling MFXVideoDECODE_GetVideoParam. Other Finish (MFX_ERR_MORE_DATA indicates all surfaces drained) Other error codes may be bugs. Contact Intel Support for more info

30 Launching the OpenCL Kernel Init Decode Process clenqueueacquiredx9mediasurfaceskhr(m_clqueue, 4, m_oclsurfaces, 0, NULL, NULL); // Y plane clsetkernelarg(m_clkernel, 0, sizeof(cl_mem), &m_poclbuffers[ocl_in][mididxin].ocl_y); // In clsetkernelarg(m_clkernel, 1, sizeof(cl_mem), &m_poclbuffers[ocl_out][mididxout].ocl_y); // Out clsetkernelarg(m_clkernel, 2, sizeof(cl_int), &m_framesizey[1]); // Frame height clenqueuendrangekernel(m_clqueue, m_clkernel, 2, NULL, m_globalworksizey, m_localworksizey, 0, NULL, NULL); // UV plane clsetkernelarg(m_clkernel, 0, sizeof(cl_mem), &m_poclbuffers[ocl_in][mididxin].ocl_uv); // In clsetkernelarg(m_clkernel, 1, sizeof(cl_mem), &m_poclbuffers[ocl_out][mididxout].ocl_uv); // Out clsetkernelarg(m_clkernel, 2, sizeof(cl_int), &m_framesizeuv[1]); // Frame height clsts = clenqueuendrangekernel(m_clqueue, m_clkernel, 2, NULL, m_globalworksizeuv, m_localworksizeuv, 0, NULL, NULL); clsts = clenqueuereleasedx9mediasurfaceskhr(m_clqueue, 4, m_oclsurfaces, 0, NULL, NULL); 30 30

31 Use Case: Color Format Conversion Filter R G B R G B R Y0 Y1 Y2 Y3 other filters? Encode U0 V0 U1 V1 RGB3 (packed format) NV12 (planar format) kernel void RGB3_to_NV12( read_only image2d_t src, write_only image2d_t Ydst, write_only image2d_t UVdst, ) { coord calculations read_imageui(src) (calculate Y,U,V from src RGB) For additional efficiency chain more operations here write_imageui(ydst) write_imageui(uvdst) } 31 31

32 Use Case: Bitrate Control (BRC) Two paths to improve codec quality 1. Core algorithms 2. Bitrate control VME data for QP decisions Encode Motion estimation kernel output and custom quality metrics can be combined in novel ways to build custom BRC with your own secret sauce. pre-check frames Custom quality metric filter Decode bitstream 32 32

33 Visualize FF+GPGPU+CPUs working together with Intel VTune Amplifier 33

34

35 Why OpenCL + Intel Media SDK? Decode What Media SDK Covers (high level) Process OpenCL Covers Fuller range/lower level Encode Media SDK provides optimized implementations for Codecs Frame Processing Operations For video processing tasks not in Media SDK s scope, extend with OpenCL Make use of growing GPU capabilities Keep pipelines on GPU Example uses: color conversions, custom bitrate control Fixed Function Performance Add your innovation via GPGPU Build something awesome! 35

36 Unleash the Full Range of Gen Graphics Capabilities for Video Processing Applications CALL TO ACTION Develop fast, efficient, competitive media apps - access media accelerators in Intel GPUs Servers & Embedded: Intel Media Server Studio free Community Edition For clients: Intel Media SDK & Intel SDK for OpenCL Applications free standalone tools Get Started - use Media Samples & Tutorials 36 36

37 For more information, visit: Intel Media Server Studio & how-to articles Intel Media SDK & Intel SDK for OpenCL Applications The webinar materials & recording will be ed to you soon. Watch for more webinars in Q3 & Q4.

38 Legal Notices, Disclaimers & Intel technologies features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. No computer system can be absolutely secure. Check with your system manufacturer or retailer or learn more at intel.com. Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance. Consult other sources of information to evaluate performance as you consider your purchase. For more complete information about performance and benchmark results, visit All information provided here is subject to change without notice. Contact your Intel representative, sales office or distributor to obtain the latest Intel product specifications and roadmaps. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. The cost reduction scenarios described in this document are intended to enable you to get a better understanding of how the purchase of a given Intel product, combined with a number of situation-specific variables, might affect your future cost and savings. Nothing in this document should be interpreted as either a promise of or contract for a given level of costs. INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER, AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT. Intel, the Intel logo, Xeon, Core, Iris Pro, and VTune are trademarks of Intel Corporation in the U.S. and other countries. Intel s compilers may or may not optimize to the same degree for non-intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice. Notice revision #

39

40 How to Get the Intel Media SDK Intel Media Server Studio 3 Editions (includes Free Community) Platform / Device Targets Select SKUs of Intel Xeon & Core processor-based platforms Applications for media, communications infrastructure, video processing/conferencing, digital surveillance, video cloud & data center Format Support - HEVC, AVC, MPEG-2, MPEG-Audio See Technical Specifications for System Requirements Intel Media SDK - FREE Platform / Device Targets Intel Core or Core M processors Select SKUs of Intel Celeron, Pentium & Atom processors with Intel HD Graphics supporting Intel Quick Sync Video Client devices Desktop/mobile applications OS - Windows only* See Technical Specifications for System Requirements software.intel.com/intel-media-server-studio software.intel.com/media-sdk 40

41 Intel Media Server Studio Editions At a Glance Premium Feature/Components Community Edition Essentials Edition Professional Edition Media SDK Graphics Drivers Code Samples OpenCL Code Builder and Runtime Metrics Monitor (Linux* only) Intel Premier Support HEVC Decoder & Encoder, GPU Assist APIs Audio Decoder & Encoder Video Quality Caliper Intel VTune Amplifier Premium Telecine Interlace Reverser 41

42