UNCOMPRESSED HIGH-RESOLUTION WORKFLOWS WITH DELL EMC ISILON

Transcription

1 White Paper UNCOMPRESSED HIGH-RESOLUTION WORKFLOWS WITH DELL EMC ISILON UNDERSTANDING THE REQUIREMENTS AND CHALLENGES OF STORAGE SYSTEM SPECIFICATION AND DESIGN Abstract This document examines the challenges of specifying and implementing storage systems to support high-performance creative applications. Highperformance creative applications are defined as being those that are used to manipulate multiple streams of high resolution video in uncompressed formats, or high quality compressed formats. It is intended that architects and system integrators use this document as a guide to both understanding the challenges of working with high-performance creative applications, and to designing solutions built with Dell EMC products to deliver against those challenges. H16818 November, 2017

2 EXECUTIVE SUMMARY AND SCOPE... 3 OVERVIEW OF APPLICATION DEMANDS... 4 PERFORMANCE... 4 COLLABORATIVE WORKING... 4 OVERVIEW OF FILE TYPES... 5 METADATA FILES... 5 MEDIA FILES... 5 FRAME SIZE AND COLOUR DEPTH... 6 VIDEO ENCODING... 8 IMAGE SEQUENCE... 8 CLIP BASED MEDIA COMPRESSION COMPARATIVE BENEFITS PERFORMANCE REFERENCE PERFORMANCE METRICS THROUGHPUT LATENCY COLD DATA CONFIGURATION GUIDELINES WORKSTATION SPECIFICATION DELL EMC ISILON AND ONEFS VERSION NETWORK CONNECTIVITY FILESHARING CONFIGURATION ADVANCED CONFIGURATION AND OPTIMISATIONS ONEFS OPTIMISATIONS METADATA READ/WRITE ACCELERATION STREAMING MODE FILENAME PREFETCH MORE INFORMATION CLIENT AND APPLICATION OPTIMISATIONS SUMMARY AND CONCLUSION... 20

3 EXECUTIVE SUMMARY AND SCOPE This document examines the challenges of specifying and implementing storage systems to support high-performance creative applications. Creative applications from different vendors are generally characterised by a number of similar requirements: Relatively high throughput single stream performance. Relatively low latency response. In the context of this document, high-performance creative applications are defined as being those that are used to manipulate multiple streams of high resolution video in uncompressed formats, or high quality compressed formats. As well as the performance demands made by creative applications of storage systems, each application will make particular demands to support the overhead of content creation - for example media management, metadata and other small files, and sharing. These requirements are not covered by the scope of this document. It is intended that architects and system integrators use this document as a guide to both understanding the challenges of working with high-performance creative applications, and to designing solutions built with Dell EMC products to deliver against those challenges. In particular, it is assumed that a system will be designed around clustered storage using Dell EMC Isilon Scale-Out NAS and its underlying system: OneFS, and Dell Precision Workstations. The requirements of high-performance workflows may be satisfied by using Dell EMC Isilon H600 or F800 all-flash (or later variants), with OneFS 8.1 or later. It is recommended that the reader refer to other documents published by Dell EMC regarding optimisations for specific creative applications and client operating systems. Available documents are listed at:

4 OVERVIEW OF APPLICATION DEMANDS Creative applications include those that deal with editing, compositing, 3D effects and finishing; and the demands of these applications are high - particularly at high frame rates and resolutions. Consequently, these kinds of applications require a highly performant workstation, networking and storage infrastructure. PERFORMANCE When designing a system, architects will need to consider both quantitative and subjective requirements. Throughput Latency Demands of different codecs Scrubbing Multi-track Subjective feel Though hardest to quantify and measure, the subjective feel of an application as it is used by a Creative Artist is arguably the most important measure of a successful system design. An Artist may describe a system that responds well and is easy to use as snappy, whilst a system that does not respond well and is difficult to use may be described as sluggish. For a system to be snappy, the storage component must deliver data to the application in a way that doesn t interrupt the creative process. COLLABORATIVE WORKING A primary reason for selecting a shared storage solution is to support a collaborative environment. A fundamental requirement of collaborative working is that Artists should be able to read and write files to the storage, and that they should be able to access the files of other Artists in a controlled manner. Additional requirements may include: NFS and SMB multi-protocol support. Read-whilst-write across accessing clients. Active Directory binding, and complex ACL implementation. In some environments, the creative application or the user themselves will manage the files on the shared storage. In other environments (in particular, larger or more complex environments), a separate Media Asset Management layer will be configured. Media Asset Management is outside of the scope of this document.

5 OVERVIEW OF FILE TYPES Most creative applications work with a number of different file types: Metadata: Action Only Formats Sequence Exchange Formats Media: Image Sequences Clip Based Media o o o o QuickTime clips Media Exchange Format (MXF) clips MP4 clips Other Streaming Codecs Each file type presents a different challenge to each part of the system, and the file types being used by the required workflow must be considered. METADATA FILES Metadata and sequence information files are typically small in size and frequently updated. These files contain information about the sequences and media being created or manipulated. For example, such a file could contain information about which clips are used in a sequence, or information about how the colours should be corrected. Metadata and sequence information files are relatively small, and whilst they should not present any meaningful contribution to overall throughput requirements, it is important to consider that the files will be frequently accessed and updated. However, this document focuses on the performance requirements of creative applications, and so designing a storage system to accommodate the requirements of metadata and sequence information is outside of its scope. MEDIA FILES Because of their large size, media files are normally considered to represent the most important challenge to an Architect when designing systems to support creative applications. There are three important factors that influence the size of media files, and that impact the demands of supporting systems: Frame Size and Colour Depth Encoding Method Compression Method (if any)

6 FRAME SIZE AND COLOUR DEPTH A key factor that impacts the size of encoded video files, is the number of pixels in each frame. There are a number of common sizes, with the most common being: Common Name Application Horizontal Size (pixels) Vertical Size (pixels) Total number of Pixels HD Broadcast Television K (DCI) Cinema UHD Broadcast Television K (DCI) Cinema Table 1. Common formats, resolutions and pixel counts HD: High Definition typically used in television UHD: Ultra High Definition typically used in television DCI: Digital Cinema Initiatives typically used in digital cinema The colour of each pixel is represented with a number of bits according to the required colour depth. High dynamic range media requires encoding at a greater colour depth. Colour Depth 10-bit 12-bit 16-bit Application Standard Dynamic Range High Dynamic Range (HDR) High Dynamic Range (HDR) Table 2. Colour depth and normal application

7 Figure 1. Relative Video Frame Size and Colour Depth The uncompressed byte size of each frame can be approximated by multiplying the number of pixels in the frame by the colour bit depth, and then again by the number of channels. For example: a typical UHD frame has pixels (3840 x 2160), is encoded using three channels (red, green and blue RGB) and at a colour depth of 10 bits; giving a total of bits (or 29.66MBytes). (3840 x 2160) x 10 x 3 / (8 x 1024 x 1024) = 29.66MBytes To calculate a more accurate byte size, it is necessary to consider how the data is encoded in the underlying data stream which is built on 32-bit words. With 10-bit encoding, the data for three pixels will fit in a single word leaving two bits remaining. To improve performance, most applications arrange to start the bitstream of each pixel at a boundary of a 32- bit word in the underlying stream, so those final two bits are not used and instead are padded with null data. Our calculation for the uncompressed byte size of each frame should therefore be slightly modified: (3840 x 2160) x ((10 x 3) + 2) / (8 x 1024 x 1024) = 31.64MBytes Most applications will also add additional header and payload information, so the byte size of each frame can be even higher.

8 Common Name Resolution Total number of Pixels Bit Depth Frame Byte Size HD 1920x MBytes 2K (DCI) 2048X MBytes UHD 3840x MBytes 4K (DCI) 4096x MBytes Table 3. Uncompressed Frame Byte Size The strategy for data padding to match 32-bit word boundaries varies with the application, but it should be noted that image sequences encoded at a colour depth of 16-bit will not require any padding, whilst sequences encoded at 12-bit may require a complex strategy to avoid wasteful padding. VIDEO ENCODING A sequence of video frames can be encoded for digital storage in one of two ways: Image Sequence Clip Based Media IMAGE SEQUENCE An image sequence (such as a DPX sequence) represents each frame of video with a separate file. The files for a sequence are stored in the same directory, have a fixed base name with a decimal or hexadecimal incrementing suffix, and a.dpx extension. Different applications use different naming structures and standards within this form, but some illustrative examples would be: filebasename_scene-01_00056.dpx filebasename_scene-01_00057.dpx filebasename_scene-01_00058.dpx filebasename_scene-05_0fc09.dpx filebasename_scene-05_0fc0a.dpx filebasename_scene-08_0fc0b.dpx Image sequences can be more challenging for a storage system, because it can be difficult to ensure that file read-ahead and caching strategies yield a benefit. OneFS can be optimised to deliver increased throughput performance for image sequences using the FileName PreFetch option - details of which are covered later in this document.

9 Image sequences are particularly suited to VFX (visual effects) workflows, because artists very often work on one frame video frame at a time. Most creative applications work with a number of different image sequence formats, including: DPX Raw Open EXR DPX OVERVIEW The picture information of each image in a DPX sequence is encoded as a bit map. Each pixel is recorded as an integer value. Consequently, the file size and data rate for a.dpx sequence is steady and predictable. Because of the simple encoding method, a DPX sequence makes the lowest demands of the application workstation. However, the larger file size leads to a relatively high demand on storage and networking systems. RAW OVERVIEW Modern single sensor cameras capture image information using a technique developed by Bryce Bayer in the 1970s. An arrangement of coloured masks is overlaid on the pixels of the camera sensor with twice as many green filters as either red or blue. The data that is captured is referred to as a RAW image, and must be processed by the application using a process termed de-bayering before it can be displayed. Consequently, a RAW image is smaller in size than a DPX image, and may be considered to be compressed. The de- Bayering of the image requires processing power of the host workstation. For more information about Bayer Filtering, check Different cameras implement Bayer Filtering in different ways, with a number of proprietary approaches. For a list, check OPEN EXR OVERVIEW Open EXR is a more complex format. Pixel values are represented using floating-point code values rather than integer code values ( in 10-bit). This is separate from the colour depth encoding of each pixel. An Open EXR image may have a colour depth of 10 bits, 12 bits, or 16 bits per RGB channel. Files may be uncompressed, or compressed using one of a number of algorithms; and each file may contain multiple additional layers of information: Geometry (vector normals) Arbitrarily number of extra channels (in addition to RGB): Specular Highlights Bump Maps Alpha Mattes Texture Maps (multiple images in the same file). The Open EXR format is often used to encode HDR (High Dynamic Range) image sequences. A high dynamic range image contains more colour and luminance depth information put simply, more and smaller steps between complete

10 black and complete white. Compared to an image encoded using a format with a standard dynamic range, an HDR image shows much more detail in very dark and very light parts of the scene. Because of the variety of possible file elements and encoding strategies, it can be difficult to predict the data rate of an Open EXR image sequence. Care should be taken to understand the exact requirements of any proposed format. It should also be noted that Open EXR image sequences may make a relatively high demand of the application workstation. MORE INFORMATION More information about DPX, ARRI Raw and Open EXR can be found at: CLIP BASED MEDIA A Clip based media file represents an entire sequence of video frames (and additionally very often interleaved audio, timecode and metadata) in a single file on the storage. Because an entire sequence of video frames is stored in a single file, it s more straight-forward for conventional caching and read-ahead algorithms to yield benefit. For example, when in Streaming Mode, OneFS will read-ahead and stage files as they are accessed by the client. Most creative applications work with a number of different clip-based codecs, including: RED XAVC ProRes DNX More information about these formats - including their respective strengths and use-cases - can be found at: COMPRESSION Both image sequences and clip-based media can be compressed. Compression can yield benefits in required storage capacity and throughput, but can generate computational overhead and reduce the perceived visual and audio quality of the media.

11 There are two ways of compressing media files in Media and Entertainment workflows: Lossy, and loss-less. Both methods can reduce the amount of data requiring to be stored, but different approaches are taken in the underlying algorithms. The terms lossy and loss-less refer to the visible (or audible) representation of the images being compressed, and can be characterised by our ability to reverse the compression. LOSS-LESS COMPRESSION Loss-less compression is completely reversible - it is possible to recreate exactly the original images (or sounds). This type of compression will be the most familiar to data scientists who are used to using compression utilities to reduce the size of many types of file - and then re-inflating them to their exact original state when required. LOSSY COMPRESSION Lossy compression is not reversible. It is not possible to recreate exactly the original source - because some data is discarded in the compression process. Lossy compression is built on algorithms that exploit the characteristics and limitations of human perception. For example, perhaps the most fundamental type of lossy compression exploits the fact that humans can perceive spatial resolution (in other words, the number of dots on the screen) in colour at only about half of that in luminance. The upshot is that compression algorithms can encode the colour parts of an image at only half the spatial resolution of the luminance parts - and we can t tell the difference. More complex lossy compression (for example, JPEG and similar) is built on algorithms that divide images into segments, and then describe those segments using patterns or waveforms; or on algorithms that identify parts of an audio scene that cannot be heard because they are masked by other elements. Depending on the aggressiveness of the algorithms being used in lossy compression, humans may be able to perceive sound or image degradation. The term Visually Loss-less is used when it s impossible or nearly impossible to discern the difference between the original and a compressed image - even though some original data has been discarded in the compression process. I-FRAME AND LONG GOP ENCODING Frames in an image sequence may be individually compressed - using either loss-less or lossy algorithms. Files in an OpenEXR image sequence are often compressed using a loss-less algorithm - for example run-length encoding or scan-line zip. Files in a JPEG2000 image sequence may be compressed using a lossy (or visually loss-less) algorithm based on wavelet transforms. (Note that JPEG2000 uses a very different encoding algorithm to JPEG). Clip based media is normally compressed using a lossy algorithm, using one of two strategies: I-Frame (Intra-Frame) clip-based codecs perform compression only in the spatial domain. Each frame of video is compressed individually using algorithms that are similar to those used when compressing still images - making it relatively simple to cut and splice material at frame boundaries. Creation workflows typically perform best when using I-Frame codecs. Long GOP (Group of Pictures) clip-based codecs additionally perform temporal compression. Each frame of video in a sequence is compared to adjacent and near-by frames, and additional compression is achieved by storing only the differences between those frames, rather than each entire frame. Long GOP codecs are more

12 efficient on storage and throughput, but are more computationally expensive to create and consume. They are best suited to delivery workflows. COMPARATIVE BENEFITS It follows that different codecs have different benefits, and present different challenges to each part of a system. Fundamentally, there are three key parameters - and a system can be optimised for any two, at the expense of the third. Data Rate Encode and Decode Overhead Image Quality For example, an uncompressed DPX sequence is very high quality and has a very low decode overhead, but it has a relatively high data rate. A high quality XAVC clip will have a relatively low data rate, but a much higher decode overhead. Figure 2. Quality, Data Rate and Decode Overhead Comparison PERFORMANCE REFERENCE Some Artists - particularly those dealing with visual effects - normally require to work with image sequences (rather than clip based media), and very often prefer to work with uncompressed material. Consequently, it s important that Architects design a full system that is able to support the high-performance demands of these workflows. This document focusses on the configuration and optimisations to deliver the best performance when working with uncompressed DPX image sequences at UHD resolution.

13 PERFORMANCE METRICS Creative applications make the highest demands of workstations, networking and storage in any implementation, and so systems must be architected to accommodate those performance requirements. There are two key system performance metrics that architects must consider when specifying storage solutions for a creative application: Throughput Latency Throughput performance is important to ensure that the infrastructure can sustain the streaming requirements of the application. The higher the throughput that can be achieved, the better. Latency performance is important to ensure that the throughout performance is consistent, and achieved quickly. The lower the latency that can be achieved, the better. The responsiveness of the application and its usability in real-world workflows is highly dependent on both throughput and latency performance. If the infrastructure can t support the required throughput, users are likely to report that video playback is stuttering or that it freezes. If the infrastructure can t support the required low latency, users are likely to report that the application is sluggish or unresponsive - particularly when scrubbing on a timeline, or rapidly selecting and manipulating content. THROUGHPUT Throughput performance can be considered as a measure of the required sustained data transfer rate. To sustain playback at the application layer, the system must be able to support at least the required throughput over an indeterminate duration. Throughput for uncompressed image sequences can be calculated by multiplying the frame byte size by the number of frames per second according to the sequence. The throughput for a selection of 10-bit RGB frame resolutions and rates is outlined in the table below.

14 Common Name Hres Vres Frame Rate Frame Byte Size MBytes Throughput MBytes per second HD fps K (DCI) fps UHD fps K (DCI) fps HD fps K (DCI) fps UHD fps K (DCI) fps Table 4. Codec, frame size and throughput. LATENCY Latency performance can be considered to be a measure of the the required instantaneous data transfer rate. To offer a responsive experience at the application layer, the system must be able to deliver frames in a timely manner after they are requested. One measure of latency is the time to first byte - how long it takes data to start flowing after it is requested by the application. For image sequence workflows (where each frame of video is stored as a separate file), it is assumed that the application normally will request each frame sequentially whilst the preceding frame is being displayed. It follows therefore that for steady and sustained playback, each file must be requested and retrieved in less than the time for which each frame is displayed. Perceived performance can be increased (perceived latency reduced) by configuring intelligent read-ahead (or precaching) at the application, client operating system or storage layer. An application that is configured to read-ahead (or pre-cache) will intelligently request frames or files in advance of their being required, and will typically stage them in client RAM. OneFS can be configured to aggressively pre-fetch and stage media requests by setting the SmartPools policy for the required directories to Streaming. Note that in Streaming Mode, other optimisations are also applied at the cluster - including details regarding the layout of data across the underlying storage units. OneFS can be optimised to deliver increased throughput performance for image sequences using the FileName PreFetch option. Client operating systems may implement their own intelligent caching algorithms.

15 Configuring any part of a system to intelligently read-ahead (or pre-cache) should be done with care and consideration. No matter how intelligent the algorithm used, there is still a possibility that it will pre-fetch or stage media that will not ever be required by the application - thereby unnecessarily consuming system resources. A simple example of such a false fetch would be where a user halts media playback part way through a sequence. A summary of file sizes and frame durations for a number of 10-bit image sequence formats helps quantify the required latency targets Use Case Frame Rate Common Name Hres Vres Frame Size MBytes Frame Duration Cinema 24fps 4K (DCI) ms PAL* 25fps UHD ms PAL* 50fps UHD ms NTSC* 30fps UHD ms NTSC* 60fps UHD ms Table 5. *Note: The references to PAL and NTSC are used to indicate a loose alignment to legacy television standards and regional variations, and do not refer to any form of colour encoding, number of TV lines, or refresh rate. COLD DATA Whilst the optimisations of Streaming Mode and FileName PreFetch can enable performance and usability improvements, care must be taken to consider overall system performance when dealing with Cold Data. Cold Data is data or media that is requested by the application but that is not cached or pre-staged in any part of the environment by the application, the client operating system, or the storage. It is data that must be retrieved from the underlying storage units - normally assumed to be the component spinning disc or solid-state devices - and processed by the application within the required time. Scenarios where a creative application may generate a request for Cold Data include: Playing media instantaneously, without first cueing. Scrubbing backwards-and-forwards along a timeline in an unpredictable way. Jumping to, or skipping over a part of a sequence. Playing a sequence that is comprised of a number of other unrelated sequences which typically occurs at an edit point in a sequence.

16 Figure 3. Cold Data at Edit Point on Timeline Architects must consider a number of strategies to deliver performance when Cold Data is requested: Raw storage performance. Intelligent application read-ahead (or pre-cache). The challenges of Cold Data exist for both image sequence and clip-based media formats. CONFIGURATION GUIDELINES WORKSTATION SPECIFICATION Dell EMC recommends a suitably specified Dell Precision Workstation or similar for use with most high-performance creative applications. Specification and configuration of the workstation is outside of the scope of this document. DELL EMC ISILON AND ONEFS VERSION It is assumed that architects will specify Dell EMC Isilon H600 and F800 all-flash and OneFS version 8.1 or later to support the highest performance creative applications. NETWORK CONNECTIVITY In order to achieve the throughputs required for typical high-performance workflows, it is required to configure a full 40GbE network - including Dell EMC Isilon storage, network infrastructure and client interface. The choice of Network Interface Card, and recommendations regarding network interface optimisation are dependent on the specific environment of each installation, and are therefore outside of the scope of this document. FILESHARING CONFIGURATION Most creative applications are hosted on a Linux, Windows or macos platform, using NFS or SMB filesharing from the Isilon cluster.

17 The creation of an NFS Export or SMB share on the Isilon storage cluster can be achieved using the administrative web interface of OneFS. Details are available at NFS and SMB optimisation guidelines for connections from different operating systems are dependent on the specific environment of each installation, and are therefore outside of the scope of this document. ADVANCED CONFIGURATION AND OPTIMISATIONS When working with high-performance applications, it is important to consider optimisations for the entire system, and the mutual interactions of those optimisations. Attention in particular should be given to optimisations for: OneFS - The Isilon storage cluster. Application - The creative application itself. ONEFS OPTIMISATIONS Architects should consider three key optimisations for OneFS when deploying in high-performance creative application environments. Metadata Read/Write Acceleration Streaming Mode FileName PreFetch METADATA READ/WRITE ACCELERATION OVERVIEW File access performance can in some workflows be optimised by enabling Metadata read/write acceleration. When enabled, this optimisation sees OneFS use SSDs for reading and writing filesystem metadata - which can improve access time and reduce latency. Where all underlying storage units are SSD (as in the case of Dell EMC Isilon F800 allflash), this is not a meaningful or required optimisation. ENABLING Metadata read/write optimisation can be enabled as a File Pool Policy at the OneFS Web User Interface. More information and guidance is available in published documentation: STREAMING MODE OVERVIEW Streaming performance of OneFS can be optimised by enabling Streaming Mode at the file pool, or at the directory level. Streaming performance is important when playing media linearly - for example when playing a video sequence in a time line.

18 Consideration should be given to other workflows hosted in the same pool or directory before enabling Streaming mode. Best performance for all workflows on a single cluster might be achieved by selectively enabling Streaming Mode on a limited number of directories. OPERATION Streaming Mode optimises two behaviours of OneFS to deliver increased streaming performance: Data is striped across more underlying storage units (disc drives or SSDs). Data is pre-fetched aggressively. ENABLING Streaming mode can be enabled as a File Pool Policy at the OneFS Web User Interface. More information and guidance is available in published documentation: - ifs_t_configure_default_io_optimization_settings.html - ifs_t_modify_file_and_directory_properties.html FILENAME PREFETCH OVERVIEW Cluster-wide FileName PreFetch can be enabled to optimise the streaming performance of OneFS when working with image sequences. Consideration should be given to other workflows hosted on the same cluster, and the impact of possible false fetches (where resources are used pre-fetching files that are not ever requested by the client) should be evaluated. OPERATION FileName PreFetch enables a performance optimisation by detecting when image sequences are requested by a client. When a sequence is detected, OneFS will pre-fetch files from the underlying storage units (disc drives or SSDs) before they are requested. Figure 4. FileName PreFetch conceptual overview The FileName PreFetch algorithm is optimised to detect image sequences with either decimal or hexadecimal filename numerical increments.

19 ENABLING FileName PreFetch can be enabled across the cluster at the command line of OneFS, authenticated as Root. Working at the command line of OneFS - particularly when authenticated as Root - should be attempted only by authorised and qualified System Administrators. Extreme care should be exercised. Mistakes can cause data loss. To enable FileName PreFetch: isi_for_array sysctl isi.access.streaming.fnprefetch.fn_enabled=1 isi_for_array sysctl isi.access.streaming.prefetch.prefetch_by_cluster=1 MORE INFORMATION For more information about Streaming Mode and FileName PreFetch, check: For detailed guidance on File Pool Policies and data layout and access strategies, check: Guidance for using the command line to make administrative changes is available at: Guidance for using the web interface to make administrative changes is available at: CLIENT AND APPLICATION OPTIMISATIONS Overall performance, and the subjective quality of the user experience often can be enhanced by optimising the application to take full advantage of the power of Isilon and OneFS. Specific application optimisations are covered in separate documents.

20 SUMMARY AND CONCLUSION Accommodating uncompressed high-resolution workflows can represent a challenge to Architects and System Integrators. This document aims to outline the nature of the performance required by creative applications, and to direct the reader to other resources in order to design systems to fulfil those requirements. Learn more about Dell EMC Isilon Contact a Dell EMC Expert All Rights Reserved. Dell, EMC and other trademarks are trademarks of Dell Inc. or its subsidiaries. Other trademarks may be trademarks of their respective owners.reference Number: XXXXXXXXX