BEST PRACTICES FOR USING SMB3 MULTICHANNEL FOR 4K VIDEO PLAYBACK

BEST PRACTICES FOR USING SMB3 MULTICHANNEL FOR 4K VIDEO PLAYBACK Uncompressed 4K image sequence playback with Dell EMC Isilon, SMB3 Multichannel, and the Assimilate SCRATCH play application ABSTRACT This white paper explains the configuration requirements for playback of uncompressed 4K image sequences over SMB3 multichannel. This document details the DELL EMC Isilon OneFS 7.1.1 Operating System or greater configuration requirements as well as the architectural requirements for greater than 10GbE connectivity. The Assimilate SCRATCH Play application is used as the reference platform for 4K playback in the document. February, 2017 WHITE PAPER

The information in this publication is provided as is. Dell Inc. makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any software described in this publication requires an applicable software license. Copyright 2017 Dell Inc. or its subsidiaries. All Rights Reserved. Dell, EMC, and other trademarks are trademarks of Dell Inc. or its subsidiaries. Other trademarks may be the property of their respective owners. Published in the USA 02/17 White Paper H13448.1 Dell EMC believes the information in this document is accurate as of its publication date. The information is subject to change without notice. 2

TABLE OF CONTENTS ABSTRACT...1 EXECUTIVE SUMMARY...4 AUDIENCE... 4 DELL EMC ISILON ONEFS IMPROVEMENTS...4 SMB MULTICHANNEL SUPPORT...5 SMB 3 MULTICHANNEL BEST PRACTICES...6 CLIENT DRIVER TUNING... 6 NETWORK CONSIDERATIONS... 6 ISILON REQUIREMENTS... 7 NETWORK VALIDATION... 7 FILE NAME PREFETCH...8 ASSIMILATE SCRATCH PLAY...8 CONCLUSION...8 3

EXECUTIVE SUMMARY There's no denying the inevitability of 4K. 4K support is either the 800 pound gorilla you're media company is currently wrestling or it's at least a checkbox item on technologies you need to fit into your workflow in the coming years. It won't be long before 4K TVs are the only option at your local big box electronics store. Almost every video camera manufacturer has a 4K camera out now, including lowcost consumer cameras. Without a doubt, the time to switch to 4K acquisition and production workflows is NOW! HDTV and 2K content in the looming 4K world will be of little or no value from a quality standpoint. Isilon customers have been able to support streaming data rates up to ~800 MB/s per client since the introduction of the X400 platform. 800 MB/s per client is sufficient for most media formats, including 4K formats ranging from multiple streams of the compressed Sony XAVC codec up to Ultra HD resolutions. Media and Entertainment businesses currently working with 4K digital intermediate workflows are unable to take advantage of advanced 4K compression technologies like the Sony XAVC 4K codec. Uncompressed 4K media presents a number of challenges: File size - an hour of uncompressed 4096X3072 10bit RGB content at 24 frames per second requires over 4 TB of storage capacity. File format - uncompressed 4K DPX and EXR image sequences are comprised of a single ~50 MB file for every frame of video. Reading and processing 24 or more 50 MB files per second with low latency can be a challenge for most storage and workstation configurations. Storage throughput - your storage system must be able to sustain 1,200 MB per second in order to stream a 4096X3072 10bit RGB image sequence. The 1,200 MB per second storage throughput requirement for uncompressed 4K has historically required a high performance SAN or local attached storage. With the introduction of SMB 3.0 multi-channel support, DELL EMC Isilon can now handle single client streams that exceed the bandwidth limitation of a single 10GbE connection! Applications such as Assimilate SCRATCH that are compatible with the Windows 8 SMB 3.0 multi-channel implementation may now take advantage of SAN performance over NAS by automatically loadbalancing a single SMB stream over dual 10GbE network connections. AUDIENCE This white paper is intended for media professionals working with uncompressed 4K digital intermediate, color correction, restoration, or visual effects workflows. This paper focuses on 4K image sequence playback using Assimilate SCRATCH play, however, the Isilon tuning parameters detailed herein may be applied to any workflow utilizing high performance, dual 10 Gigabit Ethernet (10GbE) network interface cards (NICs) and applications optimized for network attached storage (NAS). DELL EMC ISILON ONEFS IMPROVEMENTS Starting with DELL EMC Isilon OneFS 7.1.1, the OneFS operating system includes a number of performance improvements: SMB Multichannel support OneFS 7.1.1 supports the Multichannel feature of SMB 3.0, which establishes a single SMB session over multiple network connections. SMB Multichannel enables increased throughput, connection failure tolerance, and automatic discovery. To take advantage of this new feature, client computers must be configured with Microsoft Windows 8 or later, or Microsoft Windows Server 2012 or later with supported network interface cards (NICs). For more information, see the SMB Multichannel section of the OneFS 7.1.1 Web Administration Guide and OneFS 7.1.1 CLI Administration Guide. SmartFlash caching In OneFS, level 1 (L1) cache uses random access memory (RAM) to store copies of system metadata and files requested from front-end networks. Level 2 (L2) cache uses RAM to store copies of file system metadata for files that are stored on the node that owns the data. SmartFlash, or level 3 (L3) cache, uses solid-state drives (SSDs) to hold file data and metadata released from L2 cache, increasing the total size of cache memory available in a cluster as well as the speed that data is retrieved. In OneFS 7.1.1, SmartFlash is enabled by default for new node pools. NDMP backup performance improvements OneFS 7.1.1 uses multiple threads to restore files, making data transfer occur as fast as the tape backup device can deliver it. Additional operational enhancements improve throughput when transferring small files. 4

SyncIQ performance enhancements To allow multiple SyncIQ workers to replicate a single file simultaneously, SyncIQ now allows for file splitting, where a large file is split into segments, each of which is processed in parallel by a different thread. Together, all of the performance improvements provide substantial increase in performance for media applications. However, one feature SMB Multichannel support is especially important, especially with 4K workloads and applications that can take advantage of it. Notablly, the SCRATCH Play universal media player combines with SMB Multichannel support to optimize streaming 4K playback over SMB3. SMB MULTICHANNEL SUPPORT SMB Multichannel supports establishing a single SMB session over multiple network connections. SMB Multichannel is a feature of the SMB 3.0 protocol that provides the following capabilities: Increased throughput OneFS can transmit more data to a client through multiple connections over a high speed network adapters or over multiple network adapters. Connection failure tolerance When an SMB Multichannel session is established over multiple network connections, the session is not lost if one of the connections has a network fault, which enables the client to continue to work. Automatic discovery SMB Multichannel automatically discovers supported hardware configurations on the client that have multiple available network paths and then negotiates and establishes a session over multiple network connections. You are not required to install components, roles, role services, or features. You must meet software and NIC configuration requirements to support SMB Multichannel on the Isilon cluster. OneFS can only support SMB Multichannel when the following software requirements are met: Windows Server 2012, 2012r2 or Windows 8, 8.1 clients SMB Multichannel must be enabled on both the Isilon cluster and the Windows client computer. It is enabled on the Isilon cluster by default. SMB Multichannel establishes a single SMB session over multiple network connections only on supported network interface card configurations. SMB Multichannel requires at least one of the following NIC configurations on the client computer: Two or more network interface cards. One or more network interface cards that support Receive Side Scaling (RSS). One or more network interface cards configured with link aggregation. Link aggregation enables you to combine the bandwidth of multiple NICs on a node into a single logical interface. SMB Multichannel automatically discovers supported hardware configurations on the client that have multiple available network paths. Each node on the Isilon cluster has at least one RSS-capable network interface card. Your client-side NIC configuration determines how SMB Multichannel establishes simultaneous network connections per SMB session. SCENARIO SINGLE RSS-CAPABLE NIC INPUT SMB Multichannel establishes a maximum of four network connections to the Isilon cluster over the NIC. The connections are more likely to be spread across multiple CPU cores, which reduces the likelihood of performance bottleneck issues and achieves the maximum speed capability of the NIC. 5

SCENARIO MULTIPLE NICS INPUT If the NICs are RSS-capable, SMB Multichannel establishes a maximum of four network connections to the Isilon cluster over each NIC. If the NICs onthe client are not RSS-capable, SMB Multichannel establishes a single network connection to the Isilon cluster over each NIC. Both configurations allow SMB Multichannel to leverage the combined bandwidth of multiple NICs and provides connection fault tolerance if a connection or a NIC fails. Note SMB Multichannel cannot establish more than eight simultaneous network connections per session. In a multiple NIC configuration, this might limit the number connections allowed per NIC. For example, if the configuration contains three RSS-capable NICs, SMB Multichannel might establish three connections over the first NIC, three connections over the second NIC and two connections over the third NIC. AGGREGATED NICS SMB Multichannel establishes multiple network connections to the Isilon cluster over aggregated NICs, which results in balanced connections across CPU cores, effective consumption of combined bandwidth, and connection fault tolerance. Note The aggregated NIC configuration inherently provides NIC fault tolerance that is not dependent upon SMB. Table 1. Possible Client-Side NIC configurations SMB 3 MULTICHANNEL BEST PRACTICES For best performance playing back uncompressed 4K image sequences, configure your Windows client with a high performance, dualport 10GbE NIC like the Intel X520 and a high-performance non-blocking 10GbE switch like the Arista 7150S series. Client connectivity is simple just attach 2 SFP+ short range optical or compatible Twinax cables from the client to the switch. No complicated Link Aggregation Control Protocol (LACP) configuration is required or desired, SMB3 multichannel will automatically detect the IP addresses of both 10GbE interfaces on the client and load balance across each of the two interfaces on the dual-ported NIC. CLIENT DRIVER TUNING In the Advanced properties tab of the NIC driver, the Jumbo Packet setting should be set to the maximum value of 9000 MTU. Later in this document, we ll discuss the need for ensuring that all clients on the network segment dedicated to high performance NAS connectivity are set to an MTU of 9000. In the NIC setting for Maximum Number of RSS Queues, the value should be set to at least match the physical core count for the client s CPU. Also, ensure that the Receive Side Scaling setting is set to Enabled. The Receive Buffers setting should be set to 4096 The Transmit Buffers setting should be set to 16384. If using Windows Server 2012, make sure the Enable Virtual Machine Queues setting under Virtualization is set to disabled. If using Windows 8.x, you may need to set the interrupt moderation rate setting in the advanced Intel NIC driver properties to disabled in order to achieve optimal performance. The NIC should be installed in at least an 8X PCIe Gen 2 slot in the 4K workstation. Additionally, the workstation must have sufficient RAM, CPU, and GPU resources to support the playback of a high-data rate 4K stream. In the tests conducted to create this document, a Lenovo D30 workstation with dual 12 core Xeon CPU, 32 GB of RAM, and an NVIDIA Quadro K5000 GPU were employed. NETWORK CONSIDERATIONS In the configuration tested for this document, each client and each node in the Isilon cluster is dual-homed for Ethernet connectivity. A standard 1 GbE Ethernet connection with a gateway, DNS configuration, and standard 1500 MTU was used for all client and Isilon node 6

access to the Internet, Active Directory authentication, and so on. A Dedicated 10GbE network is employed for all media playback requirements. By employing a dedicated media network, much in the same way one would deploy a dedicated high- performance SAN network, 3 performance issues are easily mitigated: 1. Traffic interrupts and chatter from applications unrelated to the 4K workflow do not degrade client network performance. 2. All high-performance streaming media playback is isolated to a non-routed network segment with no latency from additional network hops. 3. The requirement for all 10GbE clients on the network to use MTU of 9000 rather than a standard MTU of 1500 are easily met only clients requiring high-performance 10GbE connectivity need to connect to the dedicated MTU 9000 LAN segment and each client on the MTU 9000 LAN segment is dual-homed with a primary 1 GbE network interface for all non-streaming traffic. ISILON REQUIREMENTS The X410 nodes should have dual 10GbE network interfaces configured for MTU 9000 and connected to the same LAN segment as the MTU 9000 10GbE interfaces of the 4K playback workstations. Each Isilon 10GbE interface is given it s own static IP address in the same subnet do not bond the 10GbE connections using LACP. On the Isilon server side, you will need a cluster with at least 3 X410 nodes for each client that will handle a 4K stream, assuming you are doing a full frame 4K stream at 24 fps. An additional 4K stream can be supported for every 3rd Isilon node a three node cluster supports one stream, a five node cluster supports two streams, and so on. Table 2 shows the bit rate for a full frame 4096X3112, 24 fps stream in contrast to some other 4K formats. FRAME SIZE COLOR DEPTH FRAMES PER SECOND 4096X1714 10bit RGB 24 674 3996X2160 10bit RGB 24 829 3656X2664 10bit RGB 24 935 3840X2160 10bit RGB 30 997 4096X2160 10bit RGB 24 850 4096X3072 10bit RGB 24 1,200 4096X3112 10bit RGB 24 1,200 Table 2. Table 1. Bitrates for Various Uncompressed 4K Formats MEGABITS PER SECOND NETWORK VALIDATION In order to validate that you have maximum 10GbE Connectivity between each 10GbE interface on your client and each 10GbE interface on the Isilon cluster, run a quick validation of network integrity using the open source iperf tool. On a client with iperf installed (client_shell) and a single Isilon node (Isilon_node), run the following test: isilon_node# iperf -s -w 2M -l 1M -N client_shell# iperf -c <node 10G IP> -w 2M -l 1M -N -t 30 (this measures "write speed" using a single iperf thread, and would represent the most that any single thread from an application could achieve) Then do: isilon_node# iperf -c <client 10G IP> -w 2M -l 1M -N -t 30 client_shell# iperf -s -w 2M -l 1M -N (this measures "read speed") This test will ensure that you are getting >9 Gigabit per second throughput between your client NIC and the Isilon NIC without performance degradation due to a network or client configuration issue. 7

FILE NAME PREFETCH To improve image sequence playback performance even further, Isilon has added a feature to pre-fetch sequentially named files such as DPX, Cineon, and EXR sequences into RAM cache as soon as the first file in the sequence is read by a client. To enable this sequential file name prefetch feature on a node in your cluster, you will need to add the following sysctl settings to the /etc/mcp/override/sysctl.conf file on the node that will handle client connections for sequential file playback. Please consult Isilon customer support before implementing any of the changes detailed below: isi.access.custom1.write.prealloc=1 isi.access.custom1.write.writealloc_blocks=0 isi.access.custom1.write.realloc_blocks=0 isi.access.custom1.fnprefetch.size_limit=67108864 isi.access.custom1.fnprefetch.l1_all=1 isi.access.custom1.fnprefetch.lookahead=9 isi.access.custom1.fnprefetch.fn_enabled=1 isi.access.custom1.coalescer.coal_falloc=1 isi.access.custom1.coalescer.cregion_max_forced=16 isi.access.custom1.coalescer.cregion_min_size=2097152 isi.access.custom1.prefetch.meta_pgs=8 isi.access.custom1.prefetch.l2_ssd_disable=0 isi.access.custom1.prefetch.l2_cluster_blocks=512 isi.access.custom1.prefetch.l2_window_blocks=8192 isi.access.custom1.prefetch.l1_cluster_blocks=0 isi.access.custom1.prefetch.l1_window_blocks=1024 isi.access.custom1.prefetch.offload=1 isi.access.custom1.prefetch.aread=0 isi.access.custom1.prefetch.read_cluster_limit=256 isi.access.custom1.read_batch=0 Once the changes to /etc/mcp/override/sysctl.conf are saved, run the following command to the directory containing the image sequences, assuming the path is /ifs/data/dpx in this example: isi set -R -l streaming /ifs/data/dpx This will set the data access pattern for the folder (recursively) to Streaming mode, increasing the number of spindles used to serve the stripes of streaming media files on OneFS. Once this command completes, run the following following command: isi set -R -a custom1 /ifs/data/dpx This will set the custom1 prefetch configuration defined in /etc/mcp/override/sysctl.conf for the directory in question. For more information about the modifying sysctl settings, please see Isilon KB article 89232. The custom1 prefetch configuration must be set on a per-node basis to any node that will serve 4K image sequences. Please note that it is best to dedicate specific nodes for 4K streaming, as you will not want to have competing client traffic attached to the node in question. Further, the stress in L1 cache created by the file name prefetch configuration could be disruptive to other workflows using L1 cache on the node in question. ASSIMILATE SCRATCH PLAY Dell EMC Isilon partner Assimilate have provided a free version of the SCRATCH Play universal media player on their website at http://www.assimilateinc.com/products/scratch-play. No special configuration is required to optimize SCRATCH play for streaming 4K playback over SMB3. Simply install the application, load a CONstruct, and import media from an Isilon SMB share via a node with the file name prefetch configuration. In order to verify playback of 4K content without dropping frames, simply press ctrl + F1 to bring up the playback statistics display. CONCLUSION 4K workflows are inevitable. The 1,200 MB per second storage throughput requirement for uncompressed 4K has historically required a high performance SAN or local attached storage. With the introduction of SMB 3.0 multi-channel support, Isilon can now handle single client streams that exceed the bandwidth limitation of a single 10GbE connection. Applications such as Assimilate SCRATCH that are compatible with the Windows 8 SMB 3.0 multi-channel implementation may now take advantage of SAN performance over NAS by automatically load-balancing a single SMB stream over dual 10GbE network connections. 8