Polymorphic Acceleration for Compute Intensive Applications as a Service

Transcription

1 Polymorphic Acceleration for Compute Intensive Applications as a Service By Dr. Samit Chaudhuri Chief Technology Officer, NVXL Technology, Inc. Deep learning, genomics, and transcoding are all compute-intensive technologies whose collective potential has long been hindered by the limited compute performance of conventional computing. These technologies encompass diverse applications ranging from image recognition, medical research and smart cities to selfdriving cars, database acceleration, broadcast streaming, financial analytics, earth sciences, meteorology and more. This paper explores the technology drivers and the challenges facing these compute-intensive technologies. It also touches on the history of computer performance and discusses the limitations of current offerings available in the public cloud. Finally, this paper will detail the benefits of employing a new acceleration platform that elastically scales and is built with off-the-shelf components, along with the diversity of applications suited to this ground-breaking solution. Technology Drivers & Challenges Deep Learning Deep Learning is an artificial intelligence function that is inspired by the inner workings of the human brain when processing data and creating patterns for use in decision making. Deep learning has neural networks support supervised or unsupervised learning. Deep learning models are loosely related to information processing and communication patterns in a biological nervous system, i.e., the human brain. The challenge with deep learning is that the powerful deep learning models require terabytes or petabytes of data to train them while addressing the hundreds of thousands of requests per second from billions of users with less than tens of millisecond response time. Deep learning has been around for decades but has never been able to make significant strides due to the industry s limitations in compute performance which have long hindered the process of managing and processing such huge amounts data. With enough compute power, deep learning applications should be able to quickly identify trends and patterns that would be otherwise difficult or too time-consuming to detect and take appropriate actions.

2 Genomics Genomics is branch of biotechnology that applies techniques of genetics and molecular biology to the genetic mapping and DNA sequencing of sets of genes or the complete genomes of selected organisms. The results are then organized in databases, and have applications in many fields including science, medicine and biology. The genome itself is one complete set of the genetic material in a cell. In humans, the genome is made up of around three billion nucleotide base pairs. Genomics sequencing enables individualized information about a patient s genetic makeup to help guide the type of treatment they receive. The data for just one human genome would require 200GB the storage capacity of a typical laptop. The immense datasets being developed present a challenge because of the size it becomes difficult to share and analyze the data. Transcoding Transcoding is essentially the conversion of language or information from one form of coded representation to another. Widely deployed in broadcasting, is the direct digital-to-digital conversion of one encoding to another, such as for movie data files (e.g., PAL, SECAM, NTSC), audio files (e.g., MP3, WAV), or character encoding (e.g., UTF-8, ISO/IEC 8859). Transferring media files continues to present challenges due to the sheer size of the files and the compute intensity of the conversion process. Thus, the quest to develop compression/ decompression algorithms (codecs) that could adequately transfer huge media files while requiring as little data bandwidth as possible continues. The History of Compute Performance From 1970 to 2008, and per Moore s Law, the industry experienced a 10,000-fold improvement in compute performance. From 2008 onward, the industry experienced just an 8x to 10x performance increase. This recent slowdown been problematic due to rapidly developing applications including deep learning which come with extensive datasets and require the use of computationally intensive machine learning models. See Fig 1. Fig 1. Hardware Speed Increasing Over Time. Source: ores_law.htm

3 Limitations of Current Solutions In deep learning, neural networks must be trained faster and more efficiently. These data hungry applications require more compute power to quickly identify trends and patterns that would be otherwise difficult or too time-consuming to detect and take appropriate actions. Genomics holds the promise of solving some of the world s most devastating diseases, but the cost and time have long been prohibitive. The first human genome took 13 long years to sequence and cost $3 billion dollars. 1 Today, this same process takes roughly 48 hours and costs between $3,000 and $5,000. Moving forward, researchers are striving for the process to be completed in mere minutes. In addition, to sample the large populations of DNA necessary to drive breakthrough medical advances, the cost must be reduced drastically. In transcoding, the proliferation of mobile phones and other edge devices has tremendously increased the number of media files being transferred or uploaded on the Internet. The growth in emerging technologies like AR and VR is going to increase traffic even more in the near future. This sheer volume of media files and needed compression is driving the need for additional compute power and efficiency. Currently, the standard solution for many of these applications is structured around silo-based acceleration. This approach encompasses any given number of requests (or users) being sent to algorithms of various size and compute needs. No one size fits all when running algorithms. However, all algorithms, regardless of size or compute needs, are run through a silo server which is driven by either CPU, or CPU and additional accelerators including GPU or FPGA. To handle the surge demand of various algorithms, each silo must run at full tilt at all times which is both inefficient and costly. Essentially, the hardware choice defines the algorithm grouping addressed by the server leaving the user with no flexibility. This has all adversely impacted productivity, efficiency and Total Cost of Ownership (TCO). NVXL Acceleration Platform for Polymorphic Acceleration NVXL has engineered a scalable approach to address the ever-increasing appetite for compute performance using a new technology: polymorphic acceleration. It is the only technology that is built on the general premise that a multitude of compute-intensive applications needs a multitude of computing architectures including CPU, FPGA, GPU, or a new breed of ASICs that are currently being invented. Hence, instead of building silos of specific computing architectures, data centers can use polymorphic acceleration to offer software-defined acceleration platforms in real time using a flexible accelerator pool that can easily be expanded and upgraded. Application performance is improved at a lower cost without the risk of betting on one particular device architecture. As the name suggests, polymorphic acceleration is all about providing a single interface to multiple ways of accelerating an application. It unifies and creates synergy among various aspects of computing that have formerly been considered in isolation. The following paragraphs describe these various aspects of polymorphic acceleration. From a programming point of view, polymorphic acceleration generalizes conventional heterogenous computing by providing a common software acceleration layer so that applications can be executed on a mix of device architectures including CPU, FPGA, and GPU. Moreover, users do not need to use separate software stacks to execute on GPU or FPGA devices. Future device architectures can also be added to the mix by adding architecture specific backends.

4 From an infrastructure point of view, polymorphic acceleration brings elastic scaling into acceleration by providing a software-defined mechanism to compose acceleration systems on the fly that includes compute, storage, and networking. The software-defined layer abstracts away details of the underlying hardware infrastructure which can be built with either disaggregated or hyper-converged appliances that are built with off-the-shelf components. With unifications enabled by polymorphic acceleration comes unparalleled flexibility. Users can deploy the software layer of polymorphic acceleration in an enterprise data center or on private cloud, and later dynamically extend it into public cloud to support bursts of peak demand or capacity. Since polymorphic acceleration enables seamless scaling using a variety of device architectures enables polymorphic acceleration to provide extreme performance on demand using off-the-shelf hardware. It is the only technology that service providers can use to allow users access tens of elastic ExaFLOPs of computing power across 1000s of compute devices at a dramatically reduced TCO. Fig 2.0 NVXL Acceleration Platform Compute Nodes PCIe/RDMA Accelerator Appliances Fig 2.0 illustrates a typical infrastructure on which NVXL software can be deployed to accelerate compute-intensive applications. This standard infrastructure consists of standard compute nodes that execute application clients, standard RDMA switches that provide fast networking, and accelerator appliances that are standard server chassis containing multiple PCIe Add-In Cards (AICs) to accelerate the computation kernels. The AICs are available from all major FPGA and GPU vendors. For low-power operation, NVXL also offers smaller form-factor (2.5 ) PCIe cards with FPGAs. The FlexAccel software stack of NVXL runs on this standard infrastructure and provides polymorphic acceleration through a common software acceleration layer. Domain specific frameworks, e.g. Caffe or TensorFlow for Deep Learning inference, Spark for analytics, or GATK for genomics tap into this acceleration layer through specific adapters. Depending on the requirement of the application, FlexAccel provisions the right number and types of CPU, FPGA, or GPU devices, compiles and optimizes the application for performance, and runs the application on the provisioned set of devices. FlexAccel also integrates to the orchestration infrastructure to coordinate the execution of multiple applications on the same infrastructure. The user does not need to consider the low-level details of the accelerators and the physical fabric that interconnects the accelerators. Instead FlexAccel examines the dataflow pattern of the application, appropriately streamlines and parallelizes the data movement, and then implements the parallel solution on the provisioned set of devices and the interconnection network connecting them. See Fig 3.0

5 COMPUTE ACCELERATION (TRADITIONAL) POLYMORPHIC ACCELERATION Data flow architecture using dynamically provisioned instances Workload Workload 1 2 Allocation constrained to resources 1 2 Software driven allocation Pooled resources 3 3 Harnessing the power and flexibility of FPGAs denotes FPGAs Fig 3.0 Diversity of Applications Database Acceleration (NoSQL/ SQL) ML/DL Inference, ML/DL Advanced Algorithms, Big Data Analytics Media Processing (Transcoding) Big Data (Oil/Gas), National Security, Genomics Sequencing Fig 4.0 Diversity of Applications Addressed by NVXL Polymorphic Acceleration NVXL s technology accelerates compute-intensive data flow graphs, algorithms, or functions for CNN/DNN, genomics, transcoding, encryption, compression, database query, BLAS, and others. NVXL Acceleration Layer supports both NVXLcreated and partner-created libraries: CaffeNet, AlexNet, VGGNet, ResNet, GoogLeNet, U-Net, SqueezeNet, SSD, YOLO2, RNN, LSTM, GRU, PairHMM, and H.265 (NoSQL acceleration support will be available in the future). NVXL Acceleration Layer also provides efficient integration with industry-leading deep learning frameworks including Caffe, Caffe2, TensorFlow, GATK, FFMPEG, GATK, and Apache Spark. See Fig 4.0. Hardware Solution NVXL s FPGA-based architecture in the DS-1 UAS is far superior at processing algorithmic operations than commodity-based CPUs which execute strings of commands on a fixed number of individual cores. A single FPGA transforms algorithmic functions, such as in deep learning, into massively parallel logic elements, efficiently executing operations at exceptional speeds and with high accuracy. Twenty-four DS-1 servers, with up to 480 FPGA-based plug-in configurable accelerator modules or PCA modules, can be installed in a single industry-standard 19-inch rack. Fully configured and networked DS-1 racks can support thousands of PCA modules to meet the most demanding workloads.

6 Real-time Scaling Traditionally, many applications have had to rely on traditional scaling which is structured simply and designed around maximum demand. This method is designed to have the application running all the time and at maximum capacity, which, as previously noted, is inefficient. Time-based scaling represents an improvement over traditional scaling because it is designed for maximum use during peak times of the day, but it still has a fairly high TCO. Real time scaling, however, is designed for minimum demand. Using this approach, different algorithms can be dynamically loaded and provisioned so that the system is off doing other tasks until needed. This keeps utilization low and offers a much higher levels of efficiency and TCO allowing spare compute power to be Fig 5.0 Traditional, Time-Based and Real-Time Scaling Approaches Computing capacity Workload Peak Day Time Non-Peak Day Time Night Time Real-Time Demand Scaling Traditional Designed for max. demand Coarse granular provisioning Inefficient utilization Highest TCO Time-based scaling Designed for region max. demand Granular provisioning Less than optimal utilization Highest TCO Real-time scaling Designed for min. demand Fine granular provisioning Efficient utilization Highly elastic Massively parallel Better latency Cloud-scale Best TCO Fig 6.0 Vision Cloud for a Smart City Application In deep learning, the deep models such as GoogLeNet and ResNet 18 have very demanding compute requirements. For example, GoogLeNet requires 1.43GB operations and has 7M parameters. ResNet 18 requires 3.9GB operations and 25.5M parameters. They also require hardware that can accommodate these types of complex compute requirements. If this processing is done using traditional GPS, it would take a long time to complete whereas with the appropriate hardware acceleration, this time can be reduced dramatically. Reference Fig 6.0 to see an example of how a deep learning application, in this example, vision cloud for a Smart City application, is structured using NVXL s solution.

7 A genomics pipeline such as GATK, uses compute intensive algorithms such as Smith-Waterman, PairHMM (Pairwise Hidden Markov Model), and file compression and decompression. If processing is done using traditional CPUs, it would take a long time to complete whereas with the appropriate hardware acceleration, this time can be reduced dramatically. Reference Fig 7.0, showing human genome sequencing done in less than one minute using the NVXL platform. Fig 7.0 Real-Time Accelerated Live Media Transcoding Compute acceleration is an integral part of streaming media (transcoding) where machine learning, deep learning, and high-performance are crucial for behavior analysis, real-time transcoding, and content delivery networks. See Fig 8.0, showing parallel acceleration of multiple compute-intensive algorithms for enhanced performance. Fig 8.0 Transcoding Acceleration

8 Summary As high-performance computing in the cloud continues to evolve, deep learning, genomics and transcoding will all strive to move forward in the quest to solve highly complex problems. In the process, these technologies will become even more integrated into every aspect of our lives. NVXL is leading the way, with a revolutionary polymorphic acceleration platform that enables disaggregated, heterogeneous compute acceleration with unpreceded scalability. The NVXL acceleration platform-based solution is designed to accelerate and scale compute performance and power to new levels while providing a new paradigm in total cost of ownership. Supporting Resources Reference: