Sunrise or Sunset: Exploring the Design Space of Big Data So7ware Stacks

Transcription

1 Sunrise or Sunset: Exploring the Design Space of Big Data So7ware Stacks Panel PresentaAon at HPBDC 17 by Dhabaleswar K. (DK) Panda The Ohio State University E- mail: state.edu h<p:// state.edu/~panda

2 HPBDC 17 Panel 2 Q1: Are Big Data So7ware Stacks Mature or Not? Big Data soeware stacks like Hadoop, Spark and Memcached have been there for mulkple years Hadoop 11 years (Apache Hadoop.1. released on April, 26) Spark 5 years (Apache Spark.5.1 released on June, 212) Memcached 14 years (IniKal release of Memcached on May 22, 23) Increasingly being used in produckon environments OpKmized for commodity clusters with Ethernet and TCP/IP interface Not yet able to take full advantage of modern cluster and/or HPC technologies

3 HPBDC 17 Panel 3 Data Management and Processing on Modern Clusters SubstanKal impact on designing and uklizing data management and processing systems in mulkple Kers Front- end data accessing and serving (Online) Memcached + DB (e.g. MySQL), HBase Back- end data analykcs (Offline) HDFS, MapReduce, Spark Front-end Tier Back-end Tier Internet Data Accessing and Serving Web Server Web Server Web Server Memcached + DB Memcached (MySQL) + DB Memcached (MySQL) + DB (MySQL) NoSQL DB (HBase) NoSQL DB (HBase) NoSQL DB (HBase) Data Analytics Apps/Jobs MapReduce Spark HDFS

4 HPBDC 17 Panel 4 Who Are Using Hadoop? Focuses on large data and data analysis Hadoop (e.g. HDFS, MapReduce, RPC, HBase) environment is gaining a lot of momentum h<p://wiki.apache.org/hadoop/poweredby

5 HPBDC 17 Panel 5 Spark Ecosystem Generalize MapReduce to support new apps in same engine Two Key ObservaKons General task support with DAG MulK- stage and interackve apps require faster data sharing across parallel jobs BlinkDB Spark SQL Spark Streaming (real-time) GraphX (graph) MLlib (Machine Learning) Caffe, TensorFlow, BigDL, etc. (Deep Learning) Spark Standalone Apache Mesos YARN

6 Who Are Using Spark? Focuses on large data and data analysis with in- memory techniques Apache Spark is gaining a lot of momentum h<p://spark.apache.org/powered- by.html Network Based CompuAng Laboratory HPBDC 17 Panel 6

7 HPBDC 17 Panel 7 Q2: What are the Main Driving forces for New- generaaon Big Data So7ware Stacks?

8 HPBDC 17 Panel 8 Increasing Usage of HPC, Big Data and Deep Learning HPC (MPI, RDMA, Lustre, etc.) Big Data (Hadoop, Spark, HBase, Memcached, etc.) Deep Learning (Caffe, TensorFlow, BigDL, etc.) Convergence of HPC, Big Data, and Deep Learning!!!

9 How Can HPC Clusters with High- Performance Interconnect and Storage Architectures Benefit Big Data and Deep Learning ApplicaAons? Network Based CompuAng Laboratory HPBDC 17 Panel 9 Can the bo<lenecks be alleviated with new designs by taking advantage of HPC technologies? What are the major bo<lenecks in current Big Data processing and Deep Learning middleware (e.g. Hadoop, Spark)? Can RDMA- enabled high- performance interconnects benefit Big Data processing and Deep Learning? Can HPC Clusters with high- performance storage systems (e.g. SSD, parallel file systems) benefit Big Data and Deep Learning applicakons? How much performance benefits can be achieved through enhanced designs? How to design benchmarks for evaluakng the performance of Big Data and Deep Learning middleware on HPC clusters? Bring HPC, Big Data processing, and Deep Learning into a convergent trajectory!

10 HPBDC 17 Panel 1 Can We Run Big Data and Deep Learning Jobs on ExisAng HPC Infrastructure?

11 HPBDC 17 Panel 11 Can We Run Big Data and Deep Learning Jobs on ExisAng HPC Infrastructure?

12 HPBDC 17 Panel 12 Can We Run Big Data and Deep Learning Jobs on ExisAng HPC Infrastructure?

13 HPBDC 17 Panel 13 Can We Run Big Data and Deep Learning Jobs on ExisAng HPC Infrastructure?

14 HPBDC 17 Panel 14 Q3: What Chances are Provided for the Academia CommuniAes in Exploring the Design Spaces of Big Data So7ware Stacks?

15 HPBDC 17 Panel 15 Designing CommunicaAon and I/O Libraries for Big Data Systems: Challenges ApplicaAons Big Data Middleware (HDFS, MapReduce, HBase, Spark, grpc/tensorflow, and Memcached) Programming Models (Sockets) Benchmarks RDMA Protocols Point- to- Point CommunicaAon CommunicaAon and I/O Library Threaded Models and SynchronizaAon VirtualizaAon (SR- IOV) I/O and File Systems Networking Technologies (InfiniBand, 1/1/4/1 GigE and Intelligent NICs) QoS & Fault Tolerance Commodity CompuAng System Architectures (MulA- and Many- core architectures and accelerators) Performance Tuning Storage Technologies (HDD, SSD, NVM, and NVMe- SSD)

16 HPBDC 17 Panel 16 The High- Performance Big Data (HiBD) Project RDMA for Apache Spark RDMA for Apache Hadoop 2.x (RDMA- Hadoop- 2.x) Plugins for Apache, Hortonworks (HDP) and Cloudera (CDH) Hadoop distribukons RDMA for Apache HBase RDMA for Memcached (RDMA- Memcached) RDMA for Apache Hadoop 1.x (RDMA- Hadoop) OSU HiBD- Benchmarks (OHB) HDFS, Memcached, HBase, and Spark Micro- benchmarks hip://hibd.cse.ohio- state.edu Users Base: 23 organizakons from 3 countries More than 21,8 downloads from the project site Available for InfiniBand and RoCE Also run on Ethernet

17 RDMA for Apache Hadoop 2.x DistribuAon High- Performance Design of Hadoop over RDMA- enabled Interconnects High performance RDMA- enhanced design with nakve InfiniBand and RoCE support at the verbs- level for HDFS, MapReduce, and RPC components Enhanced HDFS with in- memory and heterogeneous storage High performance design of MapReduce over Lustre Memcached- based burst buffer for MapReduce over Lustre- integrated HDFS (HHH- L- BB mode) Plugin- based architecture supporkng RDMA- based designs for Apache Hadoop, CDH and HDP Easily configurable for different running modes (HHH, HHH- M, HHH- L, HHH- L- BB, and MapReduce over Lustre) and different protocols (nakve InfiniBand, RoCE, and IPoIB) Current release: 1.1. Based on Apache Hadoop Compliant with Apache Hadoop 2.7.1, HDP and CDH APIs and applicakons Tested with Mellanox InfiniBand adapters (DDR, QDR, FDR, and EDR) RoCE support with Mellanox adapters Various mulk- core plarorms hip://hibd.cse.ohio- state.edu Different file systems with disks and SSDs and Lustre Network Based CompuAng Laboratory HPBDC 17 Panel 17

18 Different Modes of RDMA for Apache Hadoop 2.x HHH: Heterogeneous storage devices with hybrid replicakon schemes are supported in this mode of operakon to have be<er fault- tolerance as well as performance. This mode is enabled by default in the package. HHH- M: A high- performance in- memory based setup has been introduced in this package that can be uklized to perform all I/O operakons in- memory and obtain as much performance benefit as possible. HHH- L: With parallel file systems integrated, HHH- L mode can take advantage of the Lustre available in the cluster. HHH- L- BB: This mode deploys a Memcached- based burst buffer system to reduce the bandwidth bo<leneck of shared file system access. The burst buffer design is hosted by Memcached servers, each of which has a local SSD. MapReduce over Lustre, with/without local disks: Besides, HDFS based solukons, this package also provides support to run MapReduce jobs on top of Lustre alone. Here, two different modes are introduced: with local disks and without local disks. Running with Slurm and PBS: Supports deploying RDMA for Apache Hadoop 2.x with Slurm and PBS in different running modes (HHH, HHH- M, HHH- L, and MapReduce over Lustre). Network Based CompuAng Laboratory HPBDC 17 Panel 18

19 HPBDC 17 Panel 19 RDMA for Apache Spark DistribuAon High- Performance Design of Spark over RDMA- enabled Interconnects High performance RDMA- enhanced design with nakve InfiniBand and RoCE support at the verbs- level for Spark RDMA- based data shuffle and SEDA- based shuffle architecture Support pre- conneckon, on- demand conneckon, and conneckon sharing Non- blocking and chunk- based data transfer Off- JVM- heap buffer management Easily configurable for different protocols (nakve InfiniBand, RoCE, and IPoIB) Current release:.9.4 Based on Apache Spark 2.1. Tested with Mellanox InfiniBand adapters (DDR, QDR, FDR, and EDR) RoCE support with Mellanox adapters Various mulk- core plarorms RAM disks, SSDs, and HDD hip://hibd.cse.ohio- state.edu

20 HiBD Packages on SDSC Comet and Chameleon Cloud RDMA for Apache Hadoop 2.x and RDMA for Apache Spark are installed and available on SDSC Comet. Examples for various modes of usage are available in: RDMA for Apache Hadoop 2.x: /share/apps/examples/hadoop RDMA for Apache Spark: /share/apps/examples/spark/ Please (reference Comet as the machine, and SDSC as the site) if you have any further queskons about usage and configurakon. RDMA for Apache Hadoop is also available on Chameleon Cloud as an appliance h<ps:// M. TaAneni, X. Lu, D. J. Choi, A. Majumdar, and D. K. Panda, Experiences and Benefits of Running RDMA Hadoop and Spark on SDSC Comet, XSEDE 16, July 216 Network Based CompuAng Laboratory HPBDC 17 Panel 2

21 Performance Numbers of RDMA for Apache Hadoop 2.x RandomWriter & TeraGen in OSU- RI2 (EDR) ExecuAon Time (s) Network Based CompuAng Laboratory 8 IPoIB (EDR) Reduced by 3x IPoIB (EDR) Reduced by 4x 7 OSU- IB (EDR) 6 OSU- IB (EDR) Data Size (GB) Data Size (GB) RandomWriter TeraGen Cluster with 8 Nodes with a total of 64 maps RandomWriter 3x improvement over IPoIB for 8-16 GB file size ExecuAon Time (s) TeraGen 4x improvement over IPoIB for 8-24 GB file size HPBDC 17 Panel 21

22 ExecuAon Time (s) Performance Numbers of RDMA for Apache Hadoop 2.x Sort & TeraSort in OSU- RI2 (EDR) Sort IPoIB (EDR) OSU- IB (EDR) Network Based CompuAng Laboratory Data Size (GB) Sort Cluster with 8 Nodes with a total of 64 maps and 14 reduces 61% improvement over IPoIB for 8-16 GB data Reduced by 61% ExecuAon Time (s) TeraSort IPoIB (EDR) OSU- IB (EDR) Data Size (GB) TeraSort Cluster with 8 Nodes with a total of 64 maps and 32 reduces 18% improvement over IPoIB for 8-24 GB data Reduced by 18% HPBDC 17 Panel 22

23 Design Overview of Spark with RDMA Apache Spark Benchmarks/ApplicaAons/Libraries/Frameworks Neiy Server Spark Core Shuffle Manager (Sort, Hash, Tungsten- Sort) Block Transfer Service (Neiy, NIO, RDMA- Plugin) NIO Server Java Socket Interface RDMA Server Neiy Client NIO Client RDMA Client Java NaAve Interface (JNI) NaAve RDMA- based Comm. Engine Design Features RDMA based shuffle plugin SEDA- based architecture Dynamic conneckon management and sharing Non- blocking data transfer Off- JVM- heap buffer management InfiniBand/RoCE support 1/1/4/1 GigE, IPoIB Network RDMA Capable Networks (IB, iwarp, RoCE..) Enables high performance RDMA communicakon, while supporkng tradikonal socket interface JNI Layer bridges Scala based Spark with communicakon library wri<en in nakve code X. Lu, M. W. Rahman, N. Islam, D. Shankar, and D. K. Panda, AcceleraAng Spark with RDMA for Big Data Processing: Early Experiences, Int'l Symposium on High Performance Interconnects (HotI'14), August 214 X. Lu, D. Shankar, S. Gugnani, and D. K. Panda, High- Performance Design of Apache Spark with RDMA and Its Benefits on Various Workloads, IEEE BigData 16, Dec Network Based CompuAng Laboratory HPBDC 17 Panel 23

24 HPBDC 17 Panel 24 Performance EvaluaAon on SDSC Comet SortBy/GroupBy Time (sec) IPoIB 8% RDMA Data Size (GB) Time (sec) 64 Worker Nodes, 1536 cores, SortByTest Total Time 64 Worker Nodes, 1536 cores, GroupByTest Total Time InfiniBand FDR, SSD, 64 Worker Nodes, 1536 Cores, (1536M 1536R) RDMA vs. IPoIB with 1536 concurrent tasks, single SSD per node. SortBy: Total Kme reduced by up to 8% over IPoIB (56Gbps) GroupBy: Total Kme reduced by up to 74% over IPoIB (56Gbps) IPoIB 74% RDMA Data Size (GB)

25 HPBDC 17 Panel 25 Performance EvaluaAon on SDSC Comet HiBench PageRank Time (sec) IPoIB 37% RDMA Huge BigData GiganKc Data Size (GB) Time (sec) 32 Worker Nodes, 768 cores, PageRank Total Time 64 Worker Nodes, 1536 cores, PageRank Total Time InfiniBand FDR, SSD, 32/64 Worker Nodes, 768/1536 Cores, (768/1536M 768/1536R) RDMA vs. IPoIB with 768/1536 concurrent tasks, single SSD per node. 32 nodes/768 cores: Total Kme reduced by 37% over IPoIB (56Gbps) 64 nodes/1536 cores: Total Kme reduced by 43% over IPoIB (56Gbps) IPoIB 43% RDMA Huge BigData GiganKc Data Size (GB)

26 HPBDC 17 Panel 26 EvaluaAon with BigDL on RDMA- Spark One Epoch Time (sec) IPoIB RDMA 4.58x Number of cores VGG training model on the CIFAR- 1 dataset Evaluated on SDSC Comet supercomputer IniKal Results: RDMA- based Spark outperforms default Spark over IPoIB by a factor of 4.58x

27 Design Overview of NVM and RDMA- aware HDFS (NVFS) Network Based CompuAng Laboratory HPBDC 17 Panel 27 ApplicaAons and Benchmarks Hadoop MapReduce Spark HBase RDMA Receiver RDMA Sender Co- Design (Cost- EffecAveness, Use- case) NVM and RDMA- aware HDFS (NVFS) DataNode RDMA Writer/Reader DFSClient Replicator RDMA Receiver NVFS - BlkIO NVM NVFS- MemIO SSD SSD SSD Design Features RDMA over NVM HDFS I/O with NVM Block Access Memory Access Hybrid design NVM with SSD as a hybrid storage for HDFS I/O Co- Design with Spark and HBase Cost- effeckveness Use- case N. S. Islam, M. W. Rahman, X. Lu, and D. K. Panda, High Performance Design for HDFS with Byte- Addressability of NVM and RDMA, 24th InternaAonal Conference on SupercompuAng (ICS), June 216

28 EvaluaAon with Hadoop MapReduce Average Throughput (MBps) 35 HDFS (56Gbps) 3 1.2x NVFS- BlkIO (56Gbps) 25 NVFS- MemIO (56Gbps) x 5 Write Read SDSC Comet (32 nodes) HDFS Network Based CompuAng Laboratory Average Throughput (MBps) TestDFSIO HDFS (56Gbps) 2x NVFS- BlkIO (56Gbps) NVFS- MemIO (56Gbps) 4x Write Read OSU Nowlab (4 nodes) TestDFSIO on SDSC Comet (32 nodes) TestDFSIO on OSU Nowlab (4 nodes) Write: NVFS- MemIO gains by 4x over HDFS Write: NVFS- MemIO gains by 4x over HDFS Read: NVFS- MemIO gains by 1.2x over Read: NVFS- MemIO gains by 2x over HDFS HPBDC 17 Panel 28

29 Overview of RDMA- Hadoop- Virt Architecture Hadoop Common Virtual Machines Big Data Applications CloudBurst MR-MS Polygraph Others Topology Detection Module MapReduce Map Task Scheduling Policy Extension YARN HDFS HBase Container Allocation Policy Extension Virtualization Aware Block Management Containers Others Bare-Metal nodes VirtualizaKon- aware modules in all the four main Hadoop components: HDFS: VirtualizaKon- aware Block Management to improve fault- tolerance YARN: Extensions to Container AllocaKon Policy to reduce network traffic MapReduce: Extensions to Map Task Scheduling Policy to reduce network traffic Hadoop Common: Topology DetecKon Module for automakc topology deteckon CommunicaKons in HDFS, MapReduce, and RPC go through RDMA- based designs over SR- IOV enabled InfiniBand S. Gugnani, X. Lu, D. K. Panda. Designing VirtualizaAon- aware and AutomaAc Topology DetecAon Schemes for AcceleraAng Hadoop on SR- IOV- enabled Clouds. CloudCom, 216. Network Based CompuAng Laboratory HPBDC 17 Panel 29

30 HPBDC 17 Panel 3 EvaluaAon with ApplicaAons CloudBurst Self- Join EXECUTION TIME RDMA- Hadoop Default Mode RDMA- Hadoop- Virt Distributed Mode 3% reduckon EXECUTION TIME RDMA- Hadoop Default Mode RDMA- Hadoop- Virt Distributed Mode 55% reduckon 14% and 24% improvement with Default Mode for CloudBurst and Self- Join 3% and 55% improvement with Distributed Mode for CloudBurst and Self- Join

31 Deep Learning: New Challenges for MPI RunAmes Deep Learning frameworks are a different game altogether Unusually large message sizes (order of megabytes) Most communicakon based on GPU buffers How to address these newer requirements? GPU- specific CommunicaKon Libraries (NCCL) NVidia's NCCL library provides inter- GPU communicakon 1 2 Internode Comm. (Knomial) 3 4 Intranode Comm. (NCCL Ring) PLX CPU Ring Direction PLX CUDA- Aware MPI (MVAPICH2- GDR) Provides support for GPU- based communicakon Can we exploit CUDA- Aware MPI and NCCL to support Deep Learning applicakons? Hierarchical CommunicaAon (Knomial + NCCL ring) Network Based CompuAng Laboratory HPBDC 17 Panel 31

32 Efficient Broadcast: MVAPICH2- GDR and NCCL NCCL has some limitakons Only works for a single node, thus, no scale- out on mulkple nodes DegradaKon across IOH (socket) for scale- up (within a node) We propose opkmized MPI_Bcast CommunicaKon of very large GPU buffers (order of megabytes) Scale- out on large number of dense mulk- GPU nodes Hierarchical CommunicaKon that efficiently exploits: CUDA- Aware MPI_Bcast in MV2- GDR 2 NCCL Broadcast primikve 1 Efficient Large Message Broadcast using NCCL and CUDA- Aware MPI for Deep Learning, A. Awan, K. Hamidouche, A. Venkatesh, and D. K. Panda, Number of GPUs The 23rd European MPI Users' Group MeeAng (EuroMPI 16), Sep 216 [Best Paper Runner- Up] Performance Benefits: Microso7 CNTK DL framework (25% avg. improvement ) Network Based CompuAng Laboratory HPBDC 17 Panel 32 Time (seconds) 3 Performance Benefits: OSU Micro- benchmarks MV2- GDR MV2- GDR- Opt 2.2X

33 Large Message OpAmized CollecAves for Deep Learning MV2- GDR provides opkmized colleckves for large message sizes OpKmized Reduce, Allreduce, and Bcast Good scaling with large number of GPUs Available with MVAPICH2- GDR 2.2GA Network Based CompuAng Laboratory Latency (ms) Latency (ms) Latency (ms) Reduce 192 GPUs Message Size (MB) Allreduce 64 GPUs Message Size (MB) Bcast 64 GPUs Latency (ms) Latency (ms) Latency (ms) 2 1 Message Size (MB) No. of GPUs HPBDC 17 Panel Reduce 64 MB No. of GPUs Allreduce MB No. of GPUs Bcast 128 MB

34 OSU- Caffe: Scalable Deep Learning Caffe : A flexible and layered Deep Learning framework. Benefits and Weaknesses MulK- GPU Training within a single node Performance degradakon for GPUs across different sockets Limited Scale- out OSU- Caffe: MPI- based Parallel Training Enable Scale- up (within a node) and Scale- out (across mulk- GPU nodes) network on ImageNet dataset A. A. Awan, K. Hamidouche, J. Hashmi, and D. K. Panda, S- Caffe: Co- designing MPI RunAmes and Caffe for Scalable Deep Learning on Modern GPU Clusters, PPoPP, Sep 217 OSU- Caffe is publicly available from: h<p://hidl.cse.ohio- state.edu Network Based CompuAng Laboratory Training Time (seconds) GoogLeNet (ImageNet) on 128 GPUs Invalid use case No. of GPUs Caffe OSU- Caffe (124) OSU- Caffe (248) HPBDC 17 Panel 34

35 HPBDC 17 Panel 35 Open Challenges in Designing CommunicaAon and I/O Middleware for High- Performance Big Data Processing High- Performance designs for Big Data middleware NVM- aware communicakon and I/O schemes for Big Data SATA- /PCIe- /NVMe- SSD support High- Bandwidth Memory support Threaded Models and SynchronizaKon Locality- aware designs Fault- tolerance/resiliency MigraKon support with virtual machines Data replicakon Efficient data access and placement policies Efficient task scheduling Fast deployment and automakc configurakons on Clouds OpKmizaKon for Deep Learning applicakons

36 HPBDC 17 Panel 36 Sunrise or Sunset of Big Data So7ware? Assuming 6: am as sunrise and 6: pm as sunset, We are at 8: am.

37 HPBDC 17 Panel 37 Thank You! state.edu hip:// state.edu/~panda Network- Based CompuKng Laboratory h<p://nowlab.cse.ohio- state.edu/ The High- Performance Big Data Project h<p://hibd.cse.ohio- state.edu/