FastForward I/O and Storage: ACG 5.8 Demonstration

Transcription

1 FastForward I/O and Storage: ACG 5.8 Demonstration Jaewook Yu, Arnab Paul, Kyle Ambert Intel Labs September, 2013 NOTICE: THIS MANUSCRIPT HAS BEEN AUTHORED BY INTEL UNDER ITS SUBCONTRACT WITH LAWRENCE LIVERMORE NATIONAL SECURITY, LLC WHO IS THE OPERATOR AND MANAGER OF LAWRENCE LIVERMORE NATIONAL LABORATORY UNDER CONTRACT NO. DE-AC52-07NA27344 WITH THE U.S. DEPARTMENT OF ENERGY. THE UNITED STATES GOVERNMENT RETAINS AND THE PUBLISHER, BY ACCEPTING THE ARTICLE OF PUBLICATION, ACKNOWLEDGES THAT THE UNITED STATES GOVERNMENT RETAINS A NON-EXCLUSIVE, PAID-UP, IRREVOCABLE, WORLD-WIDE LICENSE TO PUBLISH OR REPRODUCE THE PUBLISHED FORM OF THIS MANUSCRIPT, OR ALLOW OTHERS TO DO SO, FOR UNITED STATES GOVERNMENT PURPOSES. THE VIEWS AND OPINIONS OF AUTHORS EXPRESSED HEREIN DO NOT NECESSARILY REFLECT THOSE OF THE UNITED STATES GOVERNMENT OR LAWRENCE LIVERMORE NATIONAL SECURITY, LLC. 1

2 Overview Background and Demonstration objectives Demo Environment Demo1: Realistic Power-law graph generation Demo2: Graph Analytics over HDF5 graphs (GraphLab and HAL integration) Conclusion and discussion on upcoming items

3 Background: Big Data HPC Bridge Architecture and Scope of the present milestone Raw Data Big Data HPC Bridge Results ACG Ingress on a Hadoop Cluster HPC Node Node Node Node Node Node Node Node ACG Ingress Processing Computation Kernel HDF5 Adaptation Layer HDF5 HDF5 Adaptation Layer HDF5 Graph (Partitions) and Network Information Represented in HDF5

4 Objectives From SOW : Full HAL Demonstration and report Develop HAL for ACG-ingress and producing partitioned graphs (hadoop) milestone 4.6 Develop HAL to interface with graph computational kernel Computational kernel loading partitions on separate nodes and performing basic graph analytics HAL performance characterization (initial) Generation of realistic power-law graphs

5 Demonstration : Graph representation and partitioning Environment Cluster spec 16 nodes Each node with 128G RAM 8 quad-core processors Still on POSIX (not on parallel file system) SATA 6Gb/s 7200RPM 16MB Seagate Barracuda ST500DM002 Not using the SSDs for these experiments

6 Demonstration 1: Power law Graph generation

7 Demonstration 1: Power-law graphs Graph generation at a very large scale; Incremental graph I/O Near perfect power-law Speed ~ 10 5 edges/sec/machine over POSIX, it will be different over exascale stack. Demo scale : 32 m vertices, ~500m edges max. scale tried so far - graphs with ~ 0.5 triilion edges

8 Recursive Matrix Generation v Computations u 32m p u,v 1m vertices ~1 trillion possible edges 32m vertices ~ 1000 trillion possible edges Reference: D Chakabarti, Y Zhan, and C. Faloutsos R-MAT: A Recursive Model for Graph Mining, in Graph Mining: Laws, Tools, and Case Studies, Morgan & Claypool Publishers (October, 2012)

9 Comparative picture: Earlier and After Near-perfect power law distribution

10 Demonstration 2: Graphlab and HAL integration

11 Demonstrartion-2 details GraphLab (computational kernel) loading a partitioned graph from HDF5 (16 partitions for 16 nodes) 48 million vertices CNs ~ 0.5 billion edges (~165m edges/node) Disks Compute pagerank for every vertex till convergence criteria met Output Pagerank - currently storing output on hdfs for ease of post-processing

12 HAL performance overhead for generating power law graphs (CPU cycles) libhal.so skggen libhdf5 libgsl libc libstdc M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16

13 HAL performance overhead for Pagerank computation (CPU cycles) pagerank_app libpthread libstdc++ libhdf5 libc libhal.so 10 0 M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16

14 HAL performance overhead for Pagerank computation (CPU cycles) libpthread libstdc++ libhdf5 libc libhal.so 2 0 M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16

15 Conclusions and upcoming work Conclusions Covered Demo criteria 5.8: Generating realistic power-law graphs Complete HDF5 adaptation layer development, so that the computation kernel can load data from HDF5 Upcoming work Computational kernel Interface with the exascale stack (through HDF5) Detailed Benchmark and comparisons with other platforms Full scale Graph Analytics on Exascale Stack

16 Backup SOW statement The Subcontractor shall demonstrate the functionality and streaming efficiency of the prototype ACG abstraction layer when performing bulk and incremental graph I/O. A benchmark report shall be provided which measures performance and scalability.