DRepl. Optimizing Access to Application Data for Analysis and Visualization. Latchesar Ionkov Michael Lang LANL. Carlos Maltzahn UCSC

Transcription

1 DRepl Optimizing Access to Application Data for Analysis and Visualization Latchesar Ionkov Michael Lang LANL Carlos Maltzahn UCSC

2 HPC Cluster Desktop Desktop Desktop Desktop Head Node FS CN1 CN2 CNk IO1 FS CNk+1 CNk+2 CNl IO2 FS CNl+1 CNl+2 CNm IO3 FS CNm+1 CNm+2 CNn IOp FS

3 Data Storage Data stored in files Many applications use legacy formats Data is stored in format, convenient for the producer In-situ and in-transit data analysis slow

4 Objective Decouple storage data layout from application data layout(s) Make replicas with different data layouts Each application working with the data can use a layout that is optimized for it Allow both materialized (on-storage) and onthe-fly data layouts

5 Design Definitions Dataset -- abstract data model Views -- how applications see the data Replicas -- how the data is stored Provision of an easy way to express how data is used by the applications View View Dataset Replica Replica Replica

6 Example pressure=5.1 temp=33.1 density=0.4 N M psim Dataset pressure=5.1 temp=33.1 density=0.4 N M psim N M pressure N M temp N M density N M pressure N M temp N M density pressure=5.1 density=0.4 N M psim View 1 View 2 Replica 1 Replica 2

7 DRepl Dataset Description DRepl Conversion Map Dataset Language dataset { var p struct { a, b, c float32 view default { var p = p view viz { var pa { a = p var pba { b, a = p Parser T viz: field T viz: S viz: field field viz S viz: T default: default field field field S viz: S default: S default: S default: Parser replica default { view default replica viz { view viz DReplFS Replication Engine Replication Engine File Server OS Replicas Replicas Replicas Vizualization Simulation

8 Configurations Embedded Separate Node 1 Application DRepl Parallel FS Node 1 Application DRepl Parallel FS Replica 1 Replica 1 Node 2 Application DRepl Node 2 Application DRepl Replica 2 Replica 2 Node N Application DRepl Node N Application DRepl Burst Buffer Node 1 Application Burst Buffer Node Parallel FS Node 2 Application DRepl Replica 1 Node N Application Replica 3 Replica 2

9 Dataset Language Syntax Similar to C, C++, Java Dataset define data types (structs, arrays) define named data of the types View(s) define substructs and subarrays define named data based on the dataset data Replica(s)

10 Dataset Language Primary types - int8, int16, int32, int64, float32, float64, stringn Structs struct { a, b, c float64 Multidimensional arrays [50,40,21] Point Custom types type int64 Point Arithmetic expressions in the subarray definitions a[i*3, j + 2] = aa[j, i - 1] Support for different array orders -- rowmajor, row-minor, in future Hilbert and z- order

11 Language Example dataset { const N = 500 type Data struct { a, b, c float32 var data [N]Data view array-of-structs { var ds = data view struct-of-arrays { var a[i]{a = data[i] var b[i]{b = data[i] var c[i]{c = data[i] view ab rowmajor { var ab[i]{a,b = data[i] replica array-of-structs { view array-of-structs replica struct-of-arrays { view struct-of-arrays replica other { view array-of-structs view ab

12 Subarray Examples dataset { const N = 500 const M = 200 var data [N, M]float32 view v { // flip dimensions var flip[i,j] = data[j,i] // middle row var mr[i] = data[n/2, i] // each third element var te[i, j] = data[i*3, j*3]

13 DReplFS DReplFS Represent the application data formats (views) as virtual files Sim1 Viz1 ABC BAC B A1 Stored data formats (replicas) -- collection of replicas CD Sim2 ABCD AB CD

14 Transformation Rules pa T 0000 field a S 0000 dataset { var p struct { a, b, c float32 view default { var p = p viz pba T 0004 field a field b S 0008 S 0004 S 0000 view viz { var pa { a = p var pba { b, a = p p T 0000 field a field b field c S 0004 S 0008 default

15 Implementation DReplFS -- Parser, Replication Engine, File Server in Go KDreplFS -- Parser in Go, Replication Engine and File Server in the Linux kernel

16 Experiments Dataset const N = type Data struct { a, b, c float32 var data [N]Data Views array of structs (AOS) struct of arrays (SOA) partial (only b) Replicas three replicas (AOS, SOA, b) two replicas (AOS, b) one replica (AOS) Each replica on separate SSD File Servers pass-through (POSIX) kdreplfs

17 Results: Read 1000 kdreplfs POSIX 800 Bandwidth (MB/s) Array of Structs 3 Replicas Struct of Arrays 3 Replicas Partial 3 Replicas Array of Structs 2 Replicas Struct of Arrays 2 Replicas Partial 2 Replicas Array of Structs 1 Replica Struct of Arrays 1 Replica Partial 1 Replica

18 Results: Write 1000 kdreplfs-sync kdreplfs-async POSIX 800 Bandwidth (MB/s) Array of Structs 3 Replicas Struct of Arrays 3 Replicas Partial 3 Replicas Array of Structs 2 Replicas Struct of Arrays 2 Replicas Partial 2 Replicas Array of Structs 1 Replica Struct of Arrays 1 Replica Partial 1 Replica

19 Results: Combined 1400 Read Write Bandwidth (MB/s) kdreplfs 1 replica kdreplfs 2 replicas POSIX

20 Future Work Variable-sized arrays More array element orders (z-order, Hilbert) Optimizations Endianness for primary types Support for HDF5 replicas Implementation that doesn t use file servers Automatic generation of dataset definition from standard data formats (HDF5, NetCDF)