FastForward I/O and Storage: IOD M5 Demonstration (5.2, 5.3, 5.9, 5.10)

Size: px

Start display at page:

Download "FastForward I/O and Storage: IOD M5 Demonstration (5.2, 5.3, 5.9, 5.10)"

Marian Owen
5 years ago
Views:

com Xuezhao Liu xuezhao.liu@emc.com Harriet Qiu harrietz.qiu@emc.com Haiying Tang haiying.

1 FastForward I/O and Storage: IOD M5 Demonstration (5.2, 5.3, 5.9, 5.10) 1 EMC September, 2013 John Bent john.bent@emc.com Sorin Faibish faibish_sorin@emc.com Xuezhao Liu xuezhao.liu@emc.com Harriet Qiu harrietz.qiu@emc.com Haiying Tang haiying.tang@emc.com Jerry Tirrell jerry.tirrell@emc.com Kelly Zhang kelly.zhang@emc.com Zhenhua Zhang zhenhua.zhang@emc.com

2 Agenda Brief introduction Source tree structure & build Demo objectives & environment Demonstrations Upcoming work 2

3 Introduction: IOD in FF stack CN 0 CN 1 CN n Rank 0, 1... Rank a, a+1... Rank x, x+1... Parallel application Process group Process group HDF5 library + VOL plugin CNs Function shipping client Fabrics IONs ION 0 ION m Function shipping server Function shipping server s process space Function shipping server s process space IOD (With POSIX BB) DAOS client Global storage DAOS 3

4 Introduction: IOD sub-modules overview API front-end Executing-Engine Task queue Event-Queues I/O scheduler Worker thread Pool Transaction manager Container manager KV object KV-store Object manager POSIX/DAOS storage access BLOB/ARRAY PLFS Inter-IODs communication 4

5 Source tree structure & build Source tree: Include - API header files Src - library source files Src/dep - dependent third party libraries such as plfs, mdhim. Test - testing/demonstration code Tools - simple binary tools Example - simple examples Build Just executes make in top directory (Will add autoconf/cmake script later) 5

6 Demo objectives M5 demonstration is specified to support Milestones: 5.2 IOD Object Store Demonstration 5.3 N ->M Stream Demonstration 5.9 Structure Storage and Retrieval between HDF and IO Dispatcher 5.10 Multi-format Replicas in the IO Dispatcher 6

7 In-scope demonstration elements Create, unlink, and listing contents of containers on IONs Create, unlink, read, write of kv, blob, and array objects on IONs Showing the different parameters for each object E.g. get/put for kv, byte offset for blobs, semantic offsets for arrays Migration of kv and blob objects between DAOS and IONs Will be shown in both directions Showing N-M stream transformation Querying layout of blob, kv, and array objects i.e. Returning a map of data chunks to storage locations Create, read, unlink of multi-format replicas of blob objects on IONs E.g. Create a replica of a blob with a different sharding 7

8 SOW Mapping 5.2. The Subcontractor shall demonstrate the initial object storage functionality of the I/O dispatcher. This will include an abstract storage layer with which PLFS can appropriately store data using a variety of storage interfaces such as the DAOS object store, possible key-value storage systems, POSIX, etc. [Since the SOW, the anticipated key-value storage interface to Flash memory did not materialize. Therefore, this milestone will demonstrate only the DAOS and POSIX interfaces by which IOD accesses physical storage devices.] 5.3. The Subcontractor shall demonstrate the initial N->M stream transformation functionality. This transformation is important to ensure that the IO dispatcher manages concurrency appropriately. If, for example, the dispatcher receives a stream of IO from every process but only has a more limited level of concurrency into either of the storage tiers, it will aggregate streams as necessary to reach an appropriate degree of concurrency. [Since the SOW, our thinking about this milestone has crystalized such that this demonstration will show how N streams between CN s and ION s are rearranged into M streams for the migration between ION s and DAOS. Changing N IO streams from HDF into M IO streams to the burst buffers will not be demo d here; rather N streams in burst buffers into M streams to DAOS.] The Subcontractor shall demonstrate the ability of HDF to store structural knowledge alongside the data using the IO Dispatcher API. This will allow the IO Dispatcher to make intelligent structure-aware sharding decisions. The demonstration will include a layout query by which an application (or job scheduler) can query the IO Dispatcher to discover on which burst buffer(s) a structurally-defined shard resides. [There are no clarifications or revisions to this milestone since the SOW.] The Subcontractor shall demonstrate the ability of the IO Dispatcher to store copies of data in different sharding organizations. Use of the existing plfs_map command will demonstrate that bytes ranges are replicated and will reveal in which shards, or on which burst buffers, each data chunk resides. This will be demonstrated at this time using simple byterange organizations (IOD blob objects) and the more complex demonstration where the replicas are structurally meaningful (IOD array objects) will be demonstrated in a later deliverable. [The functionality of this milestone is unchanged since the SOW; but the specific tool used to demonstrate this may not be the existing plfs_map command but an IOD version. This milestone is often referred to as multi-format replicas. In future deliverables, when this functionality is extended to array objects, it will be also referred to as semantic resharding.] 8

9 IOD Solution Architecture Mapping IOD SA 4.1: HDF / IOD API We will fully cover this by providing the full IOD API although not all functions will be fully implemented as described above. Of this SA, we will demonstrate the following: That the API for reading and writing all three IOD object types is fully implemented. IOD SA 4.2: Object storage We will partially cover this by demonstrating the following: Reads and writes to IOD objects perform correctly. IOD objects can be created and unlinked correctly. IOD containers can be read to retrieve the correct set of IOD objects existent in that container. IOD SA: 4.3 Data migration We will partially cover this by demonstrating the following: Migration in both directions of blob and kv objects. This will show the N-M transformation also as we ll provide a simple iod_obj_migrate command which will transform the N streams from the source into M streams in the target where M is aligned with the number of storage devices in the target. This may involve scatter-gather operations between IOD daemons in order to flatten plfs-style logs into a nicely striped set of objects. IOD SA 4.5. Structural awareness, KV stores, and data blobs We will partially cover this by demonstrating the following: Structural storage of distributed data structures into IOD array objects Reads and writes, creates and unlink of all object types 9

10 Out-of-scope demonstration elements Exascale test-bed will not be ready. Transactional I/O (6.2) End-to-end data integrity (7.3) Migration of array objects (7.5) Multi-format replicas of KV and array objects (8.3/8.4) Asynchronous IOD functionality (8.5) 10

11 Complete demo system VNX7500 with 1 15 drive tray, 1 Voyager (60 Drives) 1 Supermicro abba (4 IB ports, GPGPU) 1 Megatron abba (4 IB ports) 4 Lustre servers (1MDS, 3OSS) (1 IB each) 8 Compute Nodes (1 IB each) 1 Cisco 3560 Gigabit switch for managemement 1 Mellanox QDR switch 11

12 Demo configuration MPI Test Program BB 0 BB 1 DAOS POSIX 12

13 And away we go! 13

14 14

EFF-IO M7.5 Demo. Semantic Migration of Multi-dimensional Arrays

EFF-IO M7.5 Demo. Semantic Migration of Multi-dimensional Arrays EFF-IO M7.5 Demo Semantic Migration of Multi-dimensional Arrays John Bent, Sorin Faibish, Xuezhao Liu, Harriet Qui, Haiying Tang, Jerry Tirrell, Jingwang Zhang, Kelly Zhang, Zhenhua Zhang NOTICE: THIS