DataCutter Joel Saltz Alan Sussman Tahsin Kurc University of Maryland, College Park and Johns Hopkins Medical Institutions

Transcription

1 DataCutter Joel Saltz Alan Sussman Tahsin Kurc University of Maryland, College Park and Johns Hopkins Medical Institutions

2 DataCutter A suite of Middleware for subsetting and filtering multi-dimensional datasets stored on archival storage systems Subsetting through Range Queries a hyperbox defined in the multi-dimensional space underlying the dataset items whose multi-dimensional coordinates fall into the box are retrieved.

3 DataCutter Restricted processing (filtering/aggregations) through Filters to reduce the amount of data transferred to the client filters can run anywhere, but intended to run near (i.e., over local area network) storage system based on filter-stream programming model -- to optimize use of limited resources, such as memory and disk space

4 Client DataCutter Client Range Query Segment Data Client Interface Service Filtering Service Filter Filter Segment Info. DataCutter Indexing Service Data Access Service Archival Storage System Segments: (File,Offset,Size) Archival Storage System (File,Offset,Size)

5 DataCutter Architecture Client Interface Service Manages client connections and client requests Manages data and information flow between different services Indexing Service Two-level hierarchical indexing -- summary and detailed index files Customizable -- Default R-tree index User can add new indexing methods

6 DataCutter Architecture Filtering Service Manages filters (registered in the system) Users can add/run new filters Data Access Service Manages storage/retrieval of data from the tertiary storage Low level system dependent I/O operations

7 DataCutter -- Subsetting Datasets are partitioned into segments used to index the dataset, unit of retrieval Indexing very large datasets Multi-level hierarchical indexing scheme Summary index files -- to index a group of segments or detailed index files Detailed index files -- to index the segments

8 DataCutter -- Filters Filters Specialized user program to process data (segments) before returning them to the client Filter-stream programming model Originally developed for Active Disks environment (Acharya, Uysal, and Saltz) Based on stream abstraction A stream denotes a supply of data Streams deliver data in fixed size buffers Communication of a filter with its environment is restricted to its input and output streams init, process, finalize interface

9 A Motivating Scenario Sample Application: generate 3D reconstructed view from new set of sensor readings compare features with reference db Grid Configuration: remote data server - reference db sensor host - large raw readings parallel computation farm available 3D reconstruction computationally intensive Reference DB feature list? Data Server Sensor? Raw Dataset sensor readings WAN? Computation Farm? Client PC

10 A Motivating Scenario (2) Application : // process relevant raw readings // generate 3D view // compute features of 3D view // find similar features in reference db // display new view and similar cases Reference DB feature list Extract ref Data Server 3D reconstruction Computation Farm WAN View result Extract raw 3D reconstruction Extract ref Sensor View result Extract raw Reference DB Raw Dataset sensor readings Client PC Raw Dataset

11 Filters Filters communicate with other filters only using streams cannot change stream endpoints are allowed to pre-disclose dynamic allocation of memory/scratch space in init phase, before processing phase Advantages location independence easier scheduling of resources filter stop and restart is defined explicitly in model

12 Placement The dynamic assignment of filters to particular hosts for execution is placement (mapping) Optimization criteria: Communication leverage filter affinity to dataset minimize communication volume on slower connections co-locate filters with large communication volume Computation expensive computation on faster, less loaded hosts

13 Restructuring Process Application Decompose Target Configuration Some set of filters f 1 f 2 f 3 f4 f 5 Placement / Schedule Execute Application

14 Software Infrastructure Prototype implementation of filter framework C++ language binding manual placement wide-area execution service one thread for each instantiated filter

15 Filter Framework class MyFilter : public AS_Filter_Base { public: int init(int argc, char *argv[ ]) { }; int process(stream_t st) { }; int finalize(void) { }; }

16 Filter Connectivity / Placement [filter.a] outs = stream1 stream3 [filter.b] ins = stream1 outs = stream2 [filter.c] ins = stream2 stream3 [placement] A = host1.cs.umd.edu B = host2.cs.umd.edu C = host3.cs.umd.edu A stream1 stream3 B C stream2

17 Execution Service 1. Read Specs Filter/Stream Placement Filter lib Application Console???.???.???.??? 3. Exec 2. Query Directory name host port **** **** **** **** **** **** Directory Daemon dir.cs.umd.edu:6000 AppExec Daemon EXEC AppExec Daemon EXEC AppExec Daemon EXEC Filter lib Filter lib Filter lib filter A Application host1.cs.umd.edu filter B Application host2.cs.umd.edu filter C Application host3.cs.umd.edu

18 Related Work Application Level AppLeS Client/Server Sockets Programming Models Infrastructure Services Legion DataCutter SRB Harmony NWS DPSS DSM MPI RPC Globus HPC++ JavaRMI, DCOM, CORBA NetSolve, Ninf Condor Pool Resource Level Grid available Resources User specified Resources Idle Resources

19 Integrating DataCutter with the Storage Resouce Broker

20 Storage Resource Broker (SRB) Middleware between clients and storage resources Remote Access to storage resources. Various types : File Systems - UNIX, HPSS, UniTree, DPSS (LBL). DB large objects - Oracle, DB2, Illustra. Uniform client interface (API).

21 Storage Resource Broker (SRB) MCAT - MetaData Catalog Datasets (files) and Collections (directories) - inodes and more. Storage resources User information - authentication, access privileges, etc. Software package Server, client library, UNIX-like utilities, Java GUI Platforms - Solaris, Sun OS, Digital Unix, SGI Irix, Cray T90.

22 SRB/DataCutter - Prototype Implementation Support for Range Queries Creation of indices over data sets (composed set of data files) Subsetting of data sets Search for files or portions of files that intersect a given range query Restricted filter operations on portions of files (data segments) before returning them to the client (to perform filtering or aggregation to reduce data volume)

23 SRB/DataCutter System Application (SRB client) Resource User File SID DBLobjID ObjSID Range Query Storage Resource Broker (SRB) Indexing Service DataCutter MCAT SRB I/O and MCAT API Filtering Service Filter Filter Application Meta-data DB2, Oracle, Illustra, ObjectStore HPSS, UniTree UNIX, ftp Distributed Storage Resources

24 SRB/DataCutter Client Interface Creating and Deleting Index int sfocreateindex(srbconn *conn, sfoclass class, int cattype, char *inindexname, char *outindexname, char *resourcename) int sfodeleteindex(srbconn *conn, sfoclass class, int cattype, char *indexname)

25 SRB/DataCutter Client Interface Searching Index -- R-tree index typedef struct { int dim; /* bounding box dimensions */ double *min; /* minimum in each dimension */ double *max; /* maximum in each dimension */ } sfombr; /* Bounding box structure */ typedef struct { sfombr segmentmbr; /* bounding box of the segment */ char *objid; /* object in SRB that contains the segment */ char *collectionname; /* collection where object is stored */ unsigned int offset; /* offset of the segment in the object */ unsigned int size; /* size of segment */ } segmentinfo; /* segment meta-data information */ typedef struct { int segmentcount; /* number of segments returned */ segmentinfo *segments; /* segment meta-data information */ int continueindex; /* continuation flag */ } indexsearchresult; /* search result structure */

26 SRB/DataCutter Client Interface Searching Index -- R-tree index int sfosearchindex(srbconn *conn, sfoclass class, char *indexname, void *query, indexsearchresult *myresult, int maxsegcount) typedef struct { int dim; double *min, *max; } rangequery; int sfogetmoresearchresult(srbconn *conn, int continueindex, indexsearchresult *myresult, int maxsegcount)

27 Applying Filters typedef struct { segmentinfo seginfo; /* info on segment data buffer after filter oper. */ char *segment; /* segment data buffer after filter is applied */ } segmentdata; typedef struct { int segmentdatacount; /* #segments in segmentdata array */ segmentdata *segments; /* segmentdata array */ int continueindex; /* continuation flag */ } filterdataresult;

28 Applying Filters int sfoapplyfilter(srbconn *conn, sfoclass class, char *hostname, int filterid, char *filterarg, int numofinputsegments, segmentinfo *inputsegments, filterdataresult *myresult, int maxsegcount) int sfogetmorefilterresult(srbconn *conn, int continueindex, filterdataresult *myresult, int maxsegcount)

29 Application: Virtual Microscope Interactive software emulation of high power light microscope for processing/visualizing image datasets 3-D Image Dataset (100MB to 5GB per focal plane) Client-server system organization Rectangular region queries, multiple data chunk reply pipeline style processing read_data decompress clip zoom view

30 Virtual Microscope Client

31 VM Application using SRB/DataCutter read Indexing Wide Area Network SRB/DataCutter Distributed Collection of Workstations decompress clip zoom Distributed Storage Resources Local Area Network read read image chunks decompress clip zoom convert jpeg image chunks into RGB pixels clip image to query boundaries sub-sample to the required magnification view Client view Client view stitch image pieces together and display image