Active Storage using OSD. John A. Chandy Department of Electrical and Computer Engineering

Transcription

1 Active Storage using OSD John A. Chandy Department of Electrical and Computer Engineering

2 Active Disks We already have intelligence at the disk Block management Arm scheduling Can we use that intelligence for computation? Disk controller can potentially optimize data layout and retrieval since it knows how the data is stored Avoid extra data transfer across networks, buses, or interconnect Parallel disks get you concurrency

3 Active Disks From Riedel et al, IEEE Computer,, June 2001

4 Active Disks Riedel et al. Data mining Image processing Acharya et al. Database queries Streaming model

5 Active Disks Acharya et al. Code fragments are called disklets and they operate on streams of data Three types of streams: Disk-resident streams Host-resident streams Pipe streams Communication between a disklet and its environment is restricted to its input and output streams. The source and sinks for these streams are specified by the host-resident program as a part of the installation of the disklet.

6 Performance of active disks From Acharya et al, ASPLOS, 1998

7 Active Disks Vendors haven t t provided mechanisms for active disks yet Complexity not worth it in commodity disks Clusters provide the parallelism right now

8 Object Disks Intelligence at the disk can also be used to offload some of the metadata processing Allocation of disk blocks is easier at the drive The focus of interest is an object Analogous to an inode Not named like a file, instead identified by 128-bit ID Separate metadata server provides Object location (filesystem( tree) UNIX level permissions CMU Network Attached Secure Disk (NASD)

9 Object Based Storage Architecture From Working draft SCSI object-based storage device commands (OSD-2)

10 Object Disks OSD type cluster storage nodes Panasas, Lustre,, PVFS Unlike active disks, object disks are now a reality SCSI T-10 OSD (Object-Based Storage Device) Reference implementations from Intel, DISC, and IBM No real disks yet, but coming

11 Active Disks Can we use OSDs to make Active Disks a reality? Application-aware storage Object attributes can give hints to the disk Application specific Parallel File Systems Felix et al. added a filtering layer to Lustre to provide active processing T10 OSD?

12 Active Disks using OSD Challenges What is the programming model? How do you download code to the OSD target? How do you execute code on the OSD? What are the security/capability implications

13 Active Disks using OSD Programming Model Object-oriented Attach object types to storage objects Define methods for object types RPC Call methods on OSD remotely

14 Active Disks using OSD Example: Record list object List { }; addrecord(); sortrecords(); searchrecord(string);

15 Active Disks using OSD Distribute List across multiple objects - one per OSD target Client has a single unified List view Proxy class coordinates methods on client s List with method calls on OSD Lists Client proxy manages objects in distributed List

16 Active Disks Using OSD Code is written in Java Advantages Object-oriented Machine-independent Secure/Safe Easy to move from client to target

17 Active Disks using OSD How do you move Java code from client to target within OSD framework? OSD objects allow users to set attributes on an object We set an attribute on each object where the attribute is the.class file or.jar file associated with the object methods One code attribute page with multiple code attributes Allows different users to define their own code

18 Active Disks using OSD How do you execute the method remotely within the OSD framework? Modify the READ command so that we can invoke a method Recent change to OSD spec allows for command attribute pages - i.e. attribute pages that are only valid for the life of the command Introduced by Pete Wyckoff/OSC for support of scatter/gather We use the command attribute page to specify the method and parameters From T10/08-091r0 proposed changes to OSD-2

19 Active Disks using OSD Command attribute page CODE ATTRIBUTE NUMBER METHOD IDENTIFIER NUMPARAMS PARAM_1_LEN PARAM_1 PARAM_N_LEN PARAM_N

20 Simple List example Sorting on up to 12 OSDs

21 Active Disks using OSD Issues What if we don t t know the amount of data that the method will return? SCSI requires that the client request a fixed amount of data and the device must return just that amount of data - no more, no less Probably require two READ commands First command returns size and a cookie Second command uses cookie Requires state at the target

22 Active Disks using OSD Issues Safety Should we place limits on active disk computation? Security CPU, memory, disk Capabilities can be used to protect access to commands Application-specific User-specific System-wide

23 Active Disks using OSD Issues What about other command data patterns? Read only Write only Read/write Read object - write to another object Read collection - write to another collection Object Method Processing Client New Object

24 Active Disks using OSD Other issues Is OO RPC the right model? We are looking into functional models like MapReduce/Hadoop What system services should the OSD target provide? Local file system access? Process/Thread management? Application adoption is needed Database and search are the most promising

25 Further directions Hardware Acceleration Active Storage Networks

26 Hardware acceleration Is it possible to dedicate some reconfigurable hardware space on the object disk to do hardware acceleration Encryption functions Image processing Other application-specific hardware

27 Hardware acceleration Integrated Processor Java Engine Reconfig Logic From Riedel et al, IEEE Computer,, June 2001

28 Hardware acceleration Use same OSD mechanisms to Download hardware bitstreams Execute hardware methods Issues No standard for hardware bitstreams Are there security issues?

29 Active Storage Networks Can intelligence in a storage network Accelerate computation? Accelerate storage access? Motivation Active Disks Intelligence at the disk can distribute computation to parallel disks Process data in streams Disks only have local view of data Active Storage Network Network has a global view of data Distributed caching of file system metadata and data Redundancy optimizations

30 Active Storage Network

31 Active Storage Networks Application operations Reduction operations Database queries SELECT ORDER BY.. LIMIT k m disks n items per disk O(nm) for normal disks O(n + km) for active disks O(n + k log m) for active disks with ASN Scientific applications MPI_SUM, MPI_MAX, etc.

32 Active Storage Networks Application operations Transformational operations Sorting O(mn log mn) for normal disks O(n log n) + O(mn log m) for active disks O(n log n) + O(n n log m) for active disks with ASN Scientific applications Matrix transformations Stream-based Video editing

33 File System Caching Centralized cache frees up memory at the clients Metadata caching reduces access to metadata server File layouts Directory lookups File access attributes

34 ASN Switch Implementation Embedded nodes in the switch Processor directly on the backplane FPGAs Downloadable functions (netlets) Software functions on embedded processors Hardware functions on FPGAs Can we do this in OSD? Current implementation is very system specific Can we process iscsi packets on the fly? How do we handle OSD capabilities?

35 Summary OSD implementation of Active Storage Shows promise Future directions in hardware acceleration and ASN Students Tina John and Anu Thiruvenkata Ramani