An Introduction to the Mass Storage System Reference Model, Version 5

Transcription

1 An Introduction to the Mass Storage System Reference Model, Version 5 Robert A. Coyne, Harry Hulen IBM Federal Systems Company Houston, Texas Abstract Brief histories of the IEEE Mass Storage System Reference Model and the IEEE Storage System Standards Working Group are presented. The Reference Model is explained in terms of layers of abstraction, including storage-container layers, mapping-service layers, and storage-sewice layers. The Need for Storage System Standards The history of storage system standards began with the desire for media interchange. Since systems were isolated and proprietary, the requirements were reasonably simple and various form factors and recording formats were standardized, usually after one vendor had gained a significant market presence. Later, logical formats for volume and file structure arose so higher-level semantic objects, such as ANSI labeled tapes, could be exchanged between sites. In the eighties, the IEEE Computer Society Mass Storage Systems and Technology Technical Committee attempted to bring some order to the confusion surrounding the evolving storage industry. The result was an unofficial Mass Storage System Reference Model [l, 21. Every offerer of a significant new commercial storage system in the last several years has used the language of the Reference Model to describe its function. Customers have specified compliance with the Mass Storage System Reference Model even though the IEEE has not approved it as a standard. Storage systems today are quite complex. They often include very expensive equipment and therefore must be shared. Operations and management are often labor intensive. Recent advances in storage system hardware, in particular robotic devices for controlling removable media and large, high-performance, network-attached disk arrays, are under exploited due to lack of systems software that enables sharing by large numbers of distributed clients. It is our hope and conviction that the process of developing standards for storage systems can speed the development of systems software for storage systems. Storage System Standards Working Group The IEEE P1244 Storage System Standards Working Group (IEEE SSSWG) was chartered in May 1990 to abstract the hardware and software components of existing and emerging storage systems and to develop standard software interfaces between these components. The goal of storage service standardization is the decomposition of a storage system into interoperable functional modules that vendors may offer as separate commercial products. The Working Group s ultimate goal is to develop interoperable standards that define the software interfaces and, in the distributed case, the associated protocols to each of the architectural modules identified in the Model. In this way, storage systems will become truly open systems, and competition for both standard software modules and storage devices delivered with compliant software interfaces will lead to higher quality and lower cost. The IEEE SSSWG has served as an important forum for users, developers, and researchers to come together to work on mass storage system issues. The first and perhaps most important accomplishment has been to get representatives from a greatly increased group of more than forty organizations to agree on the basic building blocks of a standard storage system architecture. The Mass Storage System Reference Model The IEEE SSSWG adopted Version 4 of the Reference Model as its technical base, and over the past two years has extended and adjusted the Reference Model to better meet the requirements of the storage community. This paper reflects current drafts of Version 5 [3]. The Model includes several noteworthy architectural features. First is the provision for location independence, one of a number of desirable transparencies that shield clients from the details of the storage system implementation. Clients access storage system objects using unique, system-oriented identifiers. Once issued, these identifiers are immutable. The location of storage system objects may change but the unique identifiers are all that any client must present for access. Another fundamental feature is the separation of control and data paths. The Reference Model architecture allows for physical separation between the logical data path, that transfers storage data directly between sources and sinks, and the logical storage control path, that carries control and status messages between clients and servers. In this way, the architecture enables separate optimization of both the data path, for high-bandwidth bulk transfers, and the /93 $ IEEE 47

2 Twelfth IEEE Svmoosium on Mass Storage Svstems control path, for small control and status messages. In addition, a physically-clustered storage system may employ a dedicated storage device control path for additional data security. The Reference Model separates the concept of file name from file access. By not coupling naming with access, multiple name servers can coexist in a single storage system, each supporting a different human-oriented name Space. The Working Group is chartered to consider storage systems of every scale, from personal computers to the largest, distributed supercomputer storage systems. It appears feasible to standardize a storage architecture that is scale-independent, with individual implementations tailored to their target scale. The Working Group is actively studying the diverse requirements of the entire range of existing and emerging storage systems, from small, stand-alone systems with simple, removable media through departmental file servers to centralized and distributed file servers on networks of supercomputers. Layers of Abstraction in Storage Systems The Mass Storage System Reference Model has not taken the form of a truly layered model such as the familiar OS1 models used in the discipline of communications. Nevertheless, there seems to be value in using the idea of layers to help convey an intuitive grasp of the relationship among elements of a model. There are several aspects of the Reference Model that can be presented in layers. There are also several aspects that appear to defy any attempt at layering as they cut across the more obvious layer boundaries. We will look briefly at three aspects of the Model that more or less fit the layered view and then deal with aspects of the Model that cut across layers. The Data Containment Hierarchy The storage system environment consists of servers and the data entities they manage. Figure 1 depicts the containment hierarchy ranging from the user s view of the data to the physical volume on which data is stored. The top two layers of the hierarchy represent the user s view of the data and are not part of the Reference Model. The top layer represents the data format, such as numbers, pixels, and text. The second layer represents the data organization, which may take the form of a full-featured database management system or file system. Below the data format and data organization is shown the storage system. The bitfile layer is simply a string of bits that are given a name and that can be stored and retrieved. Bitfile is a term that originated with the IEEE Mass Storage System Reference Model. Groups of bitfiles are organized into bitjze containers for the purpose of managing storage hierarchies. I Data Format 3 I Data Organization 1 I Bitfile 1 Bitfile Container Virtual Volume Physical Volume, Mounted State Physical Volume, Repository State Figure I. Data Containment Hierarchy A virtual volume is a collection of contiguous virtual storage space. Virtual volumes are mapped onto physical volumes. A large virtual volume may span several physical volumes, or a small one may share a physical volume with other virtual volumes. For most purposes, the client should only need to be aware of the virtual volumes, leaving the mapping of virtual volumes to physical volumes and the management of physical volumes to the storage system. This transparency shields the user from the need to keep track of the size of individual physical volumes, and it allows the storage system to handle different sizes of physical volumes and to introduce new sizes without changing the client. The physical volume may be removable or permanently mounted. In the case of removable media, there are two states that are important enough to be shown as separate layers in Figure 1. In mounted state, a physical volume becomes identified with a disk drive, a tape transport, a memory socket, or other I/O device, and thereby becomes accessible by the Mover function. The I/O device and the physical volume become bound together into a single entity having both a device identifier and a physical volume identifier. For non-removable media, the mounted state is the only possible state. For removable media, ignoring the state of chaos where a physical volume is on a cluttered table or in a briefcase, the other important state is the state of being in an assigned location. The location may be a numbered place on a shelf where it is stored and retrieved by humans, or the location may be a slot in an automated repository. In this repository state, a physical volume also has two identifiers, the physical volume identifier and the slot identifier. The volume is accessible by a person or by a robotic device that moves the entire volume from one state to another. However, the data is obviously not accessible from the repository state.

3 Storage Service Layers We now come to a layered view of the storage services that comprise the Reference Model. The Reference Model uses a client-server paradigm to describe these entities. A client is a consumer of services and a server is a provider of services. A server can act as a client and request services from other entities. In a strictly-layered reference model such as the OS1 communications model, the user acts as ultimate client and seeks services from the top service layer. This layer seeks services from the next layer, and so on down through the layers in strict sequence. The Mass Storage System Reference Model is not that well behaved; there is some jumping around between layers. However, there is a general direction to the flow and it is useful to discuss the client-server relationships in the order shown in Figure 2. The top layer represents the Clients of a storage system. A Client may be an application that directly accesses storage services. However, there will frequently be an agent or access broker such as a file system or database management system between the application and the storage system. If there is an access broker, the application may not see the storage system at all; it will be totally encapsulated by the access broker. For example, if the access broker is a Unix file system or some entity that would appear to be a Unix file system to the application, then the application only has to know the syntax and rules of a Unix file system. However, the entity providing the user view of a Unix file system has to be very much involved in the syntax and rules of the storage system, for such is the role of an access broker. The layers below the Client have no knowledge of whether the application is using an access broker or accessing the storage system directly. For this reason, the Reference Model does not distinguish between types of Clients. Client (application) Client (access broker) Figure 2. Layers of Storage Services The Bitfile Server manages the storage space on a collection of virtual volumes and presents bitfiles to its clients. Thus, the Bitfile Server converts access requests for bitfiles into logical requests for storage services. The Bitfile Server supports the functions of bitfile migration and replication. Migration refers to the movement of bitfiles, either partially or fully, from one storage medium or location to another. Replication is the duplication of bitfiles at multiple storage locations. These concepts are intertwined as both require managing multiple copies of storage objects in such a way as to appear to be a single COPY. When bitfile migration occurs from one media type to another, certain storage access semantics may become visible to clients. Some clients will wish to be shielded from the knowledge that their data has migrated from disk to tape and will suffer some delay as their data is cached to disk before random accesses. Other clients will be cognizant of the fact that their data is now resident on tape and sequential access is completely adequate for their access patterns. The Bitfile Server coordinates shared access to a bitfile by independent clients. It also provides sufficient semantics to allow clients to determine when their write requests have been committed to stable storage and to optimize requests for sequential versus random access media. The Storage Server is essentially a virtual volume manager. The Storage Server collects labeled physical volume resources and constructs and presents virtual volumes to its clients. It performs this function by mapping access operations from virtual volumes to these non-composite physical volumes. The Storage Server will compose collections of physical or virtual volumes to form such entities as volume sets, stripe sets, RAID arrays, mirror sets and replica suites. Such sets of virtual volumes may have an arbitrary hierarchical structure built from the base volumes. The Storage Server supports reconfiguration of virtual volumes onto different base volumes to allow migration of volumes and extension of existing virtual volumes. The Storage Server requests the mounting and dismounting of physical volumes to satisfy the access requests of its clients to virtual volumes. A virtual volume is not controlled to assure consistency across access by multiple clients. The next layer shown in Figure 2 is an entity which has multiple roles called the Mover. The Mover, as the name implies, is involved in moving data; however, the Mover also serves as a physical device manager. As physical device manager, a Mover encapsulates each device, and all operations on a device are performed through requests to a Mover. The Mover is the software component that knows the detailed inner workings of the storage device and its internal control connections. The external interface to the Mover includes a set of generic device operations that are adequate for reading, writing, and positioning both random 49

4 and sequential-access storage devices. In a conventional host-based storage system, a Mover can be thought of as a specialized I/O driver. In a distributed system, the Mover functions may take the form of an intelligent control unit. We will return to our discussion of the Mover in the next section, when we will look at its role in moving data. The Physical Volume Library (PVL) is a generalization of an enterprise-wide tape management system that tracks the current locations and status of all removable volumes. The PVL controls the actual mounting and dismounting of volumes. It maintains mount queues and verifies internal volume labels. It globally optimizes the use of drives, tracks the life-cycle state of cartridges and maintains scratch pools. To support moving physical volumes around in the storage system without fear of name conflicts, the Reference Model recommends (but does not require) a single PVL. In a very large or widely-distributed storage system, a PVL may be divided into branch libraries which can operate independently under a single PVL-wide policy. The Physical Volume Repository (PVR) maintains the mapping of volume identifiers to slots and drives, optimizes the apparent mount times by staging, and enforces various management and security attributes. Each PVR manages a single repository or a set of connected repositories. A PVR may employ either a mechanical robot or a human operator to perform mounts. The PVR operates under the control of a PVL. There can be any number of PVRs. The Mover, Physical Volume Library, and Physical Volume Repository together form the physical storage system. Although the storage services in Figure 2 are depicted as layers, an important feature of the Reference Model is the ability of a client to go directly to lower level servers. This is particularly important for power clients such as a real-time application or a database manager. Such a client may be designed to reach well down into the storage system and request services of the physical storage system servers such as allocating space or pre-loading a volume of media. Mapping Service Layers Originally, the Reference Model recognized a name server but no other mapping services. It is now recognized that mapping services are a key part of any future storage system standard. Consequently, a subcommittee has been established within the IEEE SSSWG to develop standards for mapping services. policy. Name services are not standardized in the Model because the universe of entities that can act as name servers, from hierarchical file systems through associative databases, prevent standardization of the naming semantics. One of the first proposals of the mapping committee has been the definition of a Storage System Object Identifier, or SSOZD. The Object in SSOID can be a bitfile, a virtual volume, or any of the layers of storage described above; or it can be any of a number of components to be described shortly, such as a Storage Server or Physical Volume Repository. The SSOID may be one of the first standards to emerge from the IEEE SSSWG. The top level of mapping services shown in Figure 3 is the Name Server. This level is recognized as essential by the Reference Model, but no attempt will be made to define the mapping function, other than to define the structure of the SSOID and the basic function interface. The specifics of the Name Server are left to the user or application developer. It may be as simple as a file or as complex as a full fledged data base management system. The second layer represents the Location Server. From the SSOID, the Client of the storage system will use this service to determine where to find the storage services that manage the bitfile. Also from the SSOID, the user will have an internal, unique, and stable handle for accessing the desired bitfile. From this, the Bitfile Server will be able to determine the location of the virtual volume to which the bitfile is assigned. The lower layers map the virtual volume to the physical volume, and for removable media, the physical volume to a location within the repository. I Name Server (name): SSOID I Location Server (SSOID): Service Location Bitfile Server (SSOID): Virtual Volume Location Storage Server (Virtual Volume ID): Physical Volume ~ ~~ Physical Volume Library (Physical Volume ID): Repository Slot Figure 3. Layers of Mapping Services The Mapping Services Subcommittee has proposed that the location and name server(s) operate within a storage system domain. A storage system domain contains one logical location server, a common authentication mechanism and security policy, and a common administration 50, _,..... ll..l-.,~-,. -

5 Twelfth IEEE Svmuosium on Mass Storage Svstems Components Common to Several Layers The Reference Model contains several components that cut across the layers previously described. These components are the Mover, Security, and Storage System Management. The Mover The Mover was introduced in an earlier section in its role as physical device manager. The Mover s other function is to manage the movement of data. In a host-based system, these two functions can fit together easily into a single software module. In a distributed system, however, the Mover will be distributed. A portion of the Mover must be physically co-resident with the storage device and network connections it is controlling. At the Client end, another portion of the Mover is the function that manages delivery of data to the Client s memory. The Mover is a very general function that must be embedded into any client or server that needs to be a source or sink for data. The Mover is one of the principal means by which control functions and data movement functions are separated. This separation may be a physical separation, or it may be a virtual overlay on a physically integrated communications architecture. Distributed systems, particularly those intended for high performance, may be designed such that the Mover is associated with a high speed data network which is physically separate from the control network. On the other hand, in more centralized storage systems, the Mover and control functions may share the same network or the same library of standard communication services. Security Security is divided into three separately implementable components: Authentication, Authorization, and Enforcement. Authentication establishes the identity of a client anfflor server. Authorization calculates the access rights of a principal to a service, while Enforcement applies the results of this calculation at the point where access is actually requested. By separating Authorization and Enforcement, different Authorization models can coexist in the same environment and one module may transfer authorization information to another for enforcement. An example implementation of this security architecture might employ encrypted capability tickets issued by a security server to authenticated clients who then present these tickets to the appropriate modules for service. Each server in the Model would include common interfaces for the three security functions, and each service request would automatically be routed through the appropriate Authentication, Authorization, and Enforcement functions before the service would act on the request. Storage System Management Storage System Management is the collection of functions concerned with control, coordination, monitoring, performance and utilization of the storage system. These functions are typically site-dependent and involve human judgment. Standards are critical in managing a large, distributed collection of heterogeneous storage resources because of the need to exchange management information and provide predictable and consistent control. The Model management framework does not attempt to define the internal facilities and policies that are needed. Instead, the Model provides a generalized framework upon which different storage management subsystems and applications may be built. The functional areas of management cover a broad array of topics. Fault management encompasses the monitoring, detection, isolation, and correction of abnormal operations and components. Configuration management is concerned with initializing, operating, reconfiguring, and shutting down the storage system and its components. Security management determines and implements the security policies of the storage system and monitors the storage system for attempted violations. Accounting management tracks the resource consumption of storage system clients, charges for storage use, and denies resources if accounts are overdrawn. Performance management evaluates the long-term performance of the system through summary statistics and adjusts the use of the system components to be most efficient and cost-effective. Several System Management functions have been identified, including migration of storage objects to a cheaper level in the storage hierarchy, defragmentation and repacking of bitfiles and volumes, initialization, addition, and deletion of new storage resources, logging of relevant storage events, alarms on various types of errors and other events, backup, bitfile recovery, system recovery, and capacity planning. An intelligent operations agent will be a necessity to manage the large, distributed storage systems that are being planned today, as these systems will be implemented on multiple, independent platforms with the attendant management complexity

6 A Composite View of the Model Now we will put Figures 1 through 3 together into a single conceptual view of the Reference Model. The three layered aspects of the Model, together with the cross-layer aspects, are shown together in Figure 4. The boxes on the left side of the figure represent the data containment hierarchy, the right side represents the layers of mapping services, and the center tower represents the storage service layers. Running vertically behind the services are the cross-layer functions: The Mover, Security, and Storage System Management. The intersections of the cross-layer functions and the storage services are not intended to represent the flow of data but represent points of contact and interaction between the cross-layer functions and the storage services. Two of the cross-layer functions, Storage System Management and Authentication, Authorization, and Enforcement, interface with all service layers. The Mover is involved with all layers of service except the Physical Volume Repository. The Mover, as shown in Figure 4, has a branch reaching over to the physical volume layer. This branch represents the Mover's role as physical device manager. Client Application Access Broker Ph sical VoYume a Figure 4. A Layered View of the Reference Model 52

7 A Traditional View of the Reference Model Readers familiar with previous versions of the Reference Model may find the layered view of the Model a departure from the traditional way the Reference Model has been depicted. A more traditional view of the Model is therefore offered as Figure 5. This representation emphasizes the client-server aspect of the Reference Model, the separation of control and data, and the ability of the Client to access all layers of service directly. Across the center of the figure, the Mover is shown as owning the storage devices and providing data paths between the devices and the Client and various servers. The heavy black lines emphasize that all data flow is through the Mover, whether centralized or distributed. The Client, which we recall may be an application or an access broker, has a primary control flow to the Bitfile Server which, in turn, has a control flow connection to the Storage Server and the Physical Storage System. This corresponds to the layered view previously described. However, there are optional control paths between the Client and the lower level servers, emphasizing the ability of the Model to support specialized clients with demiding functional and &&ormake requirements. / Storage System Management / Location Fl - Optional Control Flow Control Flow (Client to Server) Data Flow (Bidirectional) / Figure 5. A Conventional View of the Reference Model, Version 5 Acknowledgment The authors wish to thank the members of the IEEE Storage System Standards Working Group, especially those whose contributions to the Reference Model have found their way into this paper and those whose contributions have been simplified and freely interpreted by the authors. [I] [21 References Stephen W. Miller. A Reference Model for Mass Storage Systems. Advances In Computers, 27: , Sam Coleman and Stephen W. Miller, editors. Mass Storage System Reference Model: Version 4. IEEE Technical Committee on Mass Storage Systems and Technology, May [31 Sam Coleman and David Isaac, editors. Mass Storage Reference Model, Version 5, Draft 0.3. A working document of the IEEE Storage System Standards Working Group.