Distributed File System

Rochester Institute of Technology Distributed File System MS Project Proposal By: Swarup Datta sxd2009@cs.rit.edu Chairman: Dr. Hans-Peter Bischof Reader: Dr. James E. Heliotis 1 of 16

Abstract Ad hoc networks provide capability for devices to participate in a wireless network without the presence of a system that function as a central authority. They can be deployed without any fixed infrastructure making it attractive for devices with wireless capabilities. One issue for devices in an ad hoc network is accessing file systems on other devices within the same network. To solve this problem distributed file system a middleware can be developed with concepts borrowed from the Plan 9 system. The proposed middleware will allow devices to easily navigate and access directories and files on a file system residing on a remote device. 2 of 16

Table of Contents 1 INTRODUCTION...4 1.1 M2MI Architecture... 4 1.2 Overview... 4 1.3 Terms... 5 2 DESIGN...6 2.1 Exporting... 6 2.2 Importing and Name resolution... 7 2.3 Operations on file system... 10 2.4 Maintenance... 11 3 DELIVERABLES...15 4 REFERENCES...16 3 of 16

1 Introduction 1.1 M2MI Architecture Many-to-Many Invocation (M2MI) is a new paradigm to facilitate development of collaborative systems such as middleware and applications [1]. Target devices for M2MI are mobile hosts that can participate in proximal ad hoc networks. Traditional distributed system support one-to-one and one-to-many communication patterns. One of the main contributions of M2MI is that it supports many-to-many communication pattern demonstrated by applications such as chat programs. In proximal ad hoc networks, M2MI can use broadcasting to send messages to other devices. Utilization of broadcasting reduces the need for routing mechanism, such as routing table and protocol, in the devices making it ideal for small battery powered hosts. The objective of this project is to develop a distributed file system for ad hoc networks. Section 1.2 provides an overview of the distributed file system. Section 2 offer design details on the distributed file system. Section 3 details deliverables of the project. Section 4 includes references. 1.2 Overview An essential feature for systems participating in a network is the ability to share data. Current distributed file systems such as NFS from Sun Microsystem, DFS from Microsoft, and GFS from Google allow systems on traditional networks with fixed infrastructure to access data remotely. A new distributed file system can be developed using M2MI to provide hosts in an ad hoc network the ability to share and access data on remote devices. Ad hoc networks differ from traditional networks as they can be deployed without presence of a dedicated central authority. Also, ad hoc networks tend to be highly dynamic as they experience frequent changes in the network topology. These characteristics add new challenges to designing a distributed file system. The proposed file system in this document, which will be implemented as a middleware, will allow a device to function both as server by providing others to access its files. The same device will also be able to act as a client by accessing files on other hosts. The middleware will also include mechanism to adapt to any changes to the network topology as hosts join or leave a network. For the proposed middleware, features such as bind operation and union directory will be borrowed from Plan 9. Plan 9 is a distributed system where resources, including processes, memory, and network interface card, are treated as files [2]. Another feature of Plan 9 is all 4 of 16

resources, including files, are represented in a single hierarchical view called name space. Each system has a local copy of name space and can be modify it by executing a bind operation. Result of a bind operation is a union directory, which joins contents of two directories, or a directory and a file defined as arguments in bind operation. Order of joining the arguments is specified as a parameter with BEFORE, AFTER, and REPLACE as possible values. For example, using REPLACE flag a device can bind a /bin directory from a remote device to its local name space replacing its own local /bin directory. Entities in the resulting union directory can be accessed using path /bin. This can be useful when a user desires to execute binaries residing on a remote system. The remote /bin directory can also be attached in front of the local /bin directory using bind flag BEFORE. In this case, a file access in /bin will search the local /bin directory and any remote directories attached to local /bin. Use of BEFORE flag will require search of the remote directory prior to local /bin. To attach the remote /bin behind a local directory, bind flag AFTER can be used. In Plan 9 binding is not only restricted to remote directories, binding can also be performed on local directories and files. Another advantage of union directory is evident when copying or creating files. During file creation in a union directory, the file would be created on the first directory with write permission. File will not be created if create system call fails on the first directory encountered with write permission. Using the previous example, if the remote /bin has only read permission, a file create system call will create the file in the local /bin directory, assuming the local /bin has write permission. In the proposed distributed file system, bind operation and union directory will be used to import and manage access to remote file system. Bind operation will allow for integration of remote directories in a device and union directories will be generated to provide access to contents of local and remote directories. The middleware for the distributed file system (MDFS) will represent local and remote file system as n-ary tree structure. Devices running M2MI and the MDFS will be able to share local file system, and access and modify remote file systems. 1.3 Terms File System Local File System Physical File System - Hierarchical view of directories and files provided by an operating system, where a directory and file is treated as an entity. - File system constituted by MDFS middleware. - A device s physical file system managed by operating system. 5 of 16

2 Design In this section, design of the distributed file system will be discussed in details. The discussion will be divided into the following parts, exporting, importing of file system and name resolution, maintenance, and operations on the distributed file system. 2.1 Exporting When a device running MDFS wants to start sharing a file or directory, it will first use a method called exporting. Exporting will accomplish two tasks. First, it will let a device register with MDFS the file structure that is being shared. Second, exporting will publish a message notifying others information required for accessing the exported file structure. The message will include four pieces of information. An Exported Object ID (EOID) associated with an exported object to M2MI; uniquely identify the device generating the message. A tree representing the exported file structure and its associated permissions, a handle, and a flag indicating if the file structure is being exported or un-exported. Un-export undoes what exporting process performs. Handle present in the message will be required for importing, as discussed later. In the case where a new device joins a network, the device will be able to request information regarding shared file system from others. Devices that have exported directory structures, upon receiving a request, will respond with an export message. Devices will be limited to exporting file structure that is physically resident on the device. A device will not be able to export directories imported from another device. This limitation will simplify the import process. In a device, an entity known as ExportManager, placed on top of M2MI layer, will be responsible for publishing and responding to request messages. Applications will register with export manager the directory structure to be shared by the device. The smallest file structure allowable for export will be a file. Another responsibility of the ExportManager will be to generate export messages when a file structure is un-exported or the distributed file system middleware is shutdown. Figure 1 shows local file system from two devices, A and B. Directory is exported by device A. Directories, b32, b41 and are exported from device B. 6 of 16

Device A Device B b11 b21 b22 b32 Exported file system b41 Exported file system Figure 1: Exported file structures by devices A and B. 2.2 Importing and Name resolution Devices interested in accessing shared file structures will be required to perform a bind operation. A successful bind will result in integration of a remote file structure into the local file structure by creating a union directory. Contents of multiple directories merged together forms a union directory. Figure 2 shows union directory formed after device A imports b41. Contents of b41 can be accessed through directory on device A. Device A b41 Union directory Figure 2: Union directory in device A b41. During importing, a device will specify four pieces of information. The entity to be modified in the local device called the bind point, which can only be a directory. The bind point could be a directory on the physical file system or it can be a foreign directory attached to the local file system. The entity being imported from a foreign device called the binding point can be a directory or file. The other two parameters will be a handle associated with imported file system and a bind parameter. 7 of 16

The bind parameter values, REPLACE, BEFORE or AFTER same as Plan 9, will define how the imported entities are attached to the local file system. REPLACE will attach the imported file structure by replacing the specified bind point. Replace option will only temporarily remove bind point from the local file system. In figure 3, directory is attached with REPLACE bind flag. In this case is a union directory consisting on one directory. When is unmounted by device A, the local directory will be accessible to the device again. BEFORE flag will attach the imported directory structure before the specified bind point. Bind parameter AFTER will form a union directory where the imported file system is attached after the bind point. Figure 3 shows modifications to device A s local file system after importing shared directories from device B. Device A a) Device A s original file system. b41 b) Bind operation with REPLACE parameter. Directory is imported to device A s local file system. A32 is replaced in A s local file system by. 8 of 16

b41 c) Bind with AFTER parameter. b41 d) Bind operation with BEFORE parameter. Figure 3: Binding foreign file systems. The unmount operation will be implemented in the middleware to undo changes made to a file system by a bind operation. This operation will be primarily used if a device decides to stop using a shared file system. The bind flag value specified during creation of union directory will play an important role during operations on union directories. Using Figure 3(c), a listing of union directory a32 will display the contents of preceded by contents of directory a32. Same principle will apply to operations such as opening a file. Consider the file system in Figure 3(b). If a file open system call was performed with / as the path and foo as the file name, subdirectory under will be searched first. If foo exist in / directory then the file //foo will be opened by the system call. If foo is not encountered in / then / will be searched. During file copy or file open system call in a union directory, the system call will be executed in the first directory with write permission. In the case the system call fails on the first directory 9 of 16

encountered with write permission, it will not continue to the next directory. For example, assuming both and from Figure 3(b) has write permission, a file copy command to union directory will only attempt to copy the file in. If the file cannot be copied to successfully, the system call will not continue to the next directory with write permission, in this case. 2.3 Operations on file system Basic file operations that will be supported by DFS middleware are listed below in the Table 1. Modifications to an imported file system resulting from any of basic operation will be reflected in the devices exporting the respective file system. Operations Description Create Create a file or directory Open Open a file Close Close a file Read Read from a file Write Write to a file Link Create a symbolic link to a file. Remove Remove a directory or file Move Move a file or directory Ls List directory content Table 1: File system operations supported by DFS middleware. In a device, operations invoked on a file system will be first handled by a virtual file system (VFS), resident in the middleware. The VFS will function as a filter, forwarding the system calls to either the underlying local file system or to a foreign device. Once the VFS receives an operation and determines the operation was executed in an imported directory, the operation will be redirected to a separate entity in the middleware. The module responsible for handling request from VFS will be DFSManager. A request arriving at the DFSManager will be delegated to DFSManager of the appropriate device in the network. Once a device receives a system call request, the request will be placed in FIFO queue. Each request will be removed from the queue and serviced. Result of the system call will be returned to the appropriate device once completed. 10 of 16

A major issue faced while developing a distributed file system is concurrent access to files. Current distributed file systems address this problem by providing a locking mechanism. In MDFS two types of locks, rlock and wlock, will be implemented for parallel access to regular files. Rlock will be designed as shared lock and wlock will be designed as exclusive lock. Table 2 shows locks granted for different types of access. Requested lock Current lock status None rlock wlock rlock Granted Granted Denied wlock Granted Denied Denied Table 2 Locks will be granted by MDFS for a specific amount of time, determined by the device granting the lock. A device acquiring a lock will have to renew the lock before it expires. If a lock expires, the lock granter will release the expired lock. This will allow a device to remove a lock on a file when the device holding the lock leaves the network without releasing it or suffers from a failure. In the case where a device that had granted locks suffers from a failure, the device will not undertake the task of re-granting the locks that existed prior to its failure. Devices that held locks on files resident of the failed devices will have to reapply for the lock. 2.4 Maintenance Maintenance mechanisms of the DFS middleware will include detection of devices leaving a network and modifying imported file structures to reflect changes in network topology. Aside from importing and file operations, a device s local file system would need modification when a device un-mounts an imported directory or file, and when an exported file system becomes unavailable. An exported file system can become unavailable if a device un-exports its shared directory, or device suffers from a failure. Modification to a device s local file system fall under two cases. If an imported directory is a leaf node in a local file system, when the directory becomes unavailable, the local file structure can be updated by just removing the node. Figure 4 demonstrates this concept. In Figure 4(a) when directory b41 becomes unavailable, b41 is removed from the local file system resulting in the file structure in figure 4(b). In the case where directory became unavailable, the same file system, Figure 4(a), would be updated by removing directory and its subdirectories, as in Figure 4(c). 11 of 16

Device A b41 Figure 4(a): Device A s local file system from figure 3(b) Figure 4(b): File structure from Figure 3(b). Directory b41 becomes unavailable. (c) B31 becomes unavailable File system modification becomes more complicated when an imported file structure, which are represented by internal nodes in the local file structure, becomes unavailable. Figure 5 displays this condition where imported directory forms internal node in the local file structure. Device A b41 c11 c33 Figure 5. Local file system on device A. Directories c11 and c33 are imported from device C. 12 of 16

In figure 5, if directory and its subdirectories b41 and become unavailable then c11 and c33 will be inaccessible in device A. To solve the dangling file structure issue, a shadow or pseudo-file structure can be employed. Using the same example as Figure 5, when device A detects that directory is unavailable, pseudo-file system nodes will be placed in the local file structure to allow access to c11 and c33. Figure 6 shows the updated file system from Figure 5 with pseudo-file structure in place after became unavailable. Once c11 and c33 directories are un-mounted or become unavailable, the pseudo-file structure can be removed from device A s local file system. Device A b41 c11 c33 Figure 6. Directories and its subdirectories b41 and form the pseudo-file system. C11 and c33 are not part of the pseudo-file system. Note: Directories with dashed border depicts pseudo-file system. The above-mentioned schemas will also be applicable when a change in a device s physical file system requires modification to its local file system. For example, figure 7 depicts directories c11 and c12 attached to and / respectively. In the case, and / was removed from Device A s physical file system, a pseudo-file structure will be situated to provide access to c11 and c33. 13 of 16

Device A b41 c11 c33 Figure 7(a). Device A s local file system. b41 c11 c33 Figure 7(b). Directories and / are represented by a pseudo-file structure. To implement the proposed mechanism of pseudo-file structure, a locking mechanism will be required. The sole purpose of this locking mechanism will be to prevent a process or application using MDFS middleware from accessing the affected segment in the local file system. Affected segment is any part a local file system that becomes unavailable and has to be represented by a pseudo-file structure. For example in Figure 6(b), directories and its sub-directories b41 and are the affected segment of device A s local file system. The locking mechanism will lock the affected segment s parent node, denying access to its subdirectories. In Figure 8, c2 is an imported directory that becomes unavailable and has to be represented by a pseudo-file structure. The proposed locking mechanism would lock c2 s parent denying access to the whole sub tree 14 of 16

rooted by the node b3. Once the pseudo-file structure is placed in the local file system, the lock on b3 will be removed. a b1 b2 b3 b4 b5 c1 c2 c3 c4 d1 d2 d3 d4 Figure 8: A device s local file system. 3 Deliverables The distributed File system middleware, two applications design to utilize the DFS middleware, and a technical report are going to be delivered at the end of the project. One of the applications will be shell program and both applications will use the same middleware API. 15 of 16

4 References [1] Alan Kaminsky and Hans-Peter Bischof. Many-to-Many Invocation: A new framework for building collaborative applications in ad hoc networks. CSCW 2002 Workshop on Ad Hoc Communication and Collaboration in Ubiquitous Computing Environments, New Orleans, Louisiana, USA, November 2002. [2] Rob Pike, Dave Presotto, Ken Thompson, Howard, Trickey, and PhilWinterbottom. The Use of Name Spaces in Plan 9. Bell Laboratories, Murray Hill, New Jersey 07974. 16 of 16