DFS Case Studies, Part 1

Size: px

Start display at page:

Download "DFS Case Studies, Part 1"

Denis Lester
6 years ago
Views:

1 DFS Case Studies, Part 1 An abstract "ideal" model and Sun's NFS

2 An Abstract Model File Service Architecture an abstract architectural model that is designed to enable a stateless implementation of the DFS server module Sun NFS the Network File System, introduced in 1985, first commercial product/success Andrew File System developed by Carnegie Mellon University in 1986, with "production" technologies released after 1989

3 Case Study Abstract DFS Architecture The DFS is structured around three components a flat file service, a directory service and a client module The flat file and directory service each export an interface to be used by client programs The client module provides a single/simple programming interface with operations on files similar to those found in conventional file systems The design is open

4 The Abstract DFS Architecture Client computer Server computer Application program Application program Directory service Flat file service Client module

5 The Flat File Service Concerned with implementing operations on the contents of files Unique File Identifiers (UFIDs) are used to refer to files Each file has a UFID that is unique among all the files in a distributed system A request to create a file generates a new UFID for it and returns the UFID to the requester

6 The Directory Service Provides a mapping between text names for files and their associated UFID Provides the functions necessary to generate directories, to add new files to directories and to obtain UFIDs from directories It is a client of the Flat File Service Directories hold references to other directories when a hierarchic file naming scheme is adopted

7 The Client Module Runs on each client computer Provides a single API that is available to user level programs in client computers Holds information about the network locations of the flat file server and directory server processes Achieves satisfactory performance by implementing and maintaining a cache of recently used file blocks

8 Flat File Service Interface (RFCs) Read(FileId, i, n) > Data throws BadPosition Write(FileId, i, Data) throws BadPosition Create() > FileId Delete(FileId) If 1 i Length(File): Reads a sequence of up to n items from a file starting at item i and returns it in Data. If 1 i Length(File)+1: Writes a sequence of Data to a file, starting at item i, extending the file if necessary. Creates a new file of length 0 and delivers a UFID for it. Removes the file from the file store. GetAttributes(FileId) > Attr Returns the file attributes for the file. SetAttributes(FileId, Attr) Sets the file attributes

9 Comparison with UNIX Differs from UNIX mainly in the area of fault tolerance Repeatable operations excluding "Create", all the operations are idempotent, using at least once semantics Note: repeated calls on "Create" result in a new file (UFID) for each call Stateless servers can be restarted after a failure and can resume operations without any need for clients or the server to restore any state

10 Access Control Access rights have to be performed at the server The server is vulnerable to forged identities Whenever a file name is converted to a UFID, an access check is performed and the results are encoded in the form of a capability, which is used by the client in all subsequent operations Access checks are performed by the server for every file operation

11 The Directory Service Interface (RFCs) Lookup(Dir, Name) > FileId throws NotFound AddName(Dir, Name, FileId) throws NameDuplicate UnName(Dir, Name) throws NotFound GetNames(Dir, Pattern) > NameSeq Locates the text name in the directory and returns the relevant UFID. If Name is not in the directory, throws an exception. If Name is not in the directory, adds (Name, File) to the directory and updates the file s attribute record. If Name is already in the directory: throws an exception. If Name is in the directory: the entry containing Name is removed from the directory. If Name is not in the directory: throws an exception. Returns all the text names in the directory that match the regular expression Pattern.

12 Notes on the DSI Note: Exceptions caused by inadequate access rights are omitted from the definitions "AddName" adds an entry to a directory and increments the reference count "UnName" removes an entry from a directory and decrements the reference count

13 Hierarchic File System The file system consists of a number of directories arranged in a tree structure The directories hold the names of files and other directories that are accessible from that point Any file or directory can be referenced using a "pathname" a multi part name that represents a path through the tree structure

14 File Groups A collection of files located on a given server A server may hold several file groups, and groups can be moved between servers However, a file cannot change the group that it belongs to Supports the allocation of files to file servers in larger logical units that enable the service to be implemented with files stored on several servers File groups identifiers must be unique throughout a distributed system

15 Case Study Sun NFS NFS has been successful both technically and commercially Its specification was placed in the public domain in 1989 NFS version 3 is an Internet standard, defined in RFC 1813 Provides transparent access to remote files for client programs running on UNIX and other systems Every client in an NFS network can also function as a server the relationship is symmetrical

16 Important Goal of NFS To achieve a high level of support for hardware and software heterogeneity Versions of NFS are available for (most versions of): Windows, Mac OS X, Linux, BeOS and every other version of UNIX (including, of course, Sun OS and Solaris)

17 Sun NFS Architecture Client computer Server computer UNIX system calls UNIX kernel Application program Application program Virtual file system UNIX kernel Virtual file system Local Remote UNIX file system Other file system NFS client NFS protocol NFS server UNIX file system

18 The NFS Protocol A set of remote procedure calls that provide the means for clients to perform operations on a remote file store OS independent and can be configured to support UDP or TCP Version 3 is a widely deployed standard (with version 4 under advanced development and used at some sites) Sun's RPC technology was primarily developed to support NFS The RPC interface is open any client process can send a request to an NFS server, and if the request and user credentials are valid, the request is acted upon Signed user credentials and encryption of data are both optional security features

19 The NFS Virtual File System NFS provides access transparency A virtual file system (VFS) module distinguishes between local and remote files and translates between the UNIXindependent file identifiers used by NFS and the internal file identifiers normally used in UNIX and other file systems VFS keeps track of the file systems that are currently available both locally and remotely, then passes each request to the appropriate local system module

20 File Handles, Groups and v nodes The file identifiers used in NFS are called "file handles" NFS adopts the UNIX mountable filesystem as the unit of file grouping An NFS v node contains an indicator to show whether a file is local or remote If NFS is dealing with a local file, an i node (in UNIX) is used, otherwise the file handle to the remote file is used

21 NFS Client Integration NFS supplies an interface suitable for use by conventional applications programs Emulates the semantics of the standard UNIX file system primitives precisely and is integrated with the OS kernel

22 OS Kernel Integration Use programs can access files via UNIX system calls without recompilation/reloading A single client module serves all of the user level processes, with a shared cache of recently used blocks The encryption key used to authenticate user IDs passed to the server can be retained in the kernel, preventing impersonation by user level clients

23 How it Works The NFS client module cooperates with the virtual file system in each client machine It operates in a similar manner to the conventional UNIX file system However... since several clients in different host machines may simultaneously access the same remote file, a new and significant cache consistency problem arises

24 Access Control/Authentication Unlike UNIX, the NFS server is stateless NFS does NOT keep files open on behalf of clients Each request needs to be checked for validity afresh Authentication information can be supplied automatically by the underlying RPC mechanism

25 NFS Security Loophole The client formulating the request can modify the RPC calls to include the user ID of any user, impersonating the user without their knowledge or permission This can be fixed by using DES encryption of the user's authentication information within the RPC More recently, Kerberos has been integrated into NFS to provide even greater security/authentication

26 NFS Server Interface, part 1 lookup(dirfh, name) > fh, attr create(dirfh, name, attr) > newfh, attr remove(dirfh, name) status getattr(fh) > attr setattr(fh, attr) > attr read(fh, offset, count) > attr, data write(fh, offset, count, data) > attr rename(dirfh, name, todirfh, toname) > status link(newdirfh, newname, dirfh, name) > status Returns file handle and attributes for the file name in the directory dirfh. Creates a new file name in directory dirfh with attributes attr and returns the new file handle and attributes. Removes file name from directory dirfh. Returns file attributes of file fh. (Similar to the UNIX stat system call.) Sets the attributes (mode, user id, group id, size, access time and modify time of a file). Setting the size to 0 truncates the file. Returns up to count bytes of data from a file starting at offset. Also returns the latest attributes of the file. Writes count bytes of data to a file starting at offset. Returns the attributes of the file after the write has taken place. Changes the name of file name in directory dirfh to toname in directory to todirfh. Creates an entry newname in the directory newdirfh which refers to file name in the directory dirfh.

27 NFS Server Interface, part 2 symlink(newdirfh, newname, string) > status readlink(fh) > string mkdir(dirfh, name, attr) > newfh, attr rmdir(dirfh, name) > status readdir(dirfh, cookie, count) > entries statfs(fh) > fsstats Creates an entry newname in the directory newdirfh of type symbolic link with the value string. The server does not interpret the string but makes a symbolic link file to hold it. Returns the string that is associated with the symbolic link file identified by fh. Creates a new directory name with attributes attr and returns the new file handle and attributes. Removes the empty directory name from the parent directory dirfh. Fails if the directory is not empty. Returns up to count bytes of directory entries from the directory dirfh. Each entry contains a file name, a file handle, and an opaque pointer to the next directory entry, called a cookie. The cookie is used in subsequent readdir calls to start reading from the following entry. If the value of cookie is 0, reads from the first entry in the directory. Returns file system information (such as block size, number of free blocks and so on) for the file system containing a file fh.

28 NFS Mount Service A separate process runs at the user level on each NFS server called the "mount service" The /etc/exports file (well known location and name) contains the names of local filesystems that are available for remote mounting Each mount service also maintains an access list indicating which hosts are permitted to mount the filesystem Clients use a modified version of the UNIX "mount" command to request mounting of a remote filesystem

29 A Client with Remote Mounts Server 1 (root) Client Server 2 (root) (root) export... vmunix usr nfs people Remote mount students x staff Remote mount users big jon bob... jim ann jane joe Note: The file system mounted at /usr/students in the client is actually the sub tree located at /export/people in Server 1; the file system mounted at /usr/staff in the client is actually the sub tree located at /nfs/users in Server 2.

30 Hard and Soft Mounts Hard mounting the process is suspended until the request can be completed and if the remote host is unavailable for any reason the NFS client module continues to retry the request until it is satisfied Soft mounting the NFS client module returns a failure indication to the user level processes after a small number of (unsuccessful) retries As many UNIX utilities do NOT test for failure, hard mounting tends to be used exclusively with UNIX environments programs can therefore not recover gracefully when an NFS server is not available Path name translation and automounting is also supported

31 Server Caching In order to achieve adequate performance, caching is used at both the client and the server computer In conventional systems, a main memory buffer cache is used "read ahead" pre fetches the next most likely page and "delayed write" optimizes writes (with flushing performed by the "sync" command) NFS servers use the cache at the server machine just as it is used for all other files

32 Server Caching The Details When a server performs write operations, extra measures are needed to ensure that clients can be confident that the results of write operations are persistent Option 1 data in write operations is stored in memory and written to disk BEFORE a reply is sent to the client this is called Write Through Caching Option 2 data is stored in the server's memory and only written to disk when a "commit" request is received from the client (this is the standard way it works in NFS)

33 Client Caching Clients are responsible for polling the server to check the currency of the cached data that they hold A timestamp based method is used to validate cached blocks before they are used

34 Other NFS Optimizations The Sun filesystem is based on the UNIX BSD Fast File System which uses 8 k byte disk blocks The UDP packets used for the implementation of Sun RPC has been extended (in NFS) to 9 k bytes, enabling the entire disk block to be carried in one packet File attribute information (provided by the getattr system call) is piggybacked along with the results of many other operations, resulting in the reduction of explicit status information requests

35 NFS Performance Early performance figures showed that the use of NFS did not normally impose a performance penalty in comparison with access to files stored on local disks A problem remains in that frequent use of the getattr call in order to fetch timestamps from the server (for cache validation) have turned out to be expensive As write through caching is used at the server, there has been relatively poor performance when an application is made up primarily of write operations

36 NFS Summary NFS closely follows the abstract model There's good location and access transparency NFS supports different and many hardware and OS combinations NFS is stateless after a failure, it resumes immediately without the need for expensive recovery operations

37 Meeting the Design Goals, 1 of 3 Access transparency the NFS API is identical to that used by the local OS; existing programs work unaltered within an NFS setup Location transparency NFS does not enforce a single, network wide file namespace; each client sees the filesystem their way (resulting in the same file having different filenames on different clients) Mobility transparency migration transparency is not fully achieved within NFS

38 Meeting the Design Goals, 2 of 3 Scalability NFS can be built to handle very large realworld loads in an efficient and cost effective manner File replication read only replicas are supported, but file replication with updates is not Hardware and OS heterogeneity NFS is implemented on nearly all known OSes and hardware platforms; supports a variety of filing systems

39 Meeting the Design Goals, 3 of 3 Fault tolerance failure modes for remote files are similar to those for local files; once a failed server has been restarted, clients continue from the point at which the service was interrupted; client failures have no effect on the "stateless" server Consistency NFS provides a close approximation of one copy semantics, satisfying most application needs Security Kerberos can be added to NFS if required; RFC 2203 describes a "secure RPC" which can also be used; many NFS uses implement NO security whatsoever Efficiency NFS is widely using in a wide variety of situations; it can easily handle very heavy processing loads

Chapter 12 Distributed File Systems. Copyright 2015 Prof. Amr El-Kadi

Chapter 12 Distributed File Systems Copyright 2015 Prof. Amr El-Kadi Outline Introduction File Service Architecture Sun Network File System Recent Advances Copyright 2015 Prof. Amr El-Kadi 2 Introduction