Distributed File Systems Part II. Distributed File System Implementation

Transcription

1 s Part II Daniel A. Menascé Implementation File Usage Patterns File System Structure Caching Replication Example: NFS 1

2 Implementation: File Usage Patterns Static Measurements: - distribution of file size, - distribution of file types. Dynamic Measurements: - arrival rate of read requests, - arrival rate of write requests, - arrival rate of requests per file size category, - arrival rate of requests per file type. Implementation: File Usage Patterns Most files are small (10K bytes) feasible to transfer entire files rather than just blocks. Reads are more frequent than writes. Few files are shared. Most files have a short life. client caching is appropriate, create file at client and keep until deleted. Random access is rare. 2

3 File Usage Patterns on the Web Arlitt and Williamson (1996) HTML and image files account for % of requests The average size of a transferred document does not exceed 21KB Less than 3% of the requests are for distinct files. The file size distribution is Pareto with 0.40 < α < i.e., this distribution is heavy-tailed. ( ) F( x) = P[ X x] = 1 k / x α File Usage Patterns on the Web Arlitt and Williamson (1996) Ten percent of the files accessed account for 90% of server requests and 90% of the bytes transferred. File inter-reference times are exponentially distributed and independent. At least 70% of the requests come from remote sites. These requests account for at least 60% of the bytes transferred. 3

4 Implementation: System Structure Separation of file service and directory service: - directory service: symbolic name binary name (e.g. machine + i-node) - file service: reads and writes given binary names. Implementation: System Structure Separation of file service and directory service: different directory services (e.g., MS-DOS and UNIX can map to the same physical file system. File Name Lookup 4

5 Implementation: Name Lookup lookup a/b/c a SV2:23 lookup b/c 23 b SV3: 54 lookup c client 54 c SV3: I-node for file c Implementation: Name Lookup lookup a/b/c a SV2:23 23 lookup b/c b SV3: 54 client 54 lookup c c SV3: I-node for file c 5

6 Implementation: Name Lookup cache hint a/b/c SV3:231 server SV3 client c SV3: I-node for file c Implementation: State Info Stateless Servers: - no information is kept about a request after the request is served. - request has to be self contained (e.g., contain complete file names) - longer messages required. - mapping required for each request. - better tolerance to server crashes. 6

7 Implementation: State Info Stateless Servers: - no tables required no limit on the number of clients. - file locking not possible requires a locking server. - more difficult to achieve idempotency. Implementation: State Info Stateful Servers: - information kept about each client that has an open file. - request to open file needs complete file name server returns file descriptor. - other requests need file descriptor only mapping not required for each request. - difficult recovery from server crashes. 7

8 Implementation: Caching client main memory cache disk cache server main memory cache permanent file storage client Implementation: Caching server Client caching: - reduces server I/O and network traffic, - creates cache consistency problems Caching unit: files or disk blocks? Server main memory caching: - reduces I/O at the server disk 8

9 Cache Performance File Block Size = 16Kbytes; Avg. Disk Service Time = sec Avg. Client Think Time = 5 sec Network Effective Bandwidth = 1.25 Mbps CPU Time per request = 0.01 sec Client cache: Main memory hit ratio = 0.2 Disk hit ratio = 0.3 Server cache hit ratio = 0.3 network is the bottleneck. Cache Performance Throughput Throughput vs. No. Clients hcl,main= 0.2 hcl,disk = 0.3 hsv= 0.3 Series No. Clients network is the bottleneck 9

10 Cache Performance hcl,disk Max Xput Max Clients req/sec req/sec req/sec 280 cache hit Client Cache Location Issues cache miss cache within user process to/from server low overhead good when processes open/close the same file many times cache is lost when process exits. 10

11 cache hit Client Cache Location Issues cache miss cache in the call required for each access cache survives process to/from server cache hit Client Cache Location Issues cache miss cache manager process in user space to/from server code free of system code cache survives process cached block may be paged out by the! 11

12 Cache Consistency Write-through: - every time a block is modified at the client, send modification to server. - cache is effective for read traffic but not for write traffic - if client cache survives processes, then cache currency has to be validated with the server (version numbers or checksums can be used here). Cache Consistency Delayed write: - make a note that a file has been modified but do not inform the server immediately. - send all modifications as a batch to the server every 30 seconds more efficient. - reduces write traffic for temporary files that are written, read, and deleted before the server needs to be notified. - cleaner semantics is traded for better performance what other processes read is time dependent. 12

13 Cache Consistency Write on close: - adopt session semantics and write back to the server 30 seconds after file is closed deleted files are never sent to the server. - still possible for writes to be lost. If two or more processes have the file open for write, only one wins. Similar problem may arise in centralized systems if no locking is used. Cache Consistency Centralized controller: - a centralized controller keeps track of all files that are open and their respective clients. - conflicting requests to open files can be handled in three ways: deny request queue request grant request but notify all clients that have the file open to remove it from their cache and disable caching unsolicited messages to clients is required. - does not scale and is not robust. 13