File Systems In a general purpose operating system the local file system provides A naming convention A mechanism for allocating hard disk space to files An method for identifying and retrieving files, usually hierarchical Naming Convention Most current popular operating systems allow file names of up to 256 characters in length, with no restriction on spaces or periods in the name. In contrast, older file system naming conventions, such as DOS, restricted both the length of a file name and the character set which could be used. DOS in particular was quite restrictive, allowing only 11 characters in an 8.3 format. That is, the filename consisted of an 8-character name followed by a 3-character extension, at the most; the name portion could be as short as a single character, and an extension was not required at all. No other periods beyond the one which separates the name from the extension are allowed and no spaces are allowed. Legal file names in DOS include letters.txt memo1.doc abra1234 x Illegal file names include abracadabra my notes.doc my/letter.txt Long file names have always been allowed in Linux (and UNIX) but not in Windows until Windows 95B. Multiple spaces and periods are allowed in these file names. In order to provide backwards compatibility, Windows has a mechanism for compressing a long file name into the 8.3 format. This consists of taking the first six characters of the long file name, appending a tilde (~) and a single digit integer and using the result for the name portion of the file name. The extension, if any, of the long file name is then attached. For example, the long file names Neal Christensen.exe and Neal Christensen Jr.exe would be compressed into NealCh~1.exe and NealCh~2.exe Notice that spaces are removed, and the integer is changed when two files have the NTC 2/22/05 22
same letters in the first six positions. The use of an extension on the end of a filename is not universally adopted. Linux does not recognized file extensions as special in any way 15 ; typically extensions are used in Linux for compatibility with DOS/Windows based systems. In DOS (and windows) certain extensions have special meanings: the extensions bat, com, and exe all designate executable files (programs) while sys is used to denote a system file. In Windows ini was used (before the registry was in common use) to identify initialization (configuration) files for applications. Beyond these, users and applications were free to specify extensions in any way that suited them. Over time, some extensions became associated with certain applications, such as lws for a Lotus worksheet, doc for an MS Word document, and pdf for an Adobe Acrobat file. Eventually, Windows cast some of these in concrete by allowing the user to have an application automatically be opened when a file or icon with the appropriate extension was double-clicked using the mouse. File Allocation Probably the most critical task of a general purpose operating system s file system is to allocate hard disk storage for programs and data, and provide an access method for finding a file on a disk and returning it to the user. Let s first review how data is stored on a hard disk. A hard disk consists of several magnetic-oxide coated aluminum platters. A single drive may have several platters, with two recordable surfaces on each platter and a read/write head on each surface. Each surface is logically formatted into a number of concentric tracks; all of the tracks, one per surface, which are at the same distance from the drive spindle constitute a cylinder. Each track is divided up into segments called sectors each of which typically is used to store 512 bytes 16, which is the minimum block of storage which may be either written to or read from on the disk. Even a file containing only a single byte will use up an entire sector of 512 bytes. All file allocation mechanisms need to keep track of which sectors are in use and what file is stored in the sector. With the current size of hard drives well into the hundreds of Gigabytes it should be clear that a lot of storage will be used to keep track of each 15 In fact, Linux doesn t distinguish between types of files, as a rule, in any way. In Linux, programs, data, directories, and even I/O devices are all considered files and treated the same. 16 Thus, the capacity of a disk drive can be calculated by multiplying the number of heads x the number of cylinders x the number of sectors per track x 512. NTC 2/22/05 23
sector. For instance, for a hard drive containing 100Gigabytes (100 x 10 9 bytes) there will be 100 x 109/512 = 200 Million sectors (roughly) to keep track of. In order to reduce the size of storage needed for these tracking mechanisms we need to increase the size of the allocation units to something larger than single segments. These larger allocation units are called clusters and consist of some number of contiguous sectors. A typical cluster size might be between 4 Kilobytes and 32 KB, or from 8 contiguous segments to 64 segments. File Allocation Table (FAT) The FAT mechanism is used in DOS, Windows 95 and Windows 98. The original FAT system was a FAT12 system, for use in systems without hard drives. The first hard drive systems (IBM XT) used a system called FAT16 and Windows 95B and Windows 98 used a FAT32 system. The meaning of the numbers will be explained shortly. A File Allocation Table is a table containing an entry for every allocation unit (cluster) on the hard drive. It acts as a linked list in that the contents of each entry is simply a pointer to the cluster which contains the next portion of a file. For example, consider a file is large enough to occupy five clusters. The file itself is then broken up into five pieces, let s call them A, B, C, D, and E. The file system elects to store these five file parts into clusters as follows: File Part Cluster A 3 B 4 C 1 D 7 E 8 The FAT, then looks as follows NTC 2/22/05 24
FAT Entry Pointer 0 1 7 2 3 4 4 1 5 6 7 8 8 EOF 9 In addition to the FAT the file system also contains a Directory (usually more than one). Each entry in the directory contains, among other things, the File name and a pointer to the entry in the FAT corresponding to the first part of the file. In the current example, the directory pointer will point to entry three since cluster 3 contains the beginning of the file. When the first part of the file is accessed, the pointer in entry 3 of the FAT is examined and tells us that the second part of the file (B) can be found in cluster 4. The operation continues accessing the file from clusters1,7, and 8. The pointer in the FAT at entry 8 is an End of File (EOF) marker, indicating that this cluster contains the end of the file. The amount of space allocated to each entry in the FAT limits the size of possible pointers which, in turn, limits the number of addressable clusters, which then places a maximum limit on the size of a hard drive which can be accessed with a given FAT design. The original DOS FAT system was a FAT12 system, meaning that each entry contained 12 bits. Thus the largest number of clusters that can be accommodated is 2 12 or 4096 clusters. The cluster size for those systems was 1 segment (512 Bytes), so the maximum size floppy disk size theoretically usable was 4K x 512 = 2MB. Windows 98 systems use a FAT32 design, so each entry contains 32 bits. A typical cluster size in Windows 98 is 4KB (although it can be increased up to 32KB), so Windows 98 could accommodate disks on the order of 2 32 x 4KB = 16 TeraBytes. NTC 2/22/05 25
NTFS NTFS is the file system used by Windows XP, and some other systems such as OS/2. Instead of a FAT NTFS uses a Master File Table (MFT). The MFT consists of a single 1KB record for each file. This 1KB record contains all of the directory information regarding the file and also, in whatever room is left over, the start of the file s data itself. A very short file may need no more than the single MFT entry. If the file is larger than can fit in the MFT entry, then the entry contains a pointer to an extent which contains the rest of the file. Extents are allocated in 1KB increments. ext3 Linux uses the extended file system, originally called extfs. It is now in its 3 rd version and is called ext3. The allocation method used by ext3 uses index blocks, called i- nodes. Each i-node is a small table containing 15 entries. There is at least one i-node for each file, which is pointed to by the directory entry for that file. The first 12 entries in the first i-node each point directly to an allocation unit for the file. If the file is larger than can be accommodated by 12 allocation units, then the 13 th entry points to another i-node which contains 15 more pointers to allocation units. The 14 th entry points to an i- node whose entries each point to other i-nodes, which in turn point to allocation units. And, as might be expected, the 15 th entry points to an i-nodes which points to i-nodes which point to a third level of i-nodes. Paths The third facility a file system supplies the user is a mechanism for logically organizing and specifying the location of files. The general method is to provide a hierarchical structure for locating files. In this structure we view files as the leaves of a tree. The files are grouped together in directories (Linux) or folders (Windows). These directories/folders may themselves be grouped with other directories/folders into directories at a higher level in the hierarchy, and so on. At the top of the structure is the root of the directory structure. An important concept in accessing files is that of the current working directory. Think of this as the directory which will be the default target of operations barring any information to the contrary. Thus, if you try to access a file called, say, File1.doc, the current directory is the first (and frequently only) one to be searched for the file. If the file is not in the current directory an error message may be returned to you indicating that the file cannot be found. If there really is a File1.doc somewhere on the local NTC 2/22/05 26
system, you have two options 17 : either change the current directory to the one containing File1.doc, or explicitly specify the path to the appropriate directory. Consider the diagram below. Directory Tree At the top of the tree, represented by the forward slash (/), is the root of the tree. Any of the directories shown might be the current working directory; let s assume it is wwwroot. One unambiguous way to access the file dalek.htm is to specify its absolute path. An absolute path explicitly identifies all the directories in the path from the root to the file. To print dalek.htm we would use print /inetpub/wwwroot/dalek.htm However, since we have specified that wwwroot is the current working directory, the path to the current working directory, /inetpub/wwwroot can be assumed by default. With this assumption, then, the file dalek.htm can be accessed simply by using the file s 17 Actually, there are other options, such as the PATH environment variable, which are beyond the scope of this discussion. NTC 2/22/05 27
name, as in print dalek.htm This is known as a relative reference or a relative path to the file since it is specified with respect to the current directory. Let s assume that there is a file called config.ini in the inetpub directory. We could access this file from wwwroot with the absolute file specification /inetpub/config.ini or we could use the relative file specification../config.ini Note the double period symbol preceding the /. In many operating systems, including both Windows and Linux, this is a special notation meaning the parent directory of the current directory. Since inetpub is the parent of wwwroot, this example is equivalent to the absolute path shown previously. This is useful for navigating a tree in which you may not be sure of the name of your parent directory. Assume that the current directory has now been changed to gif and that we don t know the name of the parent directory of gif, but we do know that tardis.jpg is located in a sibling directory called jpeg. We would than access the file in that directory with the file specification../jpeg/tardis.jpg If the file of interest is below the current directory, we start a relative path specification with just the directory and/or file names. If the current directory is images then the following is a relative path: jpeg/tardis.jpeg One last example to contemplate; if the current directory is wwwroot then the following relative path accesses gallifray.gif. Note the double use of the parent symbol.../../images/gif/gallifray.gif In general, if a path does not start with a slash, it is relative, otherwise it is absolute. An exception to this rule occurs when the physical drive is specified as part of the path. In a Windows (but not Linux) file system a file on another drive is accessed by specifying the drive letter followed by a colon in front of the path. for instance, NTC 2/22/05 28
C:\Images\Jpeg\tardis.jpg This is also an absolute file name since the file specification does not depend on the current directory. Network File Systems The three file systems discussed above (FAT, NTFS, ext3) are all local file systems and do not address the problem of accessing files on a remote host. This can be accomplished in a general purpose operating system with the use of standalone programs such as Telnet and FTP. Telnet establishes a session with a remote host using logins and passwords, after which you access the machine as if you were sitting at the remote machine; your local host becomes a remote terminal for the host you are logged into. FTP (File Transfer Protocol) sets up a link between your local host and the remote host, but only allows you to send and receive files over the link. We discuss some other methods of accessing files on a remote host. Sharing Windows LANs use sharing to allow hosts to access each others files. In this scheme, the operating system requires two steps: 1) enable file sharing (called File and Printer Sharing in Windows) and 2) explicitly marking those directories which a remote host will be allowed to access. Once these steps have been accomplished, anyone on any host on the LAN can access the shared files. However, the name of the host (the netbios name, or hostname) sharing its files must be known to any user wishing to access them. By example, let s consider the host containing the directory tree shown above. The host s netbios name is drwho. Furthermore, assume that drwho has shared the directory jpeg, with the same name, jpeg 18 Then a remote host would access the file tardis.jpg with the following syntax \\drwo\jpeg\tardis.jpg 19 Note that we indicate that a remote host is being referenced by preceding it with a double backslash. 18 In windows, when you share a folder you have the option of giving it an alias, thereby hiding the actual folder name from remote users. 19 Note that in Windows a backlsash (\), rather than a forward slash (/), is used to identify the root of the tree and to separate folders in the path specification. Linux uses forward slashes. NTC 2/22/05 29
Network Drives A network drive is simply a remote shared folder which has been given a drive letter to make it easier to access share files on a remote system. Remember that in a DOS/Windows type of system, drives are named by the letters of the alphabet followed by a colon. Typically A: is the first floppy drive B: is the second floppy drive C: is the first hard drive D: usually a CD-ROM or 2 nd hard drive. Any unused drive letters may assigned to other devices. In the above example, using an operating systems mapping facility, we can map the shared folder to a network drive with a command such as map N: \\drwho\jpeg Once this has been done, the files in the shared folder can be accessed with the file specification N:\tardis.jpg Network File System (NFS) Linux uses NFS to allow hosts remote access to a local hosts files. Each local host specifies which subtrees it wants to allow access to in a file called exports. For each subtree in this file the administrator also identifies which users or groups of users may access the subtree and also assigns some permissions (such as read-only or readwrite ) for each user. Any user on a remote host who wants to access a file in a subtree they have been granted permission for needs to mount the subtree into their own file system. This makes the exported subtree appear as if it were part of the remote hosts own file system hierarchy. Note that the host accessing an exported file has no indication as to which host or device the mounted file system is actually on 20. 20 Although the exporting host must be known at mount time. If an administrator does the mounting than a user may not know which hosts files they are looking at. NTC 2/22/05 30