Files and directories. Updated by: Dr. Safwan Qasem Spring 2010 Original version created by: Dr. Mohamed El Bachir Menai

Files and directories Updated by: Dr. Safwan Qasem Spring 2010 Original version created by: Dr. Mohamed El Bachir Menai 1

Files and Directories Objectives Additional Features of the File System Properties of a File. Three major functions that return file information: #include <sys/types.h> #include <sys/stat.h> int stat(const char *pathname, struct stat *buf); int fstat(int filedes, struct stat *buf); int lstat(const char *pathname, struct stat *buf); Return: 0 if OK, -1 on error 2

Files and Directories UNIX kernel maintains considerable details about every file system object whether it is: File Directory Special device Named pipe (FIFO) stat(), fstat(), and lstat() return information about file system objects in a structure named stat. 3

Files and Directories Differences on stat(), fstat(), lstat(). lstat() similar to stat(), except that, when file is a symbolic link, it returns info regarding the symbolic link itself, not the referenced file. File info is returned in buf defined as the following structure: struct stat { mode_t st_mode; /* type & mode */ ino_t st_ino; /* i-node number */ dev_t st_dev; /* device no (filesystem) */ dev_t st_rdev; /* device no for special file */ nlink_t st_nlink; /* # of links */ uid_t st_uid; gid_t st_gid; off_t st_size; /* sizes in bytes */ time_t st_atime; /* last access time */ time_t st_mtime; /* last modification time */ time_t st_ctime; /* time for last status change */ long st_blksize; /* best I/O block size */ long st_blocks; /* number of 512-byte blocks allocated */ }; 4

File Types Regular Files: text, binary, etc. Directory Files: Only Kernel can update these files { (filename, i-node pointer) }. Character Special Files, e.g., tty, audio, etc. Block Special Files, e.g., disks, etc. FIFO named pipes Sockets Symbolic Links not POSIX.1 or SVR4 5

File Types File types can be determined using following macros; argument of macro is st_mode (which encodes file type) member from the stat structure. Macro S_ISREG() S_ISDIR() S_ISCHR() S_ISBLK() S_ISFIFO() S_ISLNK() S_ISSOCK() Type of File Regular file Directory Character special file Block special file FIFO or PIPE Symbolic link Socket 6

#include <sys/types.h> #include <sys/stat.h> int main(int argc, char *argv[]) { int i; struct stat buf; char *ptr; for (i = 1; i < argc; i++) { printf("%s: ", argv[i]); if (lstat(argv[i], &buf) < 0) { err_ret( lstat error"); continue; } if (S_ISREG(buf.st_mode)) ptr = "regular"; else if (S_ISDIR(buf.st_mode)) ptr = "directory"; else if (S_ISCHR(buf.st_mode)) ptr = "character special"; else if (S_ISBLK(buf.st_mode)) ptr = "block special"; else if (S_ISFIFO(buf.st_mode)) ptr = "fifo"; else if (S_ISLNK(buf.st_mode)) ptr = "symbolic link"; } exit(0); } else if (S_ISSOCK(buf.st_mode)) ptr = "socket"; else ptr = "** unknown mode **"; printf("%s\n", ptr); 7

File Types Example of execution: $ a.out /vmunix /etc /dev/ttya /dev/hd0a gives: /vmunix: regular file /etc: directory /dev/ttya: character special device /dev/hd0a: block special device $./filetype_stat /etc/passwd /etc /dev/log /dev/cdrom /dev/tty /dev/sr0../toto $./filetype_lstat /etc/passwd /etc /dev/log /dev/cdrom /dev/tty /dev/sr0../toto 8

Set-User-ID and Set-Group-ID Every process has 6 or more IDs associated with it. Real user ID who we really are Real group ID Effective user ID Effective group ID Used for file access permission checks Supplementary group ID Saved set-user-id Saved by exec function Saved set-group-id Real user ID and real group ID are taken from user entry in passwd file when user logs in. No change during a login session. Effective IDs and supplementary IDs control user file access permissions. (can be different from real user and group ID depending on set-user-id and set-group-id bits in st_mode. Saved IDs contain copies of effective IDs when a program is executed. 9

Set-User-ID and Set-Group-ID Every file has an owner and an group owner. -The owner is specified by the st_uid of the stat structure. -The group owner is specified by the st_gid member. When a program is executed, -The effective user is usually the real user. -The effective group ID is usually the real group ID. The flag st_mode can be used to set the effective user ID of the process to be the owner of the file (st_uid). Similarly the effective group ID can be set to be the group owner of the file. Example: Passwd(1) is a UNIX system program that allows anyone to change his or her password. This program is a set-user-id program. The program needs to write the new password to the password file (/etc/passwd or /etc/shadow) that should be writable only by the superuser set-user-id and set-group-id bits contained in the st_mode word can be tested through stat function using constants S_ISUID and S_ISGID. 10

File Access Permissions st_mode word encodes access permissions for files and directories as Permission Bits According to permissions, allowed operations are: For a Directory * X pass through the dir (search bit). Example: to open file /usr/student/ali, we need execute permissions in /usr and in /usr/student. * R list of files under the directory (different from X!) * W update the dir, e.g., delete or create a file (both need also X bit). For a File * X execute a file (which must be a regular file) * R can open file for reading. O_RDONLY or O_RDWR * W can open file for writing. O_WRONLY, O_RDWR, or O_TRUNC 11

Permission bits st_mode mask S_IRUSR S_IWUSR S_IXUSR S_IRGRP S_IWGRP S_IXGRP S_IROTH S_IWOTH S_IXOTH meaning User ( i.e OWNER) read User ( i.e OWNER) write User ( i.e OWNER) execute Group read Group write Group execute Other read Other write Other execute 12

Access Permissions & UID/GID File Access Test: Tests performed by OS when process opens, creates or deletes file. Depend on the owners of the file (st_uid, st_gid), the effective IDs of the process (EUID, EGID) and the supplementary groups. 1. If the effective UID == 0 superuser! access allowed 2. If the effective UID == owner UID of the file If appropriate access permissions, access allowed, otherwise denied 3. If the effective GID == owner GID of the file If appropriate access permissions, access allowed, otherwise denied 4. Check appropriate access permissions for others same way. These 4 steps are tried in sequence. Related Commands: chmod & umask 13

Ownership of a New File UID of a new file = the effective UID of the creating process. GID of a new file 2 options defined by POSIX: 1. GID of the new file = the effective GID of the process 2. GID of the new file = the GID of the directory in which the file is created. 14

Function access: file accessibility tests Accessibility tests are performed by OS on open operation, based on effective UID and GID. Sometimes, need to test accessibility for a file based on real UID and GID. #include <unistd.h> int access(const char *pathname, int mode); Returns: 0 if OK, -1 on error mode is the bitwise OR of any of the constants: R_OK, W_OK, X_OK, F_OK (file existence) 15

#include <sys/types.h> #include <fcntl.h> int main(int argc, char *argv[]) { if (argc!= 2) err_quit("usage: a.out <pathname>"); if (access(argv[1], R_OK) < 0) err_ret("access error for %s", argv[1]); else printf("read access OK\n"); if (open(argv[1], O_RDONLY) < 0) err_ret("open error for %s", argv[1]); else printf("open for reading OK\n"); exit(0); } Example of execution (fig4.8): $ a.out a.out read access OK open for reading OK 16

Function umask Defines the file mode CREATION MASK. The permissions defined in the mask will be used as default permissions for newly created files. Function umask sets the file mode creation mask for the process and returns the previous value. #include <sys/types.h> #include <sys/stat.h> mode_t umask(mode_t cmask); Returns: previous file mode creation mask cmask = bitwise OR of any of file permissions (S_IRUSR, S_IWUSR, S_IXUSR, S_IRGRP, S_IWGRP, S_IXGRP, S_IROTH, S_IWOTH, S_IXOTH). Any bits set in cmask are turned OFF in file mode. 17

#include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> // fig4.9 #define RWRWRW (S_IRUSR S_IWUSR S_IRGRP S_IWGRP S_IROTH S_IWOTH) int main(void) { umask(0); /* ensure that anyone can read a file */ if (creat("foo", RWRWRW) < 0) err_sys("creat error for foo"); /* disable all the group and other permission bits */ umask(s_irgrp S_IWGRP S_IROTH S_IWOTH); if (creat("bar", RWRWRW) < 0) err_sys("creat error for bar"); exit(0); } 18

Example of execution: $ umask 002 $ a.out $ ls l foo bar -rw------- bar -rw-rw-rw- foo $ umask 002 shell s umask command is used to print the file mode creation mask before the program is run. Changing the file mode of a process doesn t affect the mask of its parent. 19

Function umask The default umask 0002 used for normal user. With this mask default directory permissions are 775 and default file permissions are 664. The default umask for the root user is 0022. It results into default directory permissions are 755 and default file permissions are 644. For directories, Compute the base permission permissions for are 022 (rwxrwxrwx) (root user) 0777 and for files they are 0666 (rw-rw-rw). Directory permission File permission Default Permissions: 777 Subtract umask value: 022 (-) Allowed Permissions: 755 Default Permissions: 666 Subtract umask value: 022 (-) Allowed Permissions: 644 20

umask file access permission bits 0400: user-read 0200: user-write 0100: user-execute 0040: group-read 0020: group-write 0010: group-execute 0004: other-read 0002: other-write 0001: other-execute Common umask values 0: anyone can read 002: prevent others from writing 022:prevent group members and others from writing your files 027: prevent group members from writing your files and others from reading, writing, or executing your files. 21

Functions chmod & fchmod Change file access permissions for an existing file. #include <sys/types.h> #include <sys/stat.h> int chmod(const char *pathname, mode_t mode); int fchmod(int filedes, mode_t mode); Both return: 0 if OK, -1 on error chmod function operates on the specified file fchmod function operates on a file that has already been opened. Callers must be a superuser or effective UID = file UID (file owner). mode = bitwise OR of constants defined in the next table. 22

S_ISUID S_ISGID S_ISVTX S_IRWXU S_IRUSR S_IWUSR S_IXUSR S_IRWXG S_IRGRP S_IWGRP S_IXGRP S_IRWXO S_IROTH S_IWOTH S_IXOTH mode description set-user-id on execution set-group-id on execution saved-text (sticky bit) R/W/E by user (owner) Read by user Write by user Execute by user R/W/E by group Read by group Write by group Execute by group R/W/E by others (world) Read by others Write by others Execute by others 23

S_ISUID When a process runs a regular file that has the S_ISUID bit set, the effective user ID of the process is set to the owner ID of the file. SUID option tells Linux to run the process with the permissions of the owner of the program. It is indicated by s in the owner s execute bit position: rwsr-xr-x e.g. if a file is owned by root and has its SUID bit set, the program runs with root privileges and can therefore read any file on the computer. S_ISGID When a process runs a regular file that has both the S_ISGID bit and the S_IXGRP permission bit set, the effective user ID of the process is set to the group ID of the file. It is indicated by s in the group execute bit position: rwxr-sr-x 24

Sticky bit In modern Linux implementations, the sticky bit is used to protect files from being deleted by those who don t own the file. When this bit is present on a directory, the directory s files can only be deleted or renamed by a user who has write permission for the directory and is: the owner of the file, or the owner of the directory, or root The sticky bit is indicated by a t in the others execute bit, rwxr-xr-t 25

#include <sys/types.h> #include <sys/stat.h> int main(void) { struct stat statbuf; /* turn on group-execute relative to current mode*/ if (stat("foo", &statbuf) < 0) err_sys("stat error for foo"); if (chmod("foo", (statbuf.st_mode S_IXGRP) )< 0) err_sys("chmod error for foo"); /* set absolute mode to "rw-r--r--" */ if (chmod("bar", S_IRUSR S_IWUSR S_IRGRP S_IROTH) < 0) err_sys("chmod error for bar"); exit(0); } 26

Before change $ ls l foo bar -rw------- -rw-rw-rw- foo bar After change $ ls l foo bar -rw-r--r-- -rw-rwxrw- foo bar 27

Functions chown, fchown, lchown #include <sys/types.h> #include <unistd.h> int chown(const char *pathname, uid_t owner, gid_t grp); int fchown(int filedes, uid_t owner, gid_t grp); int lchown(const char *pathname, uid_t owner, gid_t grp); All three return: 0 if OK, -1 on error Only owner of a file can change file ownership. if either of the arguments owner or group is -1, the corresponding ID is left unchanged. 28

File size Field st_size of stat structure gives the size of the file in bytes. This is meaningful only for regular files, symbolic links and directories. Regular files size: st_size = 0 is acceptable. First read will return EOF. Directory size: Is usually a multiple of 16 or 512 with some exceptions. drwxr-xr-x 2 root root 4096 2010-02-27 02:16 bin drwxr-xr-x 133 root root 12288 2010-03-30 23:26 etc drwxrwxrwx 1 root root 20480 2010-03-27 03:26 host drwx------ 2 root root 16384 2010-01-11 02:31 lost+found Symbolic link size: Size is the number of bytes in the file name. lrwxrwxrwx 1 sqasem 10 2010-02-25 17:25 fig1.3 -> file/ls1.c lrwxrwxrwx 1 sqasem 18 2010-02-25 17:25 fig14.9 -> daemons/lockfile.c lrwxrwxrwx 1 sqasem 11 2010-02-25 17:25 fig3.1 -> file/seek.c lrwxrwxrwx 1 sqasem 15 2010-02-25 17:25 fig4.3 -> file/filetype.c 29

File Systems A hierarchical arrangement of directories and files starting in root /. Typical physical organization of disks and file systems is as follows: 30

File Systems i-node: fixed length entries that contain most of file info. Most info in stat structure comes from i-node. Version 7: 64Bytes, 4.4 BSD:128B File type, access permission, file size, data blocks, link count (hard links) see fig above with 2 directories pointing to same i-node. File can be deleted only when link count=0. UNLINK a file does not mean always DELETE a file!!! link count is contained in st_nlink (in stat). LINK_MAX (POSIX.1) defines max number of links. 31

Hard links UNIX filenames are pointers to files Linking to an existing file: shell ln omar ali system call link ( omar, ali ) File omar must already exist unlink simply removes a pointer File destroyed only when last link goes ali omar A link can not be created across file systems $> ln /host/teaching/sections/list76527-10to11.xls toto ln: creating hard link `toto' => `/host/teaching/sections/list76527-10to11.xls': Invalid cross-device link 32

Functions link, unlink link function creates a new dir entry that references an existing file. unlink function removes an existing dir entry and decrements the link count of the file. #include <unistd.h> int link(const char *existingpath, const char *newpath); int unlink(const char *pathname); Both return: 0 if OK, -1 on error 33

Homework for Wednesday April 11 th 2012 1. Write a program that creates two hard links on an existing file. 2. Print the fstat of the file showing that there are three links. 3. Progressively delete the 3 hard links showing that on first and second unlink the file data still exit and that on the last unlink the file was deleted.

Try this Example #include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> // fig 4.16 int main(void) { if (open("tempfile", O_RDWR O_CREAT) < 0) err_sys("open error"); if (unlink("tempfile") < 0) err_sys("unlink error"); printf("file unlinked\n"); sleep(15); printf("done\n"); exit(0); } 35

Functions rename, remove remove = rmdir if pathname is a dir. (ANSI C) rename ANSI C File: both files, newname is removed first if it exists. Needs WX permission for both residing directories Directory: both paths should be directories, newname must be empty, newname could not contain oldname. #include <stdio.h> int remove(const char *pathname); int rename(const char *oldname, const char *newname); Both return: 0 if OK, -1 on error 36

Functions rename, remove Only possible within a file system newname is unlinked newname is linked to the file pointed to by oldname oldname is unlinked e.g. rename (ali, omar) ali omar ali omar 37

Symbolic links Hard links are limited: Cannot link to a different disk Generally not allowed for a directory Symbolic links are more flexible shell ln s omar ali system call symlink ( omar, ali ) omar does not need to exist ali points to the name omar (alias) ali omar 38

Symbolic Links: symlink, readlink #include <unistd.h> /* Create a new symbolic link: */ int symlink(const char *actualpath, const char *sympath); /* read symbolic link itself into buf*/ ssize_t readlink(const char *pathname, char *buf, int bufsize); Both return: 0 if OK, -1 on error symlink: A new dir entry sympath is created that points to actualpath. readlink is an action consisting of open, read, and close of a symbolic link. Returns symlink as non-null terminated string. readlink is needed because open(...) would open the file pointed to by the symbolic link and not the link itself. 39

Functions mkdir and rmdir #include <sys/types.h> #include <sys/stat.h> int mkdir(const char *pathname, mode_t mode); Returns: 0 if OK, -1 on error The function mkdir creates a new empty directory. Access permissions mode modified according to umask. #include <unistd.h> int rmdir(const char *pathname); Returns: 0 if OK, -1 on error An empty dir is deleted. Condition: link count reaches zero, and no one still opens the dir. 40

Directory structure dirent struct is defined in the file <dirent.h>. It is very much implementation dependent. It contains at least the following two members: struct dirent { ino_t d_ino; /* i-node number */ char d_name[name_max+1]; /* null- terminated file name */ }; The DIR structure is an internal structure used by functions that handle directories. Pointer to DIR structure is returned by opendir and used with all other functions: readdir, rewinddir, closedir, telldir, seekdir. opendir makes necessary initializations of DIR struct so that readdir reads 1st entry of directory. 41

Functions opendir, readdir, rewinddir, closedir #include <sys/types.h> #include <dirent.h> DIR *opendir(const char *pathname); Returns: pointer if OK, NULL on error. struct dirent *readdir(dir *dp); Returns: pointer if OK, NULL at end of directory or error. void rewinddir(dir *dp); int closedir(dir *dp); Both return: 0 if OK, -1 on error 42

Functions telldir and seekdir #include <sys/types.h> #include <dirent.h> long telldir(dir *dp); Returns: current location in directory associated with dp, -1 on error. void seekdir(dir *dp, long loc); telldir returns an offset into the directory for later use by seekdir function. The offset returned is greater than or equal to zero. seekdir is used to restore the directory position. No success or position is returned for this call. 43

Functions telldir and seekdir, cont. Example: #include <sys/types.h> #include <dirent.h> DIR *dirp; /* open DIR pointer */ long dirpos; /* directory offset */ dirpos = telldir(dirp); /* get offset in directory */ seekdir(dirpos); /* restore directory position */ 44

Functions scandir and alphasort #include <sys/types.h> #include <dirent.h> int scandir( const char *dirname, struct dirent **namelist, int (*select)(struct dirent *), int (*compar)(const void *, const void *)); int alphasort (const void *d1, const void *d2); 45

Functions scandir and alphasort, cont. scandir() reads the directory dirname and builds an array of pointers to directory entries or -1 for an error. namelist is a pointer to an array of structure pointers. (*select)() is a pointer to a function which is called with a pointer to a directory entry (defined in <sys/types> and should return a non zero value if the directory entry should be included in the array. If this pointer is NULL, then all the directory entries will be included. The last argument is a pointer to a routine which is passed to qsort (see man qsort) -- a built in function which sorts the completed array. If this pointer is NULL, the array is not sorted. The function alphasort() can be supplied in the argument compar if you require the namelist be sorted alphabetically. 46

Functions scandir and alphasort, cont. /* Example - a simple C version of UNIX ls utility */ #include <sys/types.h> #include <sys/dir.h> #include <sys/param.h> #include <stdio.h> #define FALSE 0 #define TRUE!FALSE extern int alphasort(); char pathname[maxpathlen]; 47

Functions scandir and alphasort, cont. main() { int count,i; struct dirent **files; int file_select(); if (getwd(pathname) == NULL ) { printf("error getting path\n"); exit(0); } printf("current Working Directory = %s\n",pathname); count = scandir(pathname, &files, file_select, alphasort); /* If no files found, make a non-selectable menu item */ if (count <= 0) { printf(``no files in this directory\n''); exit(0); } printf(``number of files = %d\n'',count); for (i=1;i<count+1;++i) printf(``%s '',files[i-1]->d_name); printf(``\n''); /* flush buffer */ } 48

Functions scandir and alphasort, cont. int file_select(struct direct *entry) { if ((strcmp(entry->d_name, ``.'') == 0) (strcmp(entry->d_name, ``..'') == 0)) return (FALSE); else return (TRUE); } scandir returns the current directory (.) and the directory above this (..) as well as all files so we need to check for these and return FALSE so that they are not included in our list. Note: scandir and alphasort have definitions in sys/types.h and sys/dir.h. MAXPATHLEN and getwd definitions in sys/param.h 49

Functions scandir and alphasort, cont. We can go further than this and search for specific files: Let's write a modified file_select() that only scans for files with a.c,.o or.h suffix: int file_select(struct direct *entry) { char *ptr; char *rindex(char *s, char c); if ((strcmp(entry->d_name, ``.'')== 0) (strcmp(entry->d_name, ``..'') == 0)) return (FALSE); /* Check for filename extensions */ ptr = rindex(entry->d_name, '.') if ((ptr!= NULL) && ((strcmp(ptr, ``.c'') == 0) (strcmp(ptr, ``.h'') == 0) (strcmp(ptr, ``.o'') == 0) )) return (TRUE); else return(false); } Note: rindex() is a string handling function that returns a pointer to the last occurrence of character c in string s, or a NULL pointer if c does not occur in the string. (index() is similar function but assigns a pointer to 1st occurrence.) 50

Functions chdir, fchdir, and getcwd Functions chdir and fchdir let to specify the current directory either as a pathname or through an open file descriptor. #include <unistd.h> int chdir(const char *pathname); int fchdir(int filedes); Both return: 0 if OK, -1 on error #include <unistd.h> char *getcwd(char *buf, size_t size); Returns: buf if OK, NULL on error The getcwd() function copies the absolute pathname of the current working directory to the array pointed to by buf, which is of length size. 51