LAB 8 (Aug 4/5) Unix Utilities

Aug 4/5 Due: Aug 11 in class Name: CSE number: LAB 8 (Aug 4/5) Unix Utilities The purpose of this lab exercise is for you to get some hands-on experience on using some fundamental Unix utilities (commands). After this lab, you are expected to be able to accomplish lots tasks using command line utilities, without resorting to your GUI based utilities such as JEditors or File Manager. In some systems such GUI tools are not available and thus using command line utilities is your only option. We covered in class the following basic utilities/commands: man, pwd, ls, cd, mkdi, rmdir, cat, more, head, tail, cp, mv, rm, wc, chmod, grep/egrep. We will also cover uniq, sort, cut, find and tr etc this week. You are also expected to explore some advanced utilities such as tr, awk, sed. You can get the details of each utility by 1) going through the class slides. 2) going through the provided Guided Lab Tour for CSE1020. 3) using utility man. E.g., man sort Go through the following 95 (small) practices. Write down your answers besides the questions. Hand in this sheet. Note: Each solution should only involve one entry of utility (e.g., mv file1 file2) or a pipeline of utilities (e.g., cat file1 sort wc). 0. Login to your home directory, and change to Bourne shell by issuing sh.the prompt should change from % to $ 1. There is a file called xxx in directory /cs/dept/course/2014-15/s/2031/ Copy this file to your current directory with the same name, using one entry of utility (command). 2. Check that the file is copied here. your_command xxx 3. There are two files named xfile2 and xfile3 in directory /cs/dept/course/2014-15/s/2031/ copy these two files to your home directory using one entry of utility, and using absolute path. Assume these two files are the only files whose names begin with xfile. Hint: so you can use xfile* or File? to match these two files. (If you don t understand * and?, look for page 10 of the Guided Lab Tour for CSE1020). Note, don t confuse that with * and? that are used in (extended) regular expression.. 4. Verify that the 2 files are copied successfully to the current directory. your_command xfile2 xfile3 5. Rename file xxx to xfile1 6. Verify that the renaming is successful. your_command xfile1 xfile2 xfile3 7. Create a sub-directory named lab8a under the current (home) directory 7. mkdir lab8a 1. cp /cs/dept/course/2014-15/s/2031/xxx. 2. ls xxx 3. cp /cs/dept/course/2014-15/s/2031/xfile2 cs/dept/ /2031/xFile3. or cp /cs/dept/course/2014-15/s/2031/xfile?. or cp /cs/dept/course/2014-15/s/2031/xfile*. 4. ls xfile2 xfile3 or ls xfile* or ls xfile? 5. mv xxx xfile1 6. ls xfile1 xfile2 xfile3 or ls xfile* or ls xfile? 1

8. Verify the existence of directory lab8a by listing the directory itself. your_command drwx------ 2 your_name ugrad 4096 Jul 31 17:00 lab8a (Note, if the creation was unsuccessful, then you will get ls: cannot access lab8working: No such file or directory ) 9. Move all the 3 files (together) into lab8a (using one entry of utility) 10. Verify that the move is successful by listing files in directory lab8a (without going into that directory) total 12 -rwx------ 1 yourname yourgroup 140 July 31 16:58 xfile1 -rwx------ 1 yourname yourgroup 140 July 31 16:58 xfile2 -rwx------ 1 yourname yourgroup 86 July 31 16:58 xfile3 11. Copy directory lab8a to a new directory named lab8b (using one). 12. Verify that lab8b was created successfully total 8 drwx------ 2 yourname yourgroup 4096 July 31 17:00 lab8a drwx------ 2 yourname yourgroup 4096 July 31 17:02 lab8b 13. Verify that the two directory are identical lab8a: xfile1 xfile2 xfile3 8. ls -ld lab8a or ls -ld./lab8a 9. mv xfile1 xfile2 xfile3 lab8a or mv xfile? lab8a or mv xfile* lab8a 10. ls -l lab8a 11. cp r lab8a lab8b 12. ls ld lab8a lab8b or ls ld lab8? or ls -ld lab8* 13. ls lab8a lab8b or ls lab8? or ls lab8* lab8b: xfile1 xfile2 xfile3 14. Remove the whole directory lab8a using rmdir. What happened? 15. Remove the whole directory lab8a using a more effective utility. 16. Verify that directory lab8a has been removed ls: cannot access lab8a: No such file or directory 17. Move file1 in lab8b to current (parent) directory, using relative directory. 18. Verify that the above move was successful. Lets do this by searching for the files find. name xfile*./xfile1./lab8b/xfile2./lab8b/xfile3 or 14. rmdir lab8a rmdir: failed to remove `lab8a/': Directory not empty 15. rm-r lab8a 16. ls lab8a or ls ld lab8 or ls l lab8 18. find. name xfile* find. name xfile? 17. mv lab8b/xfile1. 2

19. Change the name of directory lab8b to lab8working 20. Verify that the renaming was successful. ls: cannot access lab8b: No such file or directory 21. Navigate to directory lab8working 22. Verify that you are in lab8working /cs/home/your_account/lab8working 23. Move xfile1 (in the parent directory) to the current directory using relative path. 24. Verify that the move was successful by listing all the files currently in lab8working. total 12 -rwx------ 1 yourname yourgroup 140 Jul 31 16:58 xfile1 -rwx------ 1 yourname yourgroup 140 Jul 31 16:58 xfile2 -rwx------ 1 yourname yourgroup 86 Jul 31 16:58 xfile3 25. Display on stdout the contents of the three files with one command (Try more xfile1 xfile2 xfile3 or more xfile?) 26. Check how many lines xfile1 contains. You should get 5. 19. mv lab8b lab8working 20. ls lab8b or ls -l lab8b or ls ld lab8b 22. pwd 21. cd lab8working or cd./lab8working 24. ls -l 23. mv../xfile1. 25. more xfile1 xfile2 xfile3 or more xfile? or more xfile* 26. wc -l xfile1 or more xfile1 wc l or cat xfile1 wc -l 27. Display the first line of xfile1 27. head -1 xfile1 28. Display the first two lines of xfile2 29. Display the last 3 lines of xfile2 30. Confirm that xfile1 and xfile2 are identical, using an utility, which should return silently (Hint: cmp or diff). 28. head -2 xfile2 29. tail -3 xfile2 30. cmp xfile1 xfile2 31. diff xfile1 xfile2 31. Confirm that xfile1 and xfile2 are identical, using another utility, which return silently (diff or cmp) 32. Show that xfile2 and xfile3 are not identical, using one utility, which will not be silent this time. Try to understand the message but don t spend too much time on it. 33. Show that xfile2 and xfile3 are not identical, using another utility, which will not be silent this time. Try to understand the message but don t spend too much time on it. Note: these two utilities are used by some professors to judge your labtest output with their solutions: gcc yourcode.c; a.out > youroutputfile; cmp youroutputfile professorsoutputfile; A student gets 0 if the last command does not return silently. 32. cmp xfile2 xfile3 33. diff xfile2 xfile3 34. Concatenate the contents of the three files into a new file xfile123, in the order of xfile1,xfile2 and xfile3. Hint: use cat and redirection >. After that, show on stdout the content of xfile123. 34. cat xfile1 xfile2 xfile3 > xfile3 cat xfile3 or more xfile3 3

more xfile123 35. Sort file xfile123 so the identical lines are adjacent now 36. Show on the stdout the count of identical lines (hint: use pipe involving sort and uniq -c). 3 3 3 2 2 37. Remove the identical lines and save the result into a file xfile123compact, without the counts (as show below). 38. Show on the stdout the content of xfile123compact 35. sort xfile123 or cat xfile123 sort 36. sort xfile123 uniq c or cat xfile123 sort uniq -c 37. sort xfile123 uniq > xfile123compact or cat xfile123 sort uniq > xfile123compact 38. cat xfile123compact or more xfile123compact 4

39. Issue chmod 775 xfile1, see the resulting permission mode of the file. Can you understand what we are doing here? 40. Issue chmod 777 xfile1, see the resulting permission mode of the file. Can you understand what we are doing here? 39. -rwxrwxr-x 40. -rwxrwxrwx 41. Change the permission of xfile123compact using octal numbers so that the permission becomes -rwxr--r-- 1 yourname yourgroup 140 Jul 31 17:23 xfile123compact 41. chmod 744 xfile123compact 42. Change the permission of file123compact by giving group an execute permission. Then show the result: -rwxr-xr-- 1 yourname yourgroup 140 Jul 31 17:23 xfile123compact 42.chmod g+x xfile123compact 43. change the permission of xfile123compact by giving group a write permission, and remove write permission of the owner of the file. You should do chmod only once. Then show the result: -r-xrwxr-- 1 yourname yourgroup 140 Jul 31 17:23 xfile123compact 44. Change the permission of xfile123compact by removing write permission from the group, and giving execution permission to the others. You should do chmod only once. Then show the result: r-xr-xr-x 1 yourname yourgroup 140 Jul 31 17:23 xfile123compact 45. Change the permission of xfile123compact by adding the write permission of user, group and other. You should not do chmod only once. Then show the result: 43. chmod g+w,u-w xfile123compact 44. chmod g-w,o+x xfile123compact rwxrwxrwx 1 yourname yourgroup 140 Jul 31 17:23 xfile123compact 45. chmod a+w xfile123compact or chmod ugo+w xfile123compact 46. Modify xfile1 by adding a new line at the end of the file. This can be done by echo this is a xxx new line >> xfile1 cat >> xfile1 and then enter a new line followed by Ctrl-D. or 47. List the files in the current directory, sorted by the modification time. xfile1 should be the first file in the list and other files are also sorted according to the modification time. 48. List the files, sorted by the modification time, in reverse order. 49. List the files, sorted by the size of the files. xfile123 should be the first file in the list and other files are also sorted according to the sizes. 50. List the files, sorted by the size of the files, in reverse order. 51. Try to get the type of the file xfile123compact (Hint: use file utility) xfile123compact: ASCII text 52. Try to get the type of one of your c source files. hello.c: ASCII C program text 52. file hello.c 53. Try to get the type of one of your a.out files. 46. echo this is a xxx new line >> xfile1 47. ls l t or ls -lt 48. ls l t -r or ls -ltr 49. ls l S or ls -ls 50. ls l S -r or ls -lsr 51. file xfile123compact a.out: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.18, not stripped 53. file a.out 5

54. Check that the link count of xfile123compact is 1, which means that entry xfile123compact is the only link to the physical file, which resides in the disk. -rwxrwxrwx 1 yourname yourgroup Jul 31 17:23 xfile123compact link count 54. ls l xfile123compact 55. Generate a hard link of xfile123compact called hardlink123 (Hint: use ln utility) Now both xfile123compact and hardlink123 'point' to the same physical file, which resides somewhere in the disk. 56. Verify that the link counts of both the two files are 2 now. This means that two pointers point to the same physical file. -rwxrw-r-- 2 yourname yourgroup 140 Jul 31 17:50 hardlink123 -rw------- 1 yourname yourgroup 140 Jul 31 17:48 xfile1 -rw------- 1 yourname yourgroup 366 Jul 31 17:10 xfile123 -rwxrwxrwx 2 yourname yourgroup 140 Jul 31 17:20 xfile123compact -rw------- 1 yourname yourgroup 140 Jul 31 16:58 xfile2 -rw------- 1 yourname yourgroup 86 Jul 31 16:58 xfile3 57. Remove file hardlink123. 55. ln xfile123compact hardlink123 56. ls -l 57. rm hardlink123 58. Verify that hardlink123 is gone, but xfile123compact is still there, and its link count becomes 1. -rw------- 1 yourname yourgroup 140 Jul 31 17:48 xfile1 -rw------- 1 yourname yourgroup 366 Jul 31 17:10 xfile123 58. ls -l -rwxrwxrwx 1 yourname yourgroup 140 Jul 31 17:20 xfile123compact -rw------- 1 yourname yourgroup 140 Jul 31 16:58 xfile2 -rw------- 1 yourname yourgroup 86 Jul 31 16:58 xfile3 59. Since the newly added line of xfile1 is the only line that contains xxx in the file, we can remove the (last) line from xfile1 by issuing two utilities:, on a single line: egrep v xxx xfile1 > tmp mv tmp xfile1 Do you understand what I am doing here? Briefly explain here: 60. Sort xfile123compact according to the numerical value of the 3rd field 61. Sort xfile123compact according to the numerical value of the 3rd field, in reverse order 59. Reverse egrep to filter out the line containing xxx, and write the result to a tmp file. Finally rename tmp to xfile1 (overwrite xfile1) 60. sort n k3 xfile123compact or cat xfile123compact sort n k3 or sort n +2-3 xfile123compact or cat xfile123compact sort n +2 3 6

62. In exercise 59 and 60, John Smith and Lisa Sue, who have the same 3 rd field value, did not get further sorted according to the 4 th field. Sort xfile123compact according to the numerical value of the 3rd field, and then based on this, further sort according to the 4th field. Your just need one command, don t sort twice. 63. Sort xfile123compact according to the year (the last field) 64. Sort xfile123compact according to the year (the last field), in reverse order. 61. sort n k3 -r xfile123compact or cat xfile123compact sort n k3 -r 62. sort n k3 k4 xfile123compact or cat xfile123compact sort n k3 k4 63. sort n k6 xfile123compact or cat xfile123compact sort n k6 64. sort n k6 r xfile123compact or cat xfile123compact sort n k6 -r 65. Use cut utility to extract the first names in the file xfile123compact. That is, display the 1 st column/field of the file. Use man or look into the slides for the cut utility. Hint: cut fx -d" " file123compact, where x is an integer. This command means treat " " as column/field delimiter, and display column/field x. John John Larry Lisa Tony 66. Use cut utility to extract the month and year in the file xfile123compact. Hint: cut fx,y -d" " or cut fx-y -d" " Jan 1966 Apr 1956 Dec 1946 Jul 1980 Mar 1950 65. cut d f 1 xfile123compact or cat xfile123compact cut d f 1 66. cut d f 5,6 xfile123compact or cat xfile123compact cut d f 5,6 7

67. [Bonus]. Extract the first name, month and year in the file xfile123compact, and output the results sorted by month. John Jan 1966 Tony Mar 1950 John Apr 1956 Lisa Jul 1980 Larry Dec 1946 68. Suppose that we want to change all the lower case letters in xfile123compact to upper case letters. This can be accomplished by a utility called tr. Try tr [a-z] [A-Z] < xfile123compact and observe the following output. JOHN DUNCAN 1111 20 JAN 1966 JOHN SMITH 1222 26 APR 1956 LARRY JONES 1223 20 DEC 1946 LISA SUE 1222 15 JUL 1980 TONY JONES 1012 20 MAR 1950 69. Use the same utility to change every digit in the file into character +. John Duncan ++++ ++ Jan ++++ John Smith ++++ ++ Apr ++++ Larry Jones ++++ ++ Dec ++++ Lisa Sue ++++ ++ Jul ++++ Tony Jones ++++ ++ Mar ++++ 67. cut d f 1,5,6 xfile123compact sort M k2 or cat xfile123compact cut d f 1,5,6 sort M k2 68. tr [a-z] [A-Z] < xfile123compact 69. tr [0-9] + < xfile123compact or tr [:digit:] + < xfile123compact The following exercises involve filtering files. Use egrep or grep E (which guarantee to accept the extended regular expression) and make sure you are in sh. 70. Display the records of people in file xfile123compact who were born in 1950s 70. egrep 195.$ xfile123compact 71. Pipe egrep or grep E and another utility to get the number of peoples in xfile123compact who were born in 1950s. You should get 2. 71. egrep 195.$ xfile123compact wc -l 72. The EECS department maintains the records of all the students, stuff and faculty members in a file /etc/passwd, one person per line. Issue a utility to find out how many students are in the list. You should get about 2313. 72. wc l /etc/passwd or cat /etc/passwd wc l 73. Retrieve your information in the file /etc/passwd, using your cse login account. 74. The (modified) class list of our class can be found at /cs/dept/course/2014-15/s/2031/2031cl. Each line of the file contains one student information. Get the number of students currently enrolled in the course. 73. egrep huiwang /etc/passwd or cat /etc/passwd egrep huiwang 8

74. wc -l /cs/dept/course/2014-15/s/2031/2031cl or cat /cs/ /2031cl wc -l You should get 44. You can use the file directly (by giving the path), or, copy the file to your current directory. If you are wondering how I modified the file, here is the single command: tr [0-9] * < original2031cl > 2031cl 75. Retrieve your record from the class list. 75. egrep yourname 2031cl or cat 2031cl egrep yourname 76. Look for the student(s) whose name contain Wang. 76. egrep Wang 2031cl or cat 2031cl egrep Wang 77. Pipe egrep and another utility to get the number of students whose name contain Wang. You should get 1 78. Look for the student(s) whose name contains Wang or Wong. 77. cat 2031cl egrep Wang wc -l 79. Pipe egrep and another utility to get the number of students whose name contain Wang or Wong. You should get 2. 78. egrep W[ao]ng 2031cl or cat 2031cl egrep W[ao]ng 79. egrep W[ao]ng 2031cl wc l or cat 2031cl egrep W[ao]ng wc -l There is a file lyrics in directory /cs/dept/course/2014-15/s/2031. Use the file directly (by giving the path), or, copy the file to your current directory, and then use grep E or egrep utility to find the lines in lyrics that: 80. does not contain the Well you know it's your bedtime, Oh you sleepy young 1970 heads dream of wonderful things, 81. contains digits #So turn off the light, 1980 Oh you sleepy young 1970 heads dream of wonderful things, sea 1980 I got there by chance. 82. contains 1980 #So turn off the light, 1980 sea 1980 I got there by chance. 83. end with 1980 #So turn off the light, 1980 84. contains sea 80. egrep v the lyrics or cat lyrics egrep v the 81. egrep [0-9] lyrics or cat lyrics egrep [0-9] 82. egrep 1980 lyrics or cat lyrics egrep 1980 83. egrep 1980$ lyrics or cat lyrics egrep 1980$ Beautiful mermaids will swim through the sea, sea 1980 I got there by chance. 85. begins with sea sea 1980 I got there by chance. 86. contains one (any) character followed by nd 84. egrep sea lyrics or cat lyrics egrep sea 85. egrep ^sea lyrics or cat lyrics egrep ^sea Say all your prayers and then, Oh you sleepy young 1970 heads dream of wonderful things, 86. egrep.nd lyrics or cat lyrics egrep.nd 87. contains one (any) character followed by nd, but as a whole world only (so wonderful does not match) Say all your prayers and then, 87. egrep w.nd lyrics or cat lyrics egrep w.nd 9

88. begins with one (any) character followed by nd 89. contains letter A or B or C or D Beautiful mermaids will swim through the sea, 90. begins with a capital letter Well you know it's your bedtime, Say all your prayers and then, Oh you sleepy young 1970 heads dream of wonderful things, Beautiful mermaids will swim through the sea, 91. begins with a non capital letter (lower case letter or any other characters) #So turn off the light, 1980 sea 1980 I got there by chance. 92. ends with a and one other character. Beautiful mermaids will swim through the sea, 93. contains a character that is either a or b or c, followed by nd Say all your prayers and then, 94. contains a character that is not a nor b nor c, followed by nd Oh you sleepy young 1970 heads dream of wonderful things, 95. Go back to your home directory, and issue the utility find. -name xfile* What do you get? 96. Issue Submission find. -name xfile* -exec mv {} {}.2015 \; 88. egrep ^.nd lyrics or cat lyrics egrep ^.nd What we intend to do here? List directory lab8working and examine what happens to the files in lab8working? Note: all egrep can be replaced with grep -E 89. egrep [ABCD] lyrics or cat lyrics egrep [ABCD] or egrep [A-D] lyrics or cat lyrics egrep [A-D] Print out this pdf, write your solutions on this sheet. Hand in the sheet in next class. 90. egrep ^[A-Z] lyrics or cat lyrics egrep ^[A-Z] 91. egrep ^[^A-Z] lyrics or cat lyrics egrep ^[^A-Z] 92. egrep a.$ lyrics or cat lyrics egrep a.$ 93. egrep [abc]nd lyrics or cat lyrics egrep [abc]nd or egrep [a-c]nd lyrics or cat lyrics egrep [a-c]nd 94. egrep [^abc]nd lyrics or cat lyrics egrep [^abc]nd or egrep [^a-c]nd lyrics or cat lyrics egrep [^a-c]nd Or, write the question numbers and your answers in a file, and submit the file submit lab8 your_file_name 95. find. -name "xfile*"./lab8working/xfile123./lab8working/xfile2./lab8working/xfile123compact./lab8working/xfile1./lab8working/xfile3 96. Find all the files whose name begins with xfile, and change name of them. Each new name now has a.2015 suffix. End of lab8 10