CS 307: UNIX PROGRAMMING ENVIRONMENT REVIEW FOR EXAM 1

Transcription

1 CS 307: UNIX PROGRAMMING ENVIRONMENT REVIEW FOR EXAM 1 Prof. Michael J. Reale Fall 2014

2 EXAM 1 OVERVIEW

3 Exam 1 Material Exam 1 will cover the following: REVIEW slides from (1) to (9) (INCLUSIVE) Quizzes 0, 1, and 2 It may also cover anything we ve touched on in the katas.

4 REVIEWS (1) and (2) (1) Course Intro (2) HISTORY OF UNIX

5 UNIX Defined UNIX = operating system Multi-user Multitasking Manages resources, data, users, and user interaction

6 UNIX Components UNIX has: Kernel main control program File system everything is a file Shell user interaction program Utilities and tools makes it a complete OS

7 UNIX Origins UNIX Third system: CTSS Multics UNIX Developed at Bell Labs (AT&T) in 1969 First developed on PDP-7 Invented by Ken Thompson Bonus: built to play his game Space Travel Co-invented by Dennis Ritchie (inventor of C) ORIGINALLY called UNICS and single-user renamed UNIX in 1970 when multi-user

8 C Origins C Language Invented by Dennis Ritchie Started working on it in 1971 Built off interpreted language called B UNIX rewritten in C in 1973, making code: Readable Modifiable Portable ( as opposed to the original assembly code)

9 UNIX Growth UNIX given away to universities (AT&T couldn t sell it public utility) 6 th edition parent of System III System V (AT&T) BSD (Berkeley) Developed by grad student Bill Joy Minix (Tannenbaum)

10 UNIX Woes 1981: Microsoft makes deal with IBM to include MS-DOS in new IBM PCs 1983: Start of UNIX Wars AT&T commercializes System V and closed source Legal battles between System V (AT&T) and BSD (Berkeley) Delays PC port of BSD until 1993 FreeBSD

11 GNU and FSF GNU Project Started in 1983 by Richard Stallman 1985: GNU manifesto and Free Software Foundation GNU = GNU s Not Unix Wanted to make free OS (didn t finish) Started with OS utilities VERY important later for Linux Pushed open source and free (as in free speech ) software

12 Linux Linux kernel Released in 1992 by Linus Torvalds (Finnish grad student) Frustrated with not having free Unix Didn t know about delayed BSD port Open Source Coded from scratch (Unix-like) Linux OS Linux kernel + GNU/FSF utilities = complete OS! Different distributions different utilities/applications bundled

13 System Interaction Graphical User Interface (GUI) Applications on top of OS Command Line Interface Also have Virtual Consoles (CTRL-ATL-F[2-9]) Both can be used locally or remotely

14 Shell Shell Two primary functions: Command interpreter Programming interface Case sensitive!

15 Users Unix is a multi-user environment Each user has: Home directory User ID Group ID Commands: Who am I: id Who are you: id username Who is connected: w / who

16 REVIEW (3) INTRODUCTION TO SHELLS, COMMANDS, MAN PAGES, TEXT EDITORS, And COMPILERS

17 Shell: Definition Shell - program that allows user to: Enter commands and run programs Redirect input and output Program/script in shell itself

18 Shell / Kernel / User Relationship Kernel: Creates shell when you login Connects user to shell Provides services and exposes resources to programs (including the shell) Disposes of shell when user is done

19 Unix Shells Thompson s (original Unix shell) Bourne (sh) Korn (ksh) Bourne Again (bash) C-Shell (csh) TCSH (tcsh)

20 Unix Shell Family Tree

21 More about Shells See available shells cat /etc/shells Can switch to different shell by calling it as program Can write your own shell (regular program)

22 Shell Customization Bourne (sh): /etc/profile then ~/.profile Bourne Again (bash) Login shells On login: /etc/profile then either ~/.bash_profile OR ~/.bash_login OR ~/.profile On logout: ~/.bash_logout Interactive, non-login shells: ~/.bashrc C-shell and descendants All shells: ~/.cshrc On login: ~/.login On logout: ~/.logout

23 Command Syntax Command_name options arguments Options = how command is run May be combined E.g., ls al instead of ls a l May have option arguments E.g., cal B 2 May be long options with double-dash E.g., less --version Arguments = additional information (e.g., filenames)

24 Command History Up arrow access previous commands Type first two letters then UP search command history TAB or CTRL+D Autocomplete

25 More About Commands Group commands with a semicolon ; Example: cd ; ls Default options and arguments Example: cal default arguments of current month and year Example: cd default argument ~ (home) Stop commands with CTRL-C (Interrupt)

26 Common Signals Sequence ^A ^E ^C ^D ^U ^L Action Move cursor to start of line Move cursor to end of line Interrupt. Break out of what you're doing eof. End of transmission, end of file Erase entire input line Clear screen above current line ^ is CTRL key

27 MAN Pages MAN pages = documentation for Unix system and programs man [section] command man -k keyword - search the man pages Quit q Help h

28 Common Text Editors pico simple text editor from Univ. of Washington nano - GNU pico clone vi - original UNIX text editor vim - vi Improved

29 GNU C/C++ Compiler gcc & g++ gcc compiles C code g++ compiles C++ code Source code file extensions must either be.c or.cpp Example: gcc -o name_of_executable source.c OR g++ -o name_of_executable source.cpp

30 REVIEW (4) UNIX FILES

31 What is a file? File Anything that can be a source or destination of data Can be READ and/or WRITTEN to In Unix/Linux, everything is a file Types of files: Ordinary or regular files Directories Pseudo files

32 Ordinary / Regular Files Ordinary / Regular files Have certain size Reside on storage device May be text or binary

33 Directories Directories Keep track of organization/location of other files Organized in tree structure (parents and children)

34 Pseudo Files NOT used to store data do not take up space Used to access a service provided by the kernel Types of pseudo files: Special / device files Physical devices or pseudo-devices In /dev directory Named pipe Proc file

35 Special / Device Files: Physical Devices Most use numbers on end of name (/dev/fd0, /dev/fd1) Hard disks use letters (/dev/hda, /dev/hdb) Numbers for partitions (/dev/hda1, /dev/hda2) IDE and SCSI/SATA /dev/hda, /dev/sda

36 Special / Device Files: Terminals Terminals TTY e.g., /dev/tty1 Bonus: Teletype machines original Unix terminals PTY pseudo-terminals for GUI window OR remote connection (e.g., /dev/ttyp1 OR /dev/pts/1)

37 Special / Device Files: Pseudo-Devices Bit buckets (write to them and data disappears) /dev/null if READ, returns EOF (end of file) /dev/zero if READ, returns ZERO (null terminator) Random number generation /dev/random waits /dev/urandom does not wait

38 Block vs. Character Character devices Device that reads or writes data one character at a time Block devices Device that reads or writes data one block at a time

39 Piping and Named Pipes Piping = passing output of one program as the input of another through a pipe Example: cat /etc/passwd wc l Above example is an Anonymous Pipe Named Pipe Also called FIFO ( first-in first-out ) Created with mkfifo command

40 Proc Files Proc files Allow you to get system and process information Processes: /proc/<process ID> System info example: /proc/cpuinfo All proc files in /proc directory

41 Identifying a File Identifying files ls l file command

42 REVIEW (5) UNIX FILESYSTEM

43 Filesystem Defined File system A disk or portion of a disk structured to store data Unix filesystem Organized like tree Root directory / Base of tree; highest level of the filesystem

44 Current and Parent Directories Every directory has two folders:. (Dot) = current directory (working directory).. (Dot-Dot) = parent directory

45 Home Directory with ~ ~ = your home directory Implemented by the shell ~someotheruser = someotheruser s home directory

46 Pathnames and Filenames Path or Pathname Specific file/directory location Absolute path path starts with root (/) Relative path - starts from your current directory (no / at beginning) Filename or Basename Actual name of a file at the end of a path

47 Common Directory Commands cd = changes current directory pwd = prints current directory ls = lists contents of directory ls l = long format listing mkdir makes a directory mkdir p = makes all in-between directories rmdir = removes a (empty) directory

48 Common File/Directory Commands mv = move and/or rename file or directory cp = copy files and directories cp R = copy recursively rm removes files (and directories with r recursive option) cat = prints out contents of file to screen more/less = read contents of file

49 Filename Restrictions and Conventions Restrictions: Any sequence of ASCII characters up to 255 characters Conventions: Start with alphabetic character Include extension (e.g., *.txt) Avoid spaces use (hyphen), _ (underscore), or. (dot) Use quotes if spaces in filename Avoid special characters (metacharacters) However, can use \ (escape character)

50 Hidden Files Hidden files begin with. (dot) Example:.login Can see hidden files with ls -a

51 procfs Filesystem /proc files = another filesystem called procfs Give information about system, kernel, and processes Super-user can modify values as well

52 REVIEW (6) (Parts 1,2, and 3) FILE GLOBBING, FILE SPACE, PERMISSIONS, and LINKS

53 Wildcards Wildcards Used to specify range of characters Shell interprets pattern and replaces it with appropriate filenames before command is run Globbing When the shell performs the substitution

54 Common Wildcards * (star) = match any sequence of zero or more characters? (question mark) = match any single character [list] = match any character in list OR matches character range (e.g., [a-z]) [^list] = match any character (or range) NOT in the list {s1,s2 } = match one of the specified strings (called Brace Expansion)

55 File Space Overview Three notions of space: Amount of DATA IN THE FILE The NUMBER OF BLOCKS the filesystem allocates for the file The NUMBER OF ALLOCATION UNITS on disk

56 Finding the Size of a File with ls ls ls l how much DATA is actually in the file Add h for human-readable sizes ls s size that each file TAKES UP ON DISK Both show total in disk space at top (NOT recursive!)

57 Finding size with du du Gives you how much space files take up on disk Includes all subdirectories and files in totals -h human-readable version -s gives you the sum total -a also lists each file (not just the directories)

58 Determining Disk Usage and Free Space df disk free-space gives you filesystem information How much space used, available, and use % -h human-readable version

59 File Permissions Every file (including every directory, etc.) has certain permissions associated with it There are three kinds of permissions: READ WRITE EXECUTE

60 Permissions in More Detail READ ( r ) Read from file or read contents of directory (ls) WRITE ( w ) Write to file OR move/copy/delete folder EXECUTE ( x ) Execute file (script/program) OR cd to directory

61 IDs Each user has a userid Each group has a groupid User can be part of one or more groups Primary and supplementary groups When you run a program, it can only do what your userid is permitted to do

62 I AM ROOT root user The master-and-commander user, primary administrator, and the user who can pretty much do whatever he/she wants Also called the super-user Home directory: /root To login as root: su To execute a command as root: sudo

63 id command id shows: userid and name Primary groupid and name Supplementary groupid s and names

64 File ownership All files are owned by: A user A group User has to be part of group (unless admin overrides this)

65 File Permissions by People Three sets of permissions: Owner permissions - for the userid who owns the file Group permissions - for the groupid who owns the file World/Other permissions - for everyone else NOT included by the previous two

66 Showing Permissions ls l will list the permissions

67 Permission String Three sets of permission strings: owner, group, and world/others For each string: r = read permission w = write permission x = execute permission - = permission not granted

68 Setting permissions with chmod chmod Sets permissions using octal number string First digit = owner, second digit = group, third digit = world/others Convert digit to binary shows which permissions toggled on or off

69 Default Permissions and umask By default, system assigns: 666 = non-executable ordinary files 777 = executable ordinary files 777 = directories Then subtracts user mask from the permissions umask command to set user mask Final permission = (system default) (umask)

70 setuid setuid ( set U-I-D ) or suid Special permission that allows other non-owner users to access a file as if they were the owner Usually used by root to allow regular users to run certain system programs Has s in owner execute permission slot (instead of x )

71 Inodes Inode One for every file and directory All inodes stored in inode table Contains size, owner, file permissions, number of hard links to file, type of file, etc. DOES NOT STORE FILENAME!

72 Hard Links (Hard) Link Link that points to an inode Connection between filename and inode Directories store link information Inode deleted ONLY when all hard links deleted Problems: Cannot span filesystems Cannot link to directories

73 Soft (or Symbolic) Links Soft (or Symbolic) Links Also called symlinks Contains pathname of original file (NOT inode) Analogous to Windows shortcut Can be used with directories and across filesystems Removing destination does NOT get rid of link (although it won t work)

74 Making Links with ln ln file newname Default: creates a HARD link to a file and calls the link newname ln s path newlink Creates SOFT LINK to path named newlink

75 REVIEW (7) MOUNT AND UNMOUNT, TOUR OF UNIX FILESYSTEM

76 Mounting Each storage device has its own filesystem Mounting Attaching the root of one filesystem to a directory in main filesystem Allows access to that attached filesystem Bonus: called mounting because of old removable disk packs

77 mount and umount Commands mount mounts a filesystem manually Example: mount /dev/fd0 /media/floppy umount unmounts a filesystem manually Example: umount /media/floppy

78 List All Mounted Systems Call mount by itself cat /etc/fstab /etc/fstab = filesystem table

79 FHS (Filesystem Hierarchy Standard) Tour /bin holds most important system programs /boot files required when booting system /dev device files /etc configuration files

80 FHS (Filesystem Hierarchy Standard) Tour (continued) /home home directories /lib essential shared libraries as well as kernel modules needed for programs in /bin and /sbin /lost+found damaged files that have been recovered by fsck

81 FHS (Filesystem Hierarchy Standard) Tour (continued) /media mount point for removable media /mnt mount point for fixed media not mounted elsewhere /opt third-party applications ( optional software ) /proc Proc files

82 FHS (Filesystem Hierarchy Standard) Tour (continued) /root home directory for root user (superuser) /sbin essential system administration programs run by superuser ( system binaries ) /srv data for services provided by local system (e.g., storage for web, ftp, etc.) /tmp temporary files; anyone can store things here, but contents removed automatically

83 FHS (Filesystem Hierarchy Standard) Tour (continued) /usr secondary filesystem used for static data /var secondary filesystem used for variable data

84 So Many Bins GENERAL-USE PROGRAMS /bin essential programs /usr/bin non-essential programs /usr/local/bin locally installed programs SYSTEM ADMINISTRATION PROGRAMS /sbin essential system admin programs run by su /usr/sbin non-essential system admin programs run by su /usr/local/sbin locally installed system programs THIRD-PARTY APPLICATIONS /opt/xxx static data for application xxx; includes programs /var/opt/xxx variable data for application xxx;

85 Root Filesystem Root filesystem Holds essential data/utilities for Unix to start Includes at minimum: /bin /boot /dev /etc /lib /root /sbin /tmp These are always stored on the boot device Once kernel started, other systems mounted automatically

86 Virtual Filesystem (VFS) Virtual Filesystem (VFS) API that handles details of working with different filesystems in OS (physical, pseudo files, etc.)

87 REVIEW (8) (Parts 1 and 2) ADVANCED SHELL USAGE: VARIABLES

88 Two Shell Families Bourne family (including sh, bash, and ksh) C-Shell family (including csh and tcsh)

89 What shell am I using? echo $0 Shell you are currently using echo $SHELL Default shell

90 Shell: Interactive AND Non-Interactive Interactive shell User submitting commands / getting feedback Login shells Non-interactive shell Shell runs shell script and terminates when done

91 Processes Process a specific instance of a program that is executing Contains information about the running program Can have multiple instances of same program each instance is its own process

92 ps command ps Shows the currently running processes By default, only shows your processes ps a Shows ALL currently running processes On some systems, it might be ps e

93 Parents and Children Example: run vi from your shell The shell process is the parent process (or parent) The vi process is the child process (or child)

94 Variables and Environment Variable thing that holds information Name identifier for the variable Value data stored in the variable Environment table of variables a process has access to

95 Variables in Unix Name Only letters, numbers, and underscore Must begin with letter or underscore Value almost always character string

96 Variables in Unix Can do the following with variables Create them Get value from them Change value Destroy them

97 Variables in Unix Two types of variables in Unix: Environment variables Shell variables

98 Environment and Shell Variables Environment variables Variables part of process s environment Similar to global variables in programming Passed from parent process to child process Shell variables Local to the currently running shell NOT passed down to child processes

99 Listing Variables env lists all environment variables set lists all shell variables

100 Printing the Value of a Variable Use $ to get value and echo command Example: echo $SHELL To be very clear, use curly braces {} Example: echo ${SHELL}

101 General Variable Information Environment variables only propagate DOWN the process tree (values do NOT go up the tree) Some variables have means as both local (shell) variables and global (environment) variables Bourne family vs. C-Shell family on how to handle this

102 C-Shell Family Variables A variable is either local OR global Shell variable local By convention, lowercase set myvar= Hello unset destroy shell variable Environment variable global By convention, UPPERCASE setenv MYVAR Hello unsetenv destroy environment variable Exception: certain special variables (both global and local copies)

103 Bourne Family Variables A variable is either local or local+global Shell variables local MYVAR= Hello new variable = shell variable by default Environment variables local AND global export MYVAR makes shell variable environment variable (local+global) export MYVAR= Hi set and export at same time ALL variables UPPERCASE names by convention unset destroys variable

104 Printing Variables env set Both shell families: prints all environment variables C-Shell: print all shell variables Bourne: print all shell variables AND environment variables (environment variables = both)

105 PATH variable PATH list of directories shell searches for command Bourne environment variable C-Shell dual shell/environment variables (path/path)

106 Controlling Shell Behavior C-Shell use shell variables Bourne use shell options set o (turn on) or set +o (turn off) Commands by themselves display options

107 REVIEW (9) QUOTING/ESCAPING, COMMAND SUBSTITUTION, SEARCH PATH

108 Metacharacters Certain characters have special meaning in the shell Examples: $ ;? * To print characters, must quote/escape them

109 \, Strong Quotes, and Weak Quotes \ (backslash) = escape character Works on anything (single quotes) = strong quotes Escapes/quotes everything (double quotes) weak quotes Quotes everything EXCEPT $ (dollar), ` (backquote), and \ (backslash)

110 Command Substitution ` (backquote) command substitution Example: set myshell=`basename ${SHELL}` NOTE: Must use backquotes on both ends!

111 Search PATH PATH variable shell searches in these directories for commands; stops when command found To add to path (ex: $HOME/bin): Bourne: export PATH= $PATH:$HOME/bin C-Shell: set path = ($path $HOME/bin) Can also add current directory. (Dot) Add on end DO NOT ADD AT ALL AS ROOT USER!

112 NOTE ON QUIZZES

113 Note on Quizzes The quizzes up to this point have covered the material up to and including REVIEW 06 (Part I) I may still ask about other material from those preceding review slides

114 QUIZ 0

115 Quiz 0: Question 1 Question 1: Main Control Program of OS = Kernel The shell does allow the user to submit commands, but ultimately the kernel is the last word on what is executed Many shells, but only one kernel Called kernel even under non-unix systems (OS term)

116 Quiz 0: Question 2 Multics was the system before Unix Some put down UNICS, but technically it was the same system as Unix (before multi-user support was added)

117 Quiz 0: Question 3 Dennis Ritchie inventor of C and co-inventor of Unix Richard Stallman starts GNU project and FSF

118 Quiz 0: Question 5 Two primary functions of a shell: Command interpreter Programming interface shell scripting

119 Quiz 0: Question 6 (BONUS) Space Travel game written by Ken Thompson (that in part prompted the development of Unix)

120 QUIZ 1

121 Quiz 1: Question 1 Apart from redirecting input/output, what are the two primary functions of the shell? Command interpreter Programming interface (shell scripting)

122 Quiz 1: Question 2

123 Quiz 1: Question 3 GENERAL syntax of a command (not a specific command) CommandName Options Arguments (Each separated by whitespace)

124 Quiz 1: Question 4 Documentation for the Unix system and programs: MAN pages

125 Quiz 1: Question 5 How does Unix define a file (in the broadest possible sense)? File = source or destination of data (anything that can be READ and/or WRITTEN to)

126 Quiz 1: Question 6 Two types of ordinary/regular files Text Binary

127 Quiz 1: Question 7 Files that are NOT directories or ordinary files Pseudo files

128 Quiz 1: Question 8 (BONUS) The two bit buckets /dev/null returns EOF /dev/zero returns zero

129 QUIZ 2

130 Quiz 2: Question 1 Unix filesystem organized like a tree; what directory is at the very base of the tree? Root directory /

131 Quiz 2: Question 2 Two special directories created automatically by the filesystem in EVERY directory:. (Dot) CURRENT (working) directory Where you are now.. (Dot-Dot) Parent directory Parent of current working directory

132 Quiz 2: Question 3 Absolute path path starts with root (/) Relative path - starts from your current directory (no / at beginning)

133 Quiz 2: Question 4 Command to print absolute path of your current working directory: pwd

134 Quiz 2: Question 5 Hidden files begin with. (dot) Example:.login

135 Quiz 2: Questions 6-10 Files in directory: a, ab, ac, abc, b, bc, cde, def ls a* a, ab, ac, abc a + (zero or more chars) ls a? ab, ac a + (ONE character) ls a?c abc a + (ONE character) + c ls [a-c]* begins with a through c (a,b, OR c) + (zero or more chars) ls [^a-c]* DOES NOT begin with a through c (a,b, OR c) + (zero or more chars)

136 Quiz 2: Question 11 (BONUS) Globbing = when shell automatically replaces wildcard patterns with matching filenames Example: the previous 5 questions