Part 1: The Unix Operating System (a) About Unix History First roots in the Bell Laboratories, early 60s Kernel rewrite in C by Ritchie / Thompson in the early 70s Source code licenses for Universities from 1974 on Different Unix development trees leading to Dialects University of California, Berkeley: BSD Unix family AT&T/Bell, IBM, Digital etc.: System-V Unix family Properties Preemptive multi tasking Full multiuser-support Uses IP as native network protocol Many typical Internet services integrated (email, FTP etc.) Large number of supported hardware architectures High reliability and maintainability Full support for remote administration Complicated due to large number of commands Most Unixes are already fully 64bit Herbert Martin Dietze <herbert@the-little-red-haired-girl.org> 2
Part 1: The Unix Operating System About Unix Unix Systems System-V: SUN Solaris, IBM AIX, DEC Unix, SCO Unix, IRIX BSD: FreeBSD, NetBSD Unix-Clones: Minix, Linux Present Time Server for the Internet and intranet (e.g. mail, web, ftp) Number Crunching for science (e.g. climate models) High end graphics (e.g. rendering of movie scenes) Future? User desktop due to increasing number of end user applications (e.g. office, graphics) triggered by popularity of Linux Recent Posix standard proposes Real Time abilities Herbert Martin Dietze <herbert@the-little-red-haired-girl.org> 3
Part 1: The Unix Operating System About Unix The Unix File System / (root directory) +-> bin/ (basic programs) +-> lib/ (basic system libraries) +-> sbin/ (basic system programs) +-> dev/ (devices) +-> etc/ (system-wide configuration) +-> home/ (users home directories) +-> herbert/ (user herbert s home) +-> tmp/ (temporary files) +-> usr/ (programs and subsystems) +-> X11/ (X11-GUI subsystem) +-> bin/ (like /usr/bin/) +-> lib/ (...) +-> bin/ (application programs) +-> include/ (C/C++ header files) +-> sbin/ (system programs) +-> local/ (manually installed tree) +-> bin/ (like /usr/bin/) +-> lib/ (...) +-> var/ (variable / writable tree) +-> tmp/ (temporary files) +-> spool/ (printjobs spooling) +-> log/ (log files) These are common to most Unix systems, however this is only a subset of the typically installed directories. Herbert Martin Dietze <herbert@the-little-red-haired-girl.org> 4
Part 1: The Unix Operating System About Unix File Types Unix knows these file types using the abbrevations below: - Regular files that may contan program code or data d Directories that may contain any number of other files l Symbolic links are references to other files p Named Pipes are communication channels for programs c Character devices like e.g. serial or parallel ports b Block devices like e.g. hard disks or floppies Devices All devices (e.g. hard disks or sound card) are represented as files in the /dev directory Operations on devices are therefore file operations Typically devices are either block or character devices Example, play a sound file (cp copies files): $ cp /usr/share/sounds/au/english.au /dev/audio Herbert Martin Dietze <herbert@the-little-red-haired-girl.org> 5
Part 1: The Unix Operating System About Unix Disks and Partitions The hard disks can be found in the /dev directory The first ide hard disk is /dev/hda 1 For partitioned disks, every partition is a separate file, e.g. /dev/hda1, /dev/hda2,... The file system consists of hard disk partitions mounted into the directory tree no drive letters! A mounted partition appears in the tree as a directory During system startup partitions get mounted automatically Removable media (e.g. CDs) must be mounted before usage 1 Naming conventions differ, here we use Linux notation. Herbert Martin Dietze <herbert@the-little-red-haired-girl.org> 6
Part 1: The Unix Operating System (b) The Unix Shell Background Unix predated the GUI by 15 years No graphics, only text Interface to the operating system is a command line User types in a command, and it gets executed Access via a (serial) ASCII terminal, through a network, or through a terminal emulator (e.g. xterm) The Shell is the interpreter, not the terminal Example herbert@paulina: ~/ $ who herbert :0 Dec 29 18:04 (console) herbert pts/1 Dec 29 18:10 (:0.0) herbert@paulina: ~/ $ _ Shell Features Can handle interactive input and shell scripts Shell scripts are text files containing shell commands Shell Syntax provides programming language constructs Most commands are external programs called by the shell Herbert Martin Dietze <herbert@the-little-red-haired-girl.org> 7
Different Shells Different Shells use different syntaxes, commands are always the same but the external The Bourne Shell (/bin/sh) is the standard on commercial Unixes rather limited functionality The Korn Shell (/bin/ksh) is an enhanced Bourne Shell better interactive features (e.g. command history) The C Shell (/bin/csh) uses a C-like command syntax and provides similar enhancements like the Korn Shell The Bourne Again Shell (/bin/bash) is the standard on Linux combines the best of the above (e.g. command history) External Commands / Programs When calling a program the shell creates a new process It then (normally) waits for the process to finish Some External Commands pwd print current working directory date print date and time who who is logged in ps process status echo echo arguments to console ls list files Herbert Martin Dietze <herbert@the-little-red-haired-girl.org> 8
Simple Commands $ pwd /home/herbert $ date Sat Dec 29 19:01:46 CET 2001 $ ls Makefile slides.aux unix.aux auto slides.dvi unix.tex programming.tex slides.tex herbert@paulina: ~/ $ ps PID TTY TIME CMD 584 pts/1 00:00:00 bash 652 pts/1 00:00:00 ps Commands with Arguments herbert@paulina: ~/ $ echo hello world hello world herbert@paulina: /home/hugo/ $ ls -a. programming.aux slides.idx slides.tex.. programming.tex slides.ilg texput.log Makefile slides.aux slides.ind unix.aux auto slides.dvi slides.log unix.tex Herbert Martin Dietze <herbert@the-little-red-haired-girl.org> 9
The Online Manual The standard documentation format is man (manual pages) The man command opens the manual for a given item, e.g. man ls opens the manual for the ls command Reading a Manual Page The pages are displayed in a Pager program The pager allows pagewise moving up- and downwards through a file and searching Traditional Unixes use more, the less command on Linux is more powerful Keyboard Shortcuts Supported by less Page down: PgDown or C-f Page up: PgUp or C-b Move linewards: The cursor keys or j / k Search forwards: / Search backwards:? Pager online help: h, sometimes? Herbert Martin Dietze <herbert@the-little-red-haired-girl.org> 10
A Manual Page for a Fictional Command FOO(1) NAME foo do a bar SYNOPSIS foo [-l] [--long] [-v] [--verbose] infile [-o outfile] [--outfile=outfile] DESCRIPTION This manual page documents the foo command. The foo command bars infile writing the output to standard output. Alternatively, an outfile can be specified to redirect the output to a file. OPTIONS -l, --long Produce a longer output with each entry in its own line. -v, --verbose Verbose mode, produces status messages to standard error while processing. This is useful for debugging purposes. -o outfile Write the output to file outfile rather than to standard output. OUTPUT DECORATION The foo command honors the FOO COLOR environment variable. If set to one of the colors red, yellow, blue or green the output will be shown in it if supported by the terminal. SEE ALSO bar(1), foobar(1) BUGS It does not actually do anything. AUTHORS John H. Doe, Yoyodyne Inc. johndoe@yoyodyne.com Herbert Martin Dietze <herbert@the-little-red-haired-girl.org> 11
Manual Sections Manual documentation is provided for different things, like user and administration commands, APIs and configuration file formats: 1 User commands 2 System API (typically C library functions) 3 Subroutines (functions from other subsystems) 4 Devices 5 Configuration files 6 Games 7 Misc. stuff not matching the other categories 8 Commands for system administration n New features o Old features When opening the page the section can be selected: man 1 ls or on some other systems: man -s 1 ls This is useful if the same name matches more than one man page. Herbert Martin Dietze <herbert@the-little-red-haired-girl.org> 12
Typical Elements of Man Pages On top is the man page name and the section number NAME contains the items names and descriptions SYNOPSIS describes the command s formal syntax DESCRIPTION contains the actual documentation FILES is a list of relevant file names SEE ALSO is a list of other relevant man pages BUGS lists known problems if any Finding the Right Man Page The apropos command lists the man pages matching a keyword: herbert@paulina: ~/ $ apropos copy cp (1) - copy files and directories cpio (1) - copy files to and from archives... herbert@paulina: ~/ $ _ Short Summary The whatis command prints the summary given names: herbert@paulina: ~/ $ whatis cp cp (1) - copy files and directories herbert@paulina: ~/ $ _ Herbert Martin Dietze <herbert@the-little-red-haired-girl.org> 13
Some More Shell Commands These always operate on one or more files: cat list a file s contents cmp compare two files diff compare two files verbosely sort sort a file s lines tail print a file s last lines Examples: herbert@paulina: ~/ $ cat hello hello world herbert@paulina: ~/ $ cmp hello hello~ hello hello~ differ: char 6, line 1 herbert@paulina: ~/ $ diff hello hello~ 1c1 < hello world --- > hello herbert@paulina: ~/ $ sort hello hello world Herbert Martin Dietze <herbert@the-little-red-haired-girl.org> 14
Files Commands These commands are useful for managing files: cp copy files mv move / rename files rm remove files mkdir create directories rmdir remove empty directories Examples: herbert@paulina: ~/ $ ls bar foo foobar herbert@paulina: ~/ $ mv foobar hugo herbert@paulina: ~/ $ rm foo bar herbert@paulina: ~/ $ mkdir yum herbert@paulina: ~/ $ ls -l total 4 -rw-rw-r-- 1 herbert staff 12 Dec 30 18:55 hugo drwxrwxr-x 2 herbert staff 1024 Dec 30 18:58 yum herbert@paulina: ~/ $ mv hugo yum herbert@paulina: ~/ $ ls yum hugo herbert@paulina: ~/ $ rmdir yum rmdir: yum: Directory not empty herbert@paulina: ~/ $ rm yum/hugo herbert@paulina: ~/ $ rmdir yum Herbert Martin Dietze <herbert@the-little-red-haired-girl.org> 15
Shell Variables The shell lets users set and use variables A variable is created when a value is assigned to it: VARIABLE=value The assignment operator = must come immediately after the variable name Variable names are case sensitive For getting a variable s value the $ operator is used: echo $VARIABLE $VARIABLE means: Take the variable VARIABLE and replace the expression by its value Visibility of Variables Variable are normally visible within their processes Forked processes will not see them the variable needs to be exported first Example: $ VAR1=foo $ VAR2=bar $ export VAR2 $ sh $ echo $VAR1 $VAR2 bar Herbert Martin Dietze <herbert@the-little-red-haired-girl.org> 16
Important Variables A lot of system and application programs and the shell are configured by variables To see all defined variables, use set A list of important variables: PATH PS1 PAGER MANPATH A : -separated list of directories to search for executable programs The primary prompt, how the shell greets its users, minimal default is often $ The default pager program, used e.g. by man A : -separated list of directories to search for man pages Shell Initialization Login Session: All configuration files are read Simple Interactive Session: Only some are read Some configuration files for the Bash for login sessions: /etc/profile the system-wide init file, gets read first $HOME/.profile the user s init file, gets read second For non-login sessions: $HOME/.bashrc the user s init file Herbert Martin Dietze <herbert@the-little-red-haired-girl.org> 17
Wildcard expansion The shell interprets wildcard operators and expands them before executing the acutal command Example: ls a* lists files and directories starting with a If no matching file is found, no expansion takes place Wildcard operators: * matches any number (even none!) of any characters? matches exactly one character [abc] matches exactly one character out of a, b, c [a-z] matches exactly one out of all (lowercase) characters Example: $ ls foo bar $ echo * foo bar $ echo f* foo $ echo x* x* Herbert Martin Dietze <herbert@the-little-red-haired-girl.org> 18
Quoting Some characters have a special meaning, like $ or * To make the shell ignore these, they have to be quoted Three kinds of quoting: 1. A backslash quotes a single character, e.g. echo \* 2. Double quotes quote most characters in a string, e.g. echo "****", but not all, e.g. echo "$foobar" 3. Single quotes quote all that double quotes quote plus some more, e.g. echo * $foobar \ Quotes can be nested, e.g. echo say "hello" Example: Brain Damaged File Names $ ls *.doc Timetable Winter 2001.doc $ cat Timetable Winter 2001.doc cat: Timetable: No such file or directory cat: Winter: No such file or directory cat: 2001.doc: No such file or directory $ cat Timetable\ Winter\ 2001.doc 1>/dev/null $ cat Timetable\ Winter\ 2001.doc 1>/dev/null cat: Timetable\ Winter\ 2001.doc: No such file or directory $ cat Timetable Winter 2001.doc 1>/dev/null $ cat "Timetable Winter 2001.doc" 1>/dev/null Herbert Martin Dietze <herbert@the-little-red-haired-girl.org> 19
Standard Streams (Files) Every console program has three open streams: stdin, No 0 the standard input, usually associated with the keyboard stdout, No 1 the standard output, usually associated with the console stderr, No 2 the error output, usually associated with the console The numbers are the File Handles through which the shell itentifies the streams When forking a program the shell temporarily passes them on Redirecting Streams Send stdout to a file: $ echo hello world >hello Send stderr to the null device: $ ls /home/* 2>/dev/null Send stderr to stdout and the result to a file: $ ls /home/* >ls.out 2>&1 How can we write messages to stderr? Herbert Martin Dietze <herbert@the-little-red-haired-girl.org> 20
File Permissions Long output of the ls command: $ ls -l total 48 drwxr-xr-x 2 hugo staff 1024 Nov 9 15:41 CVS -rw-r--r-- 1 hugo staff 42610 Dec 4 19:22 x.pdf -rw-r--r-- 1 hugo staff 3163 Dec 4 19:22 x.tex $ _ The file permissions block contains four groups: 1. (one character) the file type (set by the system): - ordinary file d directory l symbolic link c character device b block device p pipe 2. (three characters) permissions for the file owner: - flag not set r read permission w write permission x execute (if directory: enter directory) permission 3. (three characters) permissions for the file s group 4. (three characters) permissions for the rest of the world Herbert Martin Dietze <herbert@the-little-red-haired-girl.org> 21
Setting File Permissions The chmod command controls file permissions $ chmod og-rwx x.pdf $ ls -l x.pdf -rw------- 1 hugo staff 42610 Dec 4 19:22 x.pdf $ chmod +x x.pdf $ ls -l x.pdf -rwx--x--x 1 hugo staff 42610 Dec 4 19:22 x.pdf $ _ Setting File Group Members of more than one group can explicitly set the file group with chgrp: $ ls -l x.pdf -rwx--x--x 1 hugo staff 42610 Dec 4 19:22 x.pdf $ chgrp audio x.pdf $ ls -l x.pdf -rwx--x--x 1 hugo audio 42610 Dec 4 19:22 x.pdf $ _ Herbert Martin Dietze <herbert@the-little-red-haired-girl.org> 22