Introduction to Command Line Linux

Transcription

1 Eric Nodwell Copyright 2003 Eric Nodwell This document can be freely redistributed according to the terms of the GNU General Public License. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover texts, and with no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License". This is an introductory guide for people learning to use a system. It focuses on command-line usage. This is not because everyone will be doing everything at the command line. On the contrary, there are many useful GUI tools and programs in, and some people will be content to stay with these. However, in contrast to GUI programs, which can be learned largely by "click and try", the command-line requires some deliberate learning, and reading documentation. The benefit of this investment is that you will learn the tools and tricks of the command line which are in many cases faster, more powerful, and more flexible than any GUI program. There are many other introductions to on the web; this one is customized for users in UBC Physics and Astronomy. Since I don t want to cover all the possible cases and variations, I ll also assume that you re sitting at an X-terminal of some sort - in other words, a fancy GUI with lots of windows and things. I m going to assume that you ve figured out how to open a terminal or console window (look for something called Konsole or Gnome Terminal). I m going to assume that your shell is the bash shell. If you don t know what that is, don t worry - it s the default shell. You may also want to read Matt Choptuik s introduction to Unix, which you can find here [ Table of Contents The Basics : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 2 Eleven essential commands : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 2 Getting help for a specific command : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 4 File names with spaces : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 4 Copying and pasting : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 4 Recalling previous commands : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5 Be lazy: use TAB to minimize typing : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5 File permissions : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5 Useful Tools : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 6 The network is the computer: ssh and scp : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 6 Finding and searching files : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 6 Creating and extracting file archives (zip, gzip and tar) : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 7 Emacs - a powerful editor : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 8 Running long calculations : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 8 Getting the prompt back after starting a job : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 8 Keeping a job running after you log out : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 8 Suspending a long-running job and restarting it later : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 9 Running a series of jobs : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 9 More advanced scripts : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 10 When you re not in the department : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 10 Accessing data from any Windows computer : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 10 1

2 without installation - at home or travelling : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 10 More Quick Tips : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 11 Scroll with the keyboard : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 11 Copy with xclip : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 11 Lots of keyboard shortcuts : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 11 ssh and scp without typing a password each time : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 14 Send a quick message without starting any mail program : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 14 Where to go to learn more : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 14 The Basics Eleven essential commands These are the commands which you have to know in order to be basically functional with a shell (command line). 2

3 ls List directory contents. ls -l long listing with dates and permissions cd ls -a list all, including hidden files (hidden files start with a period) Change directory. cd - change to previous directory cd ~ change to home directory pwd cp cd.. change to parent directory Copy file(s). mv rm Print current (working) directory. cp -r somedir somedest Copy recursively (directory and all contents) Move (or rename) a file (or directory) Delete (remove) a file. rm -r somedir Delete recursively (directory and all contents) rm -ri somedir Delete recursively, but prompt before 3 each removal rm -rf somedir Delete recurs-

4 Getting help for a specific command To get detailed help on a particular command or program, you can try the following. Man pages Info pages KDE or Gnome Help /usr/share/doc If you want help on a specific command (ls for example), type man ls. You move around in man as you do with less: SPACEBAR is forward a page, B is back a page, Q is quit. Info pages are similar to man pages, but contain links and other complex structures. There are many key-strokes for navigating info; if you call info without any arguments, then hit H, you ll get some help on using info. This Applies only if it s KDE or Gnome program. In either case, you can launch the help browser from the Start menu on the Panel, or choose Help from the program menu, or press F1. Every package must install at least some minimal help into a subdirectory in /usr/share/doc. Sometimes it s in html format (read with a browser), sometimes it s plain text (read with less), sometimes it s compressed text (has an ending like.gz, read with zless). The home page of the project Every command or program belongs to some project (or company). Usually they have a home page which provides copious, up-to-date and indexed information (well at least you can hope that they do.) Can t find the project home page? Try [ Tip You can read man and info pages with Konqueror (the KDE web browser) or Galeon, which is nice because then they are nicely formatted and have links and such like. For example, to read the man page on the tar command, enter man:/tar in the address bar; to read the info page on tar enter info:/tar. To get the list of info topics, just enter info:/. File names with spaces Spaces on the command line separate arguments. Therefore file names containing spaces require special treatment to avoid being interpreted as two separate files. There are a couple of methods. You can use quotes: cp "file name with spaces" file_name_without_spaces You can also use the backslash, which indicates that any special significance is to be removed from the following character. Thus a space, instead of being the delimiter for arguments on a command line, is to be interpreted as a regular character. cp file\ name\ with\ spaces file_name_without_spaces Copying and pasting Copying and pasting in is really easy (yes, easier than Windows). Just mark the text you want to copy with the mouse, holding down the left mouse button in the usual way. Then click the middle mouse button (on a wheel mouse the middle button is the wheel) where you want to insert the marked text. Voila! You didn t even need to touch the keyboard! (CTRL-C and CTRL-V also work like you expect them to most of the time.) For example, suppose you just typed a command, and you want to type a another similar command. If you can still see your original command in the console, just mark it with the mouse, then click the middle mouse button anywhere 4

5 in the terminal window to insert the marked command at the prompt. You can now edit this command with the left and right arrow keys before hitting enter to execute it. There s another way to recover and edit previous commands, which I ll cover in the next section. Tip You can double-click to mark a whole work, or triple-click to mark a whole phrase. Remember - you haven t only marked it, you ve put it in the clipboard. Recalling previous commands You can scroll though your previous commands (command history) by using the UP and DOWN arrow keys. A more advanced trick is to use CTRL-R, which is particularly useful when you want to recall a command you used some time ago. Hit CTRL-R and then start typing some fragment of the command you re searching for. The prompt will automatically update to show you the most recent match to the fragment you ve typed so far. If you want the matching command previous to the one shown, hit CTRL-R again. Once you ve got the previous command you re looking for, hit ENTER to run it again, or start editing it. Be lazy: use TAB to minimize typing Some commands and especially some file names can be pretty long. Don t want to type it all? Type the first few letters, then hit TAB. The shell will fill in the rest, as far as there is a unique completion (either command or filename). If there is not a unique completion to the end, you might have to type another character or two and then hit TAB again. File permissions Since is a multi-user system, file permissions are important. In, there are separate read, write and execute permissions for the owner of the file ("user"), the "group", and everyone else ("other"). You can find out what permissions a file has with ls -l. Here s an example: -rw-rw-r-- 1 nodwell mbelab Jul 9 13:10 kate.png The series of 9 characters at the beginning of the line spcifies the permissions in order of owner, group, other. In this case, there are read and write permissions for owner and group, and only read permissions for other. You can also see that the owner is "nodwell", and the group is "mbelab". As an example of changing permissions, let s assume I have a file called foo which I want others in the "mbelab" group to be able to read and write. To do this I change the group to mbelab and set the group read and write permissions: chgrp mbelab foo chmod g+rw foo I should also make sure that the directory that this is in, and all the higher (lower?) directories are actually readable by the group. For example: chmod g+rx /home/nodwell/shared_stuff chmod g+rx /home/nodwell Notice that I also set the executable bit for the directories, which allows someone in the group to actually switch into that directory. (Not very logical I admit): 5

6 To change the group and permissions on a directory and everything in it, including subdirectories and their contents, use the -R (recursive) flag: chgrp -R mbelab /home/nodwell/shared_stuff chmod -R g+rw /home/nodwell/shared_stuff Now suppose that we want all files which are created in /home/nodwell/shared_stuff to automatically have the group mbelab. Assuming you ve already changed the group of this directory to mbelab, you can do this by setting the group sticky bit: chmod g+s /home/nodwell/shared_stuff Useful Tools The network is the computer: ssh and scp If you re using, you re rarely working on just one machine. To log in on another machine, use ssh, like this ssh yourid@physics.ubc.ca So now you have a terminal window on the remote machine. You can only run command-line stuff, right? Wrong! Try typing gedit or kedit or netscape. (I m assuming one of these is actually available on the remote machine.) You might notice that this blocks your terminal window until you close the program again. We ll find a solution to that a little later. You also need to know how to move files around between computers. That s where scp comes in. It s just like cp, but not limited to the local machine. scp somefile yourid@physics.ubc.ca:somedirectory Like cp, scp understands the -r flag, for recursive copy, which is really useful for copying whole directories. scp -r somedir nodwell@physics.ubc.ca:. (What s that dot at the end? on the remote machine.) It s shorthand for "the current directory", which by default will be your home directory Finding and searching files The fastest way to find files is with the locate command. locate is uses a database which is updated every night. This makes it very fast, but it will not find files which have been created the same day. The easiest way to use locate is just to specify a fragment of the file name, for example: locate foo For trimming down the results of your search, grep is invaluable. For example, to find all files which contain both report and eric in the complete file name (including path) we can use: locate report grep eric If you know anything about "regular expressions", then you can do a more specific search. A good intro (in fact probably as much information as you ll ever need) to regular expressions can be found at A Tao of Regular Expressions [ 6

7 If you want to specify more than the file name, for example the date modified or the file size, then the find command is the one to use. find actually searches the disk, so it can be a little bit slower. Here is an example of searching for files which are located in the directory /home/eric/misc and all subdirectories, which start with report, which have been modified in the last 2 days, and which are at least 10kb in size: find /home/eric/misc -name report* -mtime -2 -size +10k If you want to find files by their contents, this can be done by combining a number of tools together. (Combining several simple tools to do one complicated job is the Unix approach, and it is very powerful once learned.) We will use find to construct a list of files, and grep to look for lines containing a certain word. We will combine them with a pipe ( ) and the command xargs which takes the output from one command and uses it as an argument for calling a second command. This sounds complicated, but the example is not too bad: find -type f xargs grep -i report Here we are looking for files in the current directory (and subdirectories) which contain the word "report". The -type f flag ensures that we only get a list of regular files (and not directories and links for example) to pass to grep. The -i flag means ignore case, so we are looking for "Report" as well as "report". If you might have filenames with spaces or other special characters, this won t work quite as expected. In this case we must use: find -type f -print0 xargs -0 grep -i report There is a really good tutorial on finding files in the info pages. If you are using the Konqueror web browser or the Galeon web browser, you can read it by entering info:/find in the address bar. Otherwise type info find on the command line. Creating and extracting file archives (zip, gzip and tar) For a single file, you get the best compression usually with bzip2: bzip2 -zv filein > fileout.bz2 This file can be recovered with bzip2 -xv filout.bz2 > filein For multiple files, put them together in a directory and use tar tar -cvjf file.tar.bz2 directory To extract again use tar -xvjf file.tar.bz2 Although bzip2 gives the best compression, gzip is faster, so you may prefer it. two commands become With gzip compression, the above tar -cvzf file.tar.gz directory tar -xvzf file.tar.gz The most common compression format on Windows systems is pkzip: the corresponding commands available on linux are zip and unzip. You can always use the command unp to uncompress a file in any compression format. anything it couldn t uncompress.) (Well I ve never found 7

8 If you want to use a graphical tool to compress and decompress files, there is ark or file-roller. Emacs - a powerful editor Emacs is a powerful editor. It runs either a text console or graphically, as an X program. (To force it to start in the text console mode, use emacs -nw.) For programming, or HTML, or LaTeX, or other similar structured text files, it is very useful. A good tutorial for Emacs is here [ A quick reference card for emacs can be found here [emacs-refcard.pdf]. Note that the "Meta" key, which is indicated by "M-" on the reference card, is the "Alt" on PC keyboards. Running long calculations Getting the prompt back after starting a job Use & at the end of the command. This puts the job in the background. For example:./myjob 10 & To see all your background jobs (the ones started in the current shell), enter: jobs To bring one of these to the foreground, use: fg %n where n is the job number (yes it needs a % in front of it). To move it to the background again, hit CTRL-Z. This however doesn t just move the job to the background, it suspends it. To get it running again, use bg %n Keeping a job running after you log out Use nohup. For example, suppose you want to run./myjob 20. In this case, you could use: nohup nice./myjob 20 > out_file & There are several things to note here: The & at the end of the command line is there so you get the command prompt back immediately (see the previous section). We have redirected the output to a file out_file with > out_file. If you don t do this, any output to STDIN (normally the console) will automatically go into a file called nohup.out. We ve niced the job to reduce it s execution priority. This is courteous. 8

9 If you want to see what s in nohup.out (or whatever you called it) at any time, you can do the usual cat nohup.out Here s another potentially more useful command: tail -f nohup.out tail shows you the last few lines of the files. In addition, the -f flag ("follow") means that as lines get added to the file, they are displayed on the screen. This looks just like when you run the program without nohup, except when you hit CTRL-C, the program keeps on running in the background. One nice refinement is to have the output mailed to you on completion. Try this: nohup nice./longjob job_options ; cat nohup.out mail $USER & Suspending a long-running job and restarting it later First find the PID (process ID) with top or with ps. For example, I might list all jobs belonging to me with: ps -u eric To stop the job, send it a STOP signal: kill -s STOP PID (replace PID with the actual PID number). It is now no longer running, but it still occupies memory (although the memory it uses will probably be swapped out to disk). When you want to resume execution, give it a CONT signal: kill -s CONT PID Running a series of jobs Suppose you want to run ten jobs overnight. You could just start them all at once (using nohup as described above). However, it would be more efficient to run them sequentially. Besides being faster in total (less multitasking overhead), it is also much more considerate to other users to run your jobs in series instead of in parallel. An easy way to run things in sequence is to use a script. For example: #!/bin/bash # This script runs mycalc with arguments 10 through 100../mycalc 10./mycalc 20./mycalc 30./mycalc 40./mycalc 50./mycalc 60./mycalc 70./mycalc 80./mycalc 90./mycalc 100 If you save this script as docalc, you need to make it executable before you can run it. chmod +x docalc 9

10 You could run it like this nohup nice ~/docalc & More advanced scripts As an example of the things you can do with bash scripts, the example script in the previous section could also have been written like this: #!/bin/bash # This script runs mycalc with arguments 10 through 100. ARGS=" " for ARG in $ARGS do./mycalc $ARG done Obviously, if you have a lot of jobs to run, this is easier than writing a script with one line per job. This just gives you a taste of the possibilities. If you find this kind of thing useful, then you should check out Heiner s SHELLdorado [ When you re not in the department Accessing data from any Windows computer For Windows computers external to the physics local network, you need to get some ssh/scp program. Fortunately, this is easy when you know where to look. There is a free ssh (and scp) program from the physics web site [ exe] (internal access only). A publically accessible download location is here [ftp://ftp.ssh.com/pub/ssh/]. This program has a very nice GUI interface. However you need to have installation rights on the Windows computer to install and use it. For other cases, PuTTy [ comes in very handy. There is a program for an ssh terminal (PuTTY), and an scp program (pscp). The pscp program is particularly elegant, since it doesn t even need to be "installed" on the Windows computer. Just put pscp.exe in the directory to/from which you want to transfer files, then open a terminal window (usually under Accessories), cd to the directory (something like CD C:\Users\Bob), and use pscp like you would use scp at the command line on physics. See the section on scp. Yet another option is Knoppix, which is described in the next section... without installation - at home or travelling There s a fantastic distro called Knoppix [ It s a complete system on a CD. You stick the CD in any PC with a CD drive, boot it up, and you have a functioning system with much useful software, like Mozilla, OpenOffice and SSH. It does a fantastic job of hardware detection, and you can mount the hard disks if you need to read/write to them. It s also easy to establish a network connection. In fact if there s a Windows installation on the computer, it will read the network configuration from the system files on the Windows partition. When finished with Knoppix, pop out the CD and reboot and the computer reverts to its 10

11 former personality. I really recommend that you take a Knoppix CD with you when you travel. It makes it very easy to connect back to physics. More Quick Tips Scroll with the keyboard You can scroll back to see lines which have scrolled off the top of your shell window with CTRL-PAGEUP and CTRL-PAGEDOWN. Copy with xclip Here s another useful trick with X-Windows copying and pasting: you can pipe output directly into the clipboard with xclip. For example, if you want to someone a directory listing, do ls xclip then middle-click to paste into your . Lots of keyboard shortcuts Here are some common keystrokes. Most of these will work at the command line, and in many programs. KDE and Gnome programs may be an exception - their default control key mapping is sometimes closer to Windows. However you can change to Unix Default in the KDE Control Center, under Look & Feel, Key Bindings and in the Gnome Control Center, under Keyboard Shortcuts. Once you learn these key bindings, you will find that you can work very fast across many different programs. In addition to the key bindings described below, there are the standard arrow keys, page up and page down keys and so on. However the ones described below all lie under your fingers, and so are faster once you learn them. 11

12 Table 1. Common editing key bindings CTRL-A Move to the end of the line. CTRL-E Move to the beginning of the line. CTRL-F Move forward one character. CTRL-B Move backward one character. ALT-F Move forward one word. ALT-B Move backward one word. CTRL-V Move one page down. ALT-V Move one page up. CTRL-D Delete one character forward. CTRL-H Delete one character backward. ALT-D Delete one word forward. CTRL-K Delete (Kill) everything from the cursor forwards (to the end of the line). CTRL-U Delete everything from the cursor backwards (to the beginning of the line). CTRL-T Transpose two characters. ALT-T Transpose two words. ESC. Insert the last argument (word) from the previous command. Table 2. Common control key bindings 12

13 CTRL-C Terminate the current program, or sometimes cancel some operation. CTRL-\ Seriously kill the current program!! Use only after CTRL-C has failed! CTRL-D End-of-file. This is often used to signal the end of input, or to exit a program. CTRL-G Cancel the current operation. A VERY useful command to get yourself out of things. CTRL-L Redraw the console. You will find this one useful on physics, where the new mail notification tends to overwrite whatever you are working on. CTRL-Q The only time you need this is if you accidently hit CTRL-S. In that case the console is locked until you hit CTRL-Q to unlock it. Apparently once a long long time ago this was actually a useful feature. CTRL-R Recall command containing characters. After hitting CTRL-R, type the fragment you want to search for. If it returns a command which isn t the one one want, hit CTRL-R again to cycle through matches. current program CTRL-Z Suspend 13 the and put it in the background. This is very

14 ssh and scp without typing a password each time You can create and exchange ssh keys to avoid repeatedly typing your password. I will use the example of connection from a machine called berserk to a machine called physics. On berserk, create an ssh key pair with ssh-keygen -t dsa It will prompt you whether you want a pass-phrase or not. session, but it s rather more secure. It s up to you. If you choose one, you have to type this once per log-in Now copy the.ssh/id_dsa.pub file to physics. From berserk run scp.ssh/id_dsa.pub userid@physics:tempid Here we ve renamed it tempid. Then on physics run cat tempid >>.ssh/authorized_keys This will add the contents of tempid to.ssh/authorized_keys. That s it! Your can now remove the temporary file: rm tempid. Now when you use ssh to login to physics from berserk, you won t be asked for your password! Of course, you can do all the same steps, interchanging berserk and physics to go the other direction. Tip If this doesn t work for you, it might be because the permissions on your home directory are too loose. Running the following command on both berserk and physics might help: chmod -R go-rwx ~ This removes reading, writing and executing permissions from all your files for group and other, leaving only permissions for user (you). Send a quick message without starting any mail program Just type for example: mail bob@physics.ubc.ca Then enter a subject when prompted, followed by the message body. When finished, type CTRL-D. Presto! use this to send quick reminders to myself. I often Where to go to learn more Another introduction to to Unix at UBC has been written by Matt Choptuik. [ You can find it here More handy tips for using the bash shell can be found at Getting to know Bash [ A fantastic resource for unix shells in general and shell programming can be found at Heiner s SHELLdorado [ 14