Introduction to Linux/Unix Xiaoge Wang wangx147@msu.edu
How does this class works We are going to cover some basics with hands on exampes. Exercises are denoted by the following icon in this presents: Examples will be showed and code will be provided. Please try to follow the demo. The history of demo will be provided at the end of lecture.
Green and Red Sticky Use the sdcky notes provided to help me help you. No S%cky = I am working Green = I am done and ready to move on (yea!) Red = I am stuck and need more Dme and/or some help
Agenda IntroducDon Linux/Shell Tour Commands Set 1: look around Set 2: Do something Summary Pipeline Standard input and output and redirecdon Pipeline of commands Scripts Grammar(key words and special characters) Scope of script and recursive Variables, expression, expansion Control flow ExecuDon model (source vs. direct run, ) Summary
Agenda Linux/Shell Tour Commands Set 1: look around Set 2: Do something Summary Pipeline Standard input and output and redirecdon Pipeline of commands Scripts Grammar(key words and special characters) Scope of script and recursive Variables, expression, expansion Control flow ExecuDon model (source vs. direct run, ) Summary
Introduction Get ready Ticket? Equipment? Big Map Linux/Unix Shell Overview of trail Commands Shell script Get ready for HPC and more.
Exercise 1: ssh ssh yournedd@hpcc.msu.edu Connect to HPCC: go to any develop node. Windows users MobaXterm Mac users Terminal Linux users? Show mod and ssh dev-x
Mac Example of terminal
Show screen
MobaXterm
Big Picture Shell Shell
Big Picture Shell CLI GUI
OS Shell Example
Overview of the Tour Commands Set 1: Look around Set 2: Do something Summary Pipeline Standard input and output and redirecdon Pipeline of commands Scripts Grammar(key words and special characters) Scope of script and recursive Variables, expression, expansion Control flow ExecuDon model (source vs. direct run, )
Linux/Shell Tour Ready? GO!
Agenda IntroducDon Pipeline Standard Input and Output RedirecDon >, >>,.& Pipeline of commands Scripts Grammar(key words and special characters) Scope of script and recursive Variables, expression, expansion, special charactors ($, backdck, =,!, ) Control flow ExecuDon model (source vs. direct run, ) Help and self-explore (man, --help, google, sites, examples, ) Summary
Set 1: Look Around What are in your home directory? What you can tell about an item in the directory? Where are commands I can see? What is the content of a file? What sodware ready for use? How many jobs are running? Do I have jobs running? How many jobs are waidng in the queue? Who are currently on this node? Who are they? Who am I? Why machine is so slow? Who is using it?...
Demo 1 ls, pwd, cd, more, less, cat, head, tail, env, set, module list/spider, export, echo, ps, top, mod, grep,... (1) stardng from home directory: pwd: absolute path and reladve path ls: permission, ownership, others (2) Look into file content cat, more, less, head, tail, find (3) Look into command repository (PATH) echo $PATH Module list/spider/show Powertools man, whads, which (4) Look into what is running on the nodes top, ps, mod, showq, qstat who, whoami, finger, userinfo (5) Look into users who, whoami, finger, userinfo
Exercise 2 Find the number of files in your home directory Find the CPU load and memory usage of the dev node.
Set 2: Do Something AddiDon Bring in data/files Create new files Load something from HPCC SubtracDon Delete file or folder Unload modules Change/move things
Demo 2 Touch, cp, mv, rm, mkdir, module load, scp, curl, wget, vi, nano, > and >>, chmod, export, (1) Add something a) create locally: mkdir, cp mv, vi/vim/nano >, >> b) cp from PC: scp, (not cover disk drive mapping) c) down load from Internet site (not involve the GUI) d) by getexample, getexample, module load (2) Remove something, rm, mv, module purge, module unload (3) Change something chmod, mv (with careful), touch, export
Exercise 2 Copy example files to from /mnt/scratch/ class0/intro2linux.tar to you home directory. Untar it to a directory by $ tar xvf tarfilename Or copy directory intro2linux to your home directory
Summary Command format Command name OpDons: strings, integers, filenames Input/output: stdin, stdout, stderr, redirecdon Command locadon PATH and how to set Tricks Tab (Key board short cut)! Arrows
Agenda IntroducDon r Commands Set 1: look around Set 2: Do something Summary Scripts Grammar(key words and special characters) Scope of script and recursive Variables, expression, expansion Control flow ExecuDon model (source vs. direct run, ) Summary
I/O and Redirection Standard input and output >, >>, >&, <
Demo 3 Use input file to replace the stdin Stdout to create a output
Pipeline Task need more commands to complete One command s output is used for next command s input Format: Commd1 commd2 Commonly used grep, wc, sort, less, more, uniq, head, tail,
Examples of pipeline Filtering output showq grep userid SorDng the output ls l sort CounDng ls wc Beqer viewing ps more
Exercise 4 Use pipeline to find out how many people are login. Find out how many jobs in job queue. Among them, how many are running.
Let s Take a Break
Agenda IntroducDon Commands Set 1: look around Set 2: Do something Summary Pipeline Standard Input and Output and redirecdon Pipeline of commands Summary
Shell Script Combine more commands together for more sophisdcate/complicated task. Grammar Environment and scope Parameters, expressions, expansion, Control flow (branch, loops) ExecuDon (source vs. direct run, )
Environment and Scope Variables IniDalizaDon: /etc/profile ~/.profile ~/.bashrc Local variable
Demo 4 Set inidal environment such that module powertools is loaded in
Exercise 5 Add ~/intro2linux to your PATH by add following line into.bashrc file: export PATH=~/intro2linux:$PATH To check the result, use echo $PATH To see if it is added into the PATH
Control Flow if exit for while un%l case break con%nue
if First form if condi&on ; then commands Fi Second form if condi&on ; then commands else commands fi Third form if condi&on ; then commands elif condi&on ; then commands fi
What is a "condition"? Test First form test expression Second form [ expression ] Return value
Expression examples -d file True if file exists and is a directory. -e file True if file exists (regardless of type). -f file True if file exists and is a regular file. file1 -nt file2 True if file1 exists and is newer than file2. s1 = s2 True if the strings s1 and s2 are idendcal. s1 < s2 True if string s1 comes before s2 based on the ASCII value of their characters. n1 -eq n2 True if the integers n1 and n2 are algebraically equal. expression1 -a expression2 True if both expression1 and expression2 are true. NOTE: use man test to get more details of expression
Example $ more tesdf $./tesdf file1 file2
Case More branching Form case word in paqerns ) statements ;; esac Example (in case_ex) #!/bin/bash echo -n "Type a digit or a leqer > " read character case $character in # Check for leqers [a-z] [A-Z] ) echo "You typed the leqer $character" ;; # Check for digits [0-9] ) echo "You typed the digit $character" ;; # Check for anything else * ) echo "You did not type a leqer or a digit" esac
Loops: while, for, until Example 1 (in while_ex): #!/bin/bash number=0 while [ $number lt 5 ]; do echo "Number = $number # do things here number=$((number + 1)) done Example 2 (in undl_ex): #!/bin/bash number=0 un%l [ $number ge 5 ]; do echo "Number = $number # do things here number=$((number + 1)) done
Loops: for loop: for variable in words; do statements done Example (in for_ex): #!/bin/bash for number in 1 2 3 4 5; do echo Number = $((number-1)) # do things here # do not need this: number=$((number + 1)) done
Quotes and Escaping There are three recognized types: per-character escaping using a backslash: \ $stuff weak quo%ng with double-quotes: "stuff" strong quo%ng with single-quotes: 'stuff' All three forms have the very same purpose: They give you general control over parsing, expansion and expansion results. NOTE: Bash contains ANSI C-like escape sequences
Example: quotes and escaping In quote_escape_ex #/bin/bash # first $HOME is not expand, so as double quote echo \$HOME is set to \"$HOME\" # weak quote echo "My PATH is: $PATH" # strong quote echo 'My PATH is: $PATH
Expansions and substitutions Parameter expansion $WORD, ${STUFF } Pathname expansion *.txt, page_1?.html ArithmeDc expansion $(( EXPRESSION )) Command subsdtudon $( COMMAND ), ` COMMAND ` Tilde expansion ~, ~+, ~- Brace expansion {X,Y,Z}, {X..Y}, {X..Y..Z} Process subsdtudon <( COMMAND ), >( COMMAND )
Parameter Expansion PosiDonal parameters Special parameters Named variables System defined variables Script defined variables
Position Parameters The arguments given to your scripts when it is invoked (command line). It could be from $1 to $N. It has total number of $# $@ expands the list of all Note: When N consists of more than a single digit, it must be enclosed in a braces like ${N}.
Special Parameters variable Descrip%on $# The number of arguments supplied to a script. $* All the arguments are double quoted. If a script receives two arguments, $* is equivalent to $1 $2. $@ All the arguments are individually double quoted. If a script receives two arguments, $@ is equivalent to $1 $2. $? The exit status of the last command executed. $$ The process number of the current shell. For shell scripts, this is the process ID under which they are executing. $! The process number of the last background command. $0 The filename of the current script.
Named Variables Some names are taken: BASH BASH_ARGC BASH_ARGV BASH_LINENO BASH_SOURCE BASH_VERSINFO BASH_VERSION COLUMNS DIRSTACK DISPLAY EDITOR EUID GROUPS HISTFILE HISTFILESIZE HISTSIZE HOME HOSTNAME IFS LANG LANGUAGE LC_ALL LINES LOGNAME LS_COLORS MACHTYPE MAILCHECK OLDPWD OPTERR OPTIND OSTYPE PATH PIPESTATUS PPID PROMPT_COMMAND PS1 PS2 PS4 PS3 PWD SHELL SHELLOPTS SHLVL TERM UID USER USERNAME XAUTHORITY
Examples of Expansion brace_expansion_ex arthm_ex pipe_in_shell pleasant posidon_var_ex
Execution model Directly run Use source What is the difference? Example in execudon_model_ex
Agenda IntroducDon Linux/Shell Tour Commands Set 1: look around Set 2: Do something Summary Pipeline Standard Input and Output and RedirecDon Pipeline of commands Scripts Grammar(key words and special characters) Scope of script and recursive Variables, expression, expansion Control flow ExecuDon model (source vs. direct run, )
Summary Commands Pipeline: special sequence of commands Shell scripts: A composidon of commands Useful resources: man page internet exisdng command
Homework A directory contains 1000 text files. Create a list of all files that contain the word "Aries" and save the result to a files called results.txt. Daily backup of your scratch space. Send me quesdons by email with subject Intro2Linux.
Q & A Thank You!