Module 8 Pipes, Redirection and REGEX

Size: px
Start display at page:

Download "Module 8 Pipes, Redirection and REGEX"

Transcription

1 Module 8 Pipes, Redirection and REGEX

2 Exam Objective 3.2 Searching and Extracting Data from Files Objective Summary Piping and redirection Partial POSIX

3 Command Line and Redirection

4 Command Line Pipes The pipe character ( ) can be used between two commands to send the output of the first as input to the second: ls /etc head The output of ls /etc is sent to head as input.

5 Command Line Pipelines Multiple commands can be combined to form pipelines. The order in which commands are added to the pipeline can affect the output:

6 I/O Redirection Three Input/Output (I/O) streams associated with every command: Standard Input (STDIN) is normally provided by the user via the keyboard. Standard Output (STDOUT) is the output produced by the command when operating correctly. STDOUT normally appears in the same window as where command executed. Standard Error (STERR) is the is the output produced by the command when an error has occurred. STDOUT normally appears in the same window as where command executed.

7 I/O Redirection Symbols Summary of redirection possible with the bash shell: < /path/to/file (Redirect STDIN from file) > /path/to/file (Redirect STDOUT overwriting file) >> /path/to/file (Redirect STDOUT appending file) 2> /path/to/file (Redirect STDERR overwriting file) 2>> /path/to/file (Redirect STDERR appending file) &> /path/to/file (Redirect STDERR and STDOUT overwriting file) &>> /path/to/file (Redirect STDERR and STDOUT appending file)

8 The null device The null device is represented by the /dev/null file. (Otherwise known as the Bit Bucket ) This file is very useful in redirection of input and output. This file serves two purposes: any output redirected to /dev/null is discarded. /dev/null can be used for input to provide a stream of null values.

9 STDIN, STDOUT, and STDERR

10 STDIN or 0 Standard Input (STDIN) normally is provided by the keyboard but can be redirected with the < symbol. STDIN can be read by programs to get data for them to process. To signal a program that you wish to stop providing data by the keyboard via STDIN, type CTRL-D. The tr command reads its data from STDIN. It translates from one set of characters to another. If you were the user typing the data to be translated by the tr command, you would type CTRL-D when finished.

11 STDIN from keyboard In the following example, the tr command translates from lowercase to uppercase after the user typed the command and pressed Enter. Then, "alpha" was typed and Enter pressed. Finally, the user typed CTRL-D.

12 Redirecting STDIN from file The tr command translates from lowercase to uppercase with STDIN being redirected from the /etc/hosts file:

13 STDOUT or 1 Standard Out (STDOUT) is the output from the command when operating correctly. It is usually displayed in the same window where the command is executed. The echo command is used to print messages to STDOUT. It can be used to demonstrate how STDOUT can be redirected, as shown on the following slide.

14 Redirecting STDOUT In the example below, the echo Linux 1 command is executed and the output appears on STDOUT. Then, the echo Linux 1 > a.txt command redirects the output to the file a.txt. Finally, the command cat a.txt sends the file contents to STDOUT, so the output is shown.

15 Appending STDOUT redirection Using a single arrow > for STDOUT redirection will clobber, or overwrite, the file specified. Using the double arrow >> for STDOUT redirection will either create a new file or append an existing one:

16 STDERR or 2 Standard Error (STDERR) is the output of a command after an error has occurred. It is normally sent to the console/terminal where the command is executed. ls /fake is a command that will cause an error to be output to STDERR because the /fake file does not exist.

17 Redirecting STDERR ls /fake 2> /tmp/err.msg is a command that would cause an error to be sent to STDERR which is then redirected to the /tmp/err.msg file. The cat /tmp/err.msg command sends the contents of the file to STDOUT to display the file:

18 Disposing of STDERR ls /fake 2> /dev/null is a command that would cause STDERR to be redirected to the /dev/null file, in effect disposing of the error message. Notice cat /dev/null displays no visible output.

19 Working with STDERR and STDOUT find is a command that searches the filesystem. It sends output to STDOUT when it successfully locates a file that matches your criteria. It sends output to STDERR when it fails to access a directory. The find command will be used to demonstrate redirecting both STDOUT and STDERR on the following slides. More detail about the find command

20 STDERR and STDOUT Example The following example demonstrates the find command searching recursively the /etc/pki directory for any files matching "*.crt". Two lines of STDERR and two lines of STDOUT messages appear:

21 Isolating STDERR In the next example, the STDOUT output is redirected to the /dev/null file, so the STDERR output alone is sent to the terminal window:

22 Isolating STDOUT In the next example, the STDERR output is now redirected to /dev/null file, so the STDOUT output alone is sent to the terminal window:

23 Redirecting Multiple Streams Separately In the next example, the STDERR output is sent to the crt.err file and the STDOUT output is sent to the crt.txt file:

24 Redirecting Multiple Streams Combined In this example, both STDOUT and STERR are redirected into the same file, crt.all:

25 find Command

26 Searching with find command The filesystem has hundreds of directories with thousands of files making finding files challenging. The find command is a powerful tool to be able to search for files in different ways including: name size date ownership

27 Syntax of find command The find command has the following syntax: find [start_dir] [search_op] [criteria] [result] If the starting directory (start_dir) is not specified, then the current directory is assumed. The search option (search_op) is how the search will be done. For example, use the -name option to search by name.

28 Syntax of find command (cont'd) The search criteria (criteria) is the data to be used with the search option. So, if the search option was -name, then the search criteria would be the name of the file to find. The result option (result) defaults to -print, which will output the names of the files that are found. Other result options can perform actions on the files that are found.

29 Searching by file name Consider the following command: find /etc/pki -name "*.crt" Begins searching recursively from the /etc/pki directory Output any file names that match "*.crt" (anything that ends in ".crt ).

30 Displaying file detail The option -ls will create output similar to the ls -l command. (show both) The columns output are: inode, blocks used, permissions, link count, user owner, group owner, size, date/time, and file name.

31 Searching by file size The -size option can be used by find to search by its size. Large units can be specified as K, M, G, etc. Using +1M means more than one megabyte. Using -1M means less than one megabyte.

32 Useful options for find command Option Example Meaning -maxdepth -maxdepth 1 Only search specified directory and its immediate subdirectories -group -group payroll Find any files owned by the payroll group -iname -iname hosts Case insensitive search by filename -mmin -mmin -10 Find any files modified in the last ten minutes or less -type -type f Find only regular files -user -user bob Find any files owned by the user bob

33 less Command

34 Viewing files with less command The less command is a pager command designed to display only one page of data at a time. The more command is another pager command that has less features than the less command. Both commands allow the user to move back and forth with movement commands to view one page at a time.

35 The help screen in less Once in the less program, pressing the "h" key will display the help screen:

36 less movement commands As seen in the help screen, the less command has many movement commands. The most common commands are: Movement Window forward Window backward Line forward Exit Help Key Spacebar b Enter q h

37 less searching commands Type / to search from cursor to end of file. Type? to search from cursor to beginning of file. Type pattern to search and press Enter. If more than one match found, press n to go to next match or N to go to previous match.

38 head or tail

39 Filtering with head The head command displays the first ten lines of a file by default. The -n option allows for the number of lines to be displayed to be specified.

40 head with negative lines Normally the head command displays the number of lines specified from the top of the file. Using -n with a negative value, indicates how many lines from the bottom to not show. This example shows all lines from /etc/passwd except the last thirty-two.

41 Filtering with tail The tail command displays the last ten lines of a file by default. The -n option allows for the number of lines to be displayed to be specified:

42 tail with positive lines If the -n option specifies the number of lines to be displayed with a plus ( + ) prefix, then the tail command interprets that to mean to display from that line number to the end of the file:

43 Following with tail The tail command is able to monitor changes a file and print them out as they occur by using -f option. System administrators frequently follow log files in order to troubleshoot system problems. The user must terminate the tail command when following with the f option by using CTRL-C.

44 sort Command

45 Sorting files or input The sort command will rearrange its output lines according to one or more fields you specify for sorting. Fields are separated by whitespace, although with the t option, you can specify the delimiter. The default sort is in ascending order, but you can use the -r option to reverse the sorting of a field. The default sort is a dictionary sort, but you can use the -n option to make it a

46 Example of sort In the following example, the /etc/passwd file is sorted using a : character as a delimiter, by the fourth field numerically and then the third field numerically in reverse:

47 File Statistics

48 File statistics with wc command The wc command outputs up to three statistics for each file it is given as an argument. By default, wc displays the lines, words and bytes contained in each file. If provided more than one file, then it also calculates the totals of all files. To view individual statistics, specify -l for lines, -w for words or -c for bytes.

49 Example of wc command To analyze the number of lines, words and bytes in the /etc/passwd and /etc/passwd- files, the following wc command could be executed:

50 Using wc with pipes The wc command is often used with pipes so that the output of a command can be analyzed. Using wc -l as the final command in the pipe will count how many lines of output was produced. For example, to determine how many files and directories are in the /etc directory, you could execute: ls /etc wc -l

51 cut Command

52 Filtering with cut command If you want to extract columns of text, then the cut command provides two simple techniques: By delimiter, where whitespace is the default. The -d option can let you specify other delimiters and -f is used to indicate which fields to extract. By character position, using the -c option with the range of the column to extract.

53 Example of cut command The /etc/passwd file is delimited by colon with these fields: account:password:uid:gid:gecos:directory:shell To extract the first, and fifth through seventh fields: cut d: -f1,5-7 /etc/passwd

54 grep Command

55 Filtering with grep command The grep command can be used to filter standard input or the contents of a file for lines matching a specified pattern. If you want to see where a pattern, or perhaps a word, appears within a file, then the grep command is useful for that purpose.

56 Common grep options Option Purpose --color Color the matches found -v Reverse (negate) matches -c Count matches -n Number matching lines -l List matching files -i Match case insensitive -w Match pattern as a word

57 Basic Regular Expressions

58 Basic Regular Expressions Basic Regular Expressions (BRE) are able to be used with the grep command without requiring an option to use them (unlike Extended Regular Expression show later). The simplest regular expressions are just alphabetic or numeric characters that match themselves. The backslash \ can be used to escape the meaning of regular expression metacharacters, including the backslash itself.

59 BRE: the. example The. (period) character matches exactly one character. The example below shows the grep command matching the 'a' followed by two characters. The results show it matched 'abc'.

60 BRE: the [ ] example The [ ] (brackets) characters are used to match exactly one character. The characters can be listed or given as a range. If the first character listed is ^ (caret), then it means not the characters bracketed.

61 BRE: the * example The * (asterisk) character will match zero or more of the previous character. Matching "a*" is not very useful because it might match zero a's (matches every line). Matching "abcd*" would be more useful, since you would need an "abc" followed by zero or more d's.

62 BRE: the ^ example The ^ (caret) character, when appearing at the beginning of the pattern, means that pattern must appear at the beginning of the line. The ^ not at the beginning of a pattern matches itself.

63 BRE: the $ example The $ (dollar sign), when appearing at the end of the pattern, means that pattern must appear at the end of the line. The $ not at the beginning of a pattern matches itself.

64 BRE: Combining ^ and $ Combining both the ^ and $ characters allows for two special matches: '^$' is a blank line match. '^pattern$ matches if the whole line contains only the "pattern.

65 Extended Regular Expressions

66 Extended Regular Expressions The use of Extended Regular Expressions (ERE) requires the -E option when using the grep command. Extended Regular Expressions can be combined with Basic Regular Expressions. The following are ERE characters:?, +, and

67 ERE: the + example The + (plus) character will match one or more of the previous character. Matching "a+" is useful because it can match one or more a's, ensuring only lines that have at least one a are matched.

68 ERE: the? example The? (question mark) character will optionally match one of the previous character. The? character is useful for matching characters that only occasionally appear in a word. The following example illustrates this:

69 ERE: the example The (vertical bar) character will act like an or operator between two regular expressions. This alternation operator is useful to be able to match multiple patterns:

70 The xargs command The xargs command helps complex piped command sets execute more efficiently It attempts to build the longest command line possible with as many arguments as possible It tries to prevent executing the command each time for every argument

Table of contents. Our goal. Notes. Notes. Notes. Summer June 29, Our goal is to see how we can use Unix as a tool for developing programs

Table of contents. Our goal. Notes. Notes. Notes. Summer June 29, Our goal is to see how we can use Unix as a tool for developing programs Summer 2010 Department of Computer Science and Engineering York University Toronto June 29, 2010 1 / 36 Table of contents 1 2 3 4 2 / 36 Our goal Our goal is to see how we can use Unix as a tool for developing

More information

Week Overview. Simple filter commands: head, tail, cut, sort, tr, wc grep utility stdin, stdout, stderr Redirection and piping /dev/null file

Week Overview. Simple filter commands: head, tail, cut, sort, tr, wc grep utility stdin, stdout, stderr Redirection and piping /dev/null file ULI101 Week 05 Week Overview Simple filter commands: head, tail, cut, sort, tr, wc grep utility stdin, stdout, stderr Redirection and piping /dev/null file head and tail commands These commands display

More information

ITST Searching, Extracting & Archiving Data

ITST Searching, Extracting & Archiving Data ITST 1136 - Searching, Extracting & Archiving Data Name: Step 1 Sign into a Pi UN = pi PW = raspberry Step 2 - Grep - One of the most useful and versatile commands in a Linux terminal environment is the

More information

CS 307: UNIX PROGRAMMING ENVIRONMENT FIND COMMAND

CS 307: UNIX PROGRAMMING ENVIRONMENT FIND COMMAND CS 307: UNIX PROGRAMMING ENVIRONMENT FIND COMMAND Prof. Michael J. Reale Fall 2014 Finding Files in a Directory Tree Suppose you want to find a file with a certain filename (or with a filename matching

More information

5/20/2007. Touring Essential Programs

5/20/2007. Touring Essential Programs Touring Essential Programs Employing fundamental utilities. Managing input and output. Using special characters in the command-line. Managing user environment. Surveying elements of a functioning system.

More information

Basics. I think that the later is better.

Basics.  I think that the later is better. Basics Before we take up shell scripting, let s review some of the basic features and syntax of the shell, specifically the major shells in the sh lineage. Command Editing If you like vi, put your shell

More information

Lecture 3 Tonight we dine in shell. Hands-On Unix System Administration DeCal

Lecture 3 Tonight we dine in shell. Hands-On Unix System Administration DeCal Lecture 3 Tonight we dine in shell Hands-On Unix System Administration DeCal 2012-09-17 Review $1, $2,...; $@, $*, $#, $0, $? environment variables env, export $HOME, $PATH $PS1=n\[\e[0;31m\]\u\[\e[m\]@\[\e[1;34m\]\w

More information

CST Algonquin College 2

CST Algonquin College 2 The Shell Kernel (briefly) Shell What happens when you hit [ENTER]? Output redirection and pipes Noclobber (not a typo) Shell prompts Aliases Filespecs History Displaying file contents CST8207 - Algonquin

More information

Linux Text Utilities 101 for S/390 Wizards SHARE Session 9220/5522

Linux Text Utilities 101 for S/390 Wizards SHARE Session 9220/5522 Linux Text Utilities 101 for S/390 Wizards SHARE Session 9220/5522 Scott D. Courtney Senior Engineer, Sine Nomine Associates March 7, 2002 http://www.sinenomine.net/ Table of Contents Concepts of the Linux

More information

Bashed One Too Many Times. Features of the Bash Shell St. Louis Unix Users Group Jeff Muse, Jan 14, 2009

Bashed One Too Many Times. Features of the Bash Shell St. Louis Unix Users Group Jeff Muse, Jan 14, 2009 Bashed One Too Many Times Features of the Bash Shell St. Louis Unix Users Group Jeff Muse, Jan 14, 2009 What is a Shell? The shell interprets commands and executes them It provides you with an environment

More information

find starting-directory -name filename -user username

find starting-directory -name filename -user username Lab 7: Input and Output The goal of this lab is to gain proficiency in using I/O redirection to perform tasks on the system. You will combine commands you have learned in this course using shell redirection,

More information

Lab 2: Linux/Unix shell

Lab 2: Linux/Unix shell Lab 2: Linux/Unix shell Comp Sci 1585 Data Structures Lab: Tools for Computer Scientists Outline 1 2 3 4 5 6 7 What is a shell? What is a shell? login is a program that logs users in to a computer. When

More information

Review of Fundamentals

Review of Fundamentals Review of Fundamentals 1 The shell vi General shell review 2 http://teaching.idallen.com/cst8207/14f/notes/120_shell_basics.html The shell is a program that is executed for us automatically when we log

More information

The input can also be taken from a file and similarly the output can be redirected to another file.

The input can also be taken from a file and similarly the output can be redirected to another file. Filter A filter is defined as a special program, which takes input from standard input device and sends output to standard output device. The input can also be taken from a file and similarly the output

More information

DATA 301 Introduction to Data Analytics Command Line. Dr. Ramon Lawrence University of British Columbia Okanagan

DATA 301 Introduction to Data Analytics Command Line. Dr. Ramon Lawrence University of British Columbia Okanagan DATA 301 Introduction to Data Analytics Command Line Dr. Ramon Lawrence University of British Columbia Okanagan ramon.lawrence@ubc.ca Why learn the Command Line? The command line is the text interface

More information

Why learn the Command Line? The command line is the text interface to the computer. DATA 301 Introduction to Data Analytics Command Line

Why learn the Command Line? The command line is the text interface to the computer. DATA 301 Introduction to Data Analytics Command Line DATA 301 Introduction to Data Analytics Command Line Why learn the Command Line? The command line is the text interface to the computer. DATA 301: Data Analytics (2) Understanding the command line allows

More information

18-Sep CSCI 2132 Software Development Lecture 6: Links and Inodes. Faculty of Computer Science, Dalhousie University. Lecture 6 p.

18-Sep CSCI 2132 Software Development Lecture 6: Links and Inodes. Faculty of Computer Science, Dalhousie University. Lecture 6 p. Lecture 6 p.1 Faculty of Computer Science, Dalhousie University CSCI 2132 Software Development Lecture 6: Links and s 18-Sep-2017 Location: Goldberg CS 127 Time: 14:35 15:25 Instructor: Vlado Keselj Previous

More information

Part III. Shell Config. Tobias Neckel: Scripting with Bash and Python Compact Max-Planck, February 16-26,

Part III. Shell Config. Tobias Neckel: Scripting with Bash and Python Compact Max-Planck, February 16-26, Part III Shell Config Compact Course @ Max-Planck, February 16-26, 2015 33 Special Directories. current directory.. parent directory ~ own home directory ~user home directory of user ~- previous directory

More information

RH033 Red Hat Linux Essentials

RH033 Red Hat Linux Essentials RH033 Red Hat Linux Essentials Version 3.5 QUESTION NO: 1 You work as a Network Administrator for McNeil Inc. The company has a Linux-based network. A printer is configured on the network. You want to

More information

Common File System Commands

Common File System Commands Common File System Commands ls! List names of all files in current directory ls filenames! List only the named files ls -t! List in time order, most recent first ls -l! Long listing, more information.

More information

A Brief Introduction to the Linux Shell for Data Science

A Brief Introduction to the Linux Shell for Data Science A Brief Introduction to the Linux Shell for Data Science Aris Anagnostopoulos 1 Introduction Here we will see a brief introduction of the Linux command line or shell as it is called. Linux is a Unix-like

More information

5/8/2012. Exploring Utilities Chapter 5

5/8/2012. Exploring Utilities Chapter 5 Exploring Utilities Chapter 5 Examining the contents of files. Working with the cut and paste feature. Formatting output with the column utility. Searching for lines containing a target string with grep.

More information

UNIX files searching, and other interrogation techniques

UNIX files searching, and other interrogation techniques UNIX files searching, and other interrogation techniques Ways to examine the contents of files. How to find files when you don't know how their exact location. Ways of searching files for text patterns.

More information

CSE 390a Lecture 2. Exploring Shell Commands, Streams, and Redirection

CSE 390a Lecture 2. Exploring Shell Commands, Streams, and Redirection 1 CSE 390a Lecture 2 Exploring Shell Commands, Streams, and Redirection slides created by Marty Stepp, modified by Jessica Miller & Ruth Anderson http://www.cs.washington.edu/390a/ 2 Lecture summary Unix

More information

ADVANCED LINUX SYSTEM ADMINISTRATION

ADVANCED LINUX SYSTEM ADMINISTRATION Lab Assignment 1 Corresponding to Topic 2, The Command Line L1 Main goals To get used to the command line. To gain basic skills with the system shell. To understand some of the basic tools of system administration.

More information

CSCI 2132 Software Development. Lecture 7: Wildcards and Regular Expressions

CSCI 2132 Software Development. Lecture 7: Wildcards and Regular Expressions CSCI 2132 Software Development Lecture 7: Wildcards and Regular Expressions Instructor: Vlado Keselj Faculty of Computer Science Dalhousie University 20-Sep-2017 (7) CSCI 2132 1 Previous Lecture Pipes

More information

Introduction Variables Helper commands Control Flow Constructs Basic Plumbing. Bash Scripting. Alessandro Barenghi

Introduction Variables Helper commands Control Flow Constructs Basic Plumbing. Bash Scripting. Alessandro Barenghi Bash Scripting Alessandro Barenghi Dipartimento di Elettronica, Informazione e Bioingegneria Politecnico di Milano alessandro.barenghi - at - polimi.it April 28, 2015 Introduction The bash command shell

More information

CSC209H Lecture 1. Dan Zingaro. January 7, 2015

CSC209H Lecture 1. Dan Zingaro. January 7, 2015 CSC209H Lecture 1 Dan Zingaro January 7, 2015 Welcome! Welcome to CSC209 Comments or questions during class? Let me know! Topics: shell and Unix, pipes and filters, C programming, processes, system calls,

More information

Command-line interpreters

Command-line interpreters Command-line interpreters shell Wiki: A command-line interface (CLI) is a means of interaction with a computer program where the user (or client) issues commands to the program in the form of successive

More information

Reading and manipulating files

Reading and manipulating files Reading and manipulating files Goals By the end of this lesson you will be able to Read files without using text editors Access specific parts of files Count the number of words and lines in a file Sort

More information

2) clear :- It clears the terminal screen. Syntax :- clear

2) clear :- It clears the terminal screen. Syntax :- clear 1) cal :- Displays a calendar Syntax:- cal [options] [ month ] [year] cal displays a simple calendar. If arguments are not specified, the current month is displayed. In addition to cal, the ncal command

More information

sottotitolo A.A. 2016/17 Federico Reghenzani, Alessandro Barenghi

sottotitolo A.A. 2016/17 Federico Reghenzani, Alessandro Barenghi Titolo presentazione Piattaforme Software per la Rete sottotitolo BASH Scripting Milano, XX mese 20XX A.A. 2016/17, Alessandro Barenghi Outline 1) Introduction to BASH 2) Helper commands 3) Control Flow

More information

Chapter 1 - Introduction. September 8, 2016

Chapter 1 - Introduction. September 8, 2016 Chapter 1 - Introduction September 8, 2016 Introduction Overview of Linux/Unix Shells Commands: built-in, aliases, program invocations, alternation and iteration Finding more information: man, info Help

More information

Lecture 5. Essential skills for bioinformatics: Unix/Linux

Lecture 5. Essential skills for bioinformatics: Unix/Linux Lecture 5 Essential skills for bioinformatics: Unix/Linux UNIX DATA TOOLS Text processing with awk We have illustrated two ways awk can come in handy: Filtering data using rules that can combine regular

More information

CS 25200: Systems Programming. Lecture 11: *nix Commands and Shell Internals

CS 25200: Systems Programming. Lecture 11: *nix Commands and Shell Internals CS 25200: Systems Programming Lecture 11: *nix Commands and Shell Internals Dr. Jef Turkstra 2018 Dr. Jeffrey A. Turkstra 1 Lecture 11 Shell commands Basic shell internals 2018 Dr. Jeffrey A. Turkstra

More information

System Administration

System Administration Süsteemihaldus MTAT.08.021 System Administration UNIX shell basics Name service DNS 1/69 Command Line Read detailed manual for specific command using UNIX online documentation or so called manual (man)

More information

Introduction to UNIX Part II

Introduction to UNIX Part II T H E U N I V E R S I T Y of T E X A S H E A L T H S C I E N C E C E N T E R A T H O U S T O N S C H O O L of H E A L T H I N F O R M A T I O N S C I E N C E S Introduction to UNIX Part II For students

More information

CS Unix Tools. Fall 2010 Lecture 5. Hussam Abu-Libdeh based on slides by David Slater. September 17, 2010

CS Unix Tools. Fall 2010 Lecture 5. Hussam Abu-Libdeh based on slides by David Slater. September 17, 2010 Fall 2010 Lecture 5 Hussam Abu-Libdeh based on slides by David Slater September 17, 2010 Reasons to use Unix Reason #42 to use Unix: Wizardry Mastery of Unix makes you a wizard need proof? here is the

More information

Advanced training. Linux components Command shell. LiLux a.s.b.l.

Advanced training. Linux components Command shell. LiLux a.s.b.l. Advanced training Linux components Command shell LiLux a.s.b.l. alexw@linux.lu Kernel Interface between devices and hardware Monolithic kernel Micro kernel Supports dynamics loading of modules Support

More information

More Scripting and Regular Expressions. Todd Kelley CST8207 Todd Kelley 1

More Scripting and Regular Expressions. Todd Kelley CST8207 Todd Kelley 1 More Scripting and Regular Expressions Todd Kelley kelleyt@algonquincollege.com CST8207 Todd Kelley 1 Regular Expression Summary Regular Expression Examples Shell Scripting 2 Do not confuse filename globbing

More information

CS160A EXERCISES-FILTERS2 Boyd

CS160A EXERCISES-FILTERS2 Boyd Exercises-Filters2 In this exercise we will practice with the Unix filters cut, and tr. We will also practice using paste, even though, strictly speaking, it is not a filter. In addition, we will expand

More information

CS 307: UNIX PROGRAMMING ENVIRONMENT KATAS FOR EXAM 2

CS 307: UNIX PROGRAMMING ENVIRONMENT KATAS FOR EXAM 2 CS 307: UNIX PROGRAMMING ENVIRONMENT KATAS FOR EXAM 2 Prof. Michael J. Reale Fall 2014 COMMAND KATA 7: VARIABLES Command Kata 7: Preparation First, go to ~/cs307 cd ~/cs307 Make directory dkata7 and go

More information

CST Lab #5. Student Name: Student Number: Lab section:

CST Lab #5. Student Name: Student Number: Lab section: CST8177 - Lab #5 Student Name: Student Number: Lab section: Working with Regular Expressions (aka regex or RE) In-Lab Demo - List all the non-user accounts in /etc/passwd that use /sbin as their home directory.

More information

bash, part 3 Chris GauthierDickey

bash, part 3 Chris GauthierDickey bash, part 3 Chris GauthierDickey More redirection As you know, by default we have 3 standard streams: input, output, error How do we redirect more than one stream? This requires an introduction to file

More information

Paolo Santinelli Sistemi e Reti. Regular expressions. Regular expressions aim to facilitate the solution of text manipulation problems

Paolo Santinelli Sistemi e Reti. Regular expressions. Regular expressions aim to facilitate the solution of text manipulation problems aim to facilitate the solution of text manipulation problems are symbolic notations used to identify patterns in text; are supported by many command line tools; are supported by most programming languages;

More information

Shells and Shell Programming

Shells and Shell Programming Shells and Shell Programming 1 Shells A shell is a command line interpreter that is the interface between the user and the OS. The shell: analyzes each command determines what actions are to be performed

More information

When talking about how to launch commands and other things that is to be typed into the terminal, the following syntax is used:

When talking about how to launch commands and other things that is to be typed into the terminal, the following syntax is used: Linux Tutorial How to read the examples When talking about how to launch commands and other things that is to be typed into the terminal, the following syntax is used: $ application file.txt

More information

Week 5 Lesson 5 02/28/18

Week 5 Lesson 5 02/28/18 Week 5 Lesson 5 02/28/18 Important Announcements Extra Credits If you haven t done so, send your pictures to risimms@cabrillo.edu for 3 points EXTRA CREDIT. Join LinkedIn for 3 points Perkins/VTEA Survey

More information

Basic Linux (Bash) Commands

Basic Linux (Bash) Commands Basic Linux (Bash) Commands Hint: Run commands in the emacs shell (emacs -nw, then M-x shell) instead of the terminal. It eases searching for and revising commands and navigating and copying-and-pasting

More information

UNIX, GNU/Linux and simple tools for data manipulation

UNIX, GNU/Linux and simple tools for data manipulation UNIX, GNU/Linux and simple tools for data manipulation Dr Jean-Baka DOMELEVO ENTFELLNER BecA-ILRI Hub Basic Bioinformatics Training Workshop @ILRI Addis Ababa Wednesday December 13 th 2017 Dr Jean-Baka

More information

Introduction to UNIX. Introduction. Processes. ps command. The File System. Directory Structure. UNIX is an operating system (OS).

Introduction to UNIX. Introduction. Processes. ps command. The File System. Directory Structure. UNIX is an operating system (OS). Introduction Introduction to UNIX CSE 2031 Fall 2012 UNIX is an operating system (OS). Our goals: Learn how to use UNIX OS. Use UNIX tools for developing programs/ software, specifically shell programming.

More information

Introduction to UNIX. CSE 2031 Fall November 5, 2012

Introduction to UNIX. CSE 2031 Fall November 5, 2012 Introduction to UNIX CSE 2031 Fall 2012 November 5, 2012 Introduction UNIX is an operating system (OS). Our goals: Learn how to use UNIX OS. Use UNIX tools for developing programs/ software, specifically

More information

Unix Internal Assessment-2 solution. Ans:There are two ways of starting a job in the background with the shell s & operator and the nohup command.

Unix Internal Assessment-2 solution. Ans:There are two ways of starting a job in the background with the shell s & operator and the nohup command. Unix Internal Assessment-2 solution 1 a.explain the mechanism of process creation. Ans: There are three distinct phases in the creation of a process and uses three important system calls viz., fork, exec,

More information

Review of Fundamentals. Todd Kelley CST8207 Todd Kelley 1

Review of Fundamentals. Todd Kelley CST8207 Todd Kelley 1 Review of Fundamentals Todd Kelley kelleyt@algonquincollege.com CST8207 Todd Kelley 1 GPL the shell SSH (secure shell) the Course Linux Server RTFM vi general shell review 2 These notes are available on

More information

CSE 390a Lecture 2. Exploring Shell Commands, Streams, Redirection, and Processes

CSE 390a Lecture 2. Exploring Shell Commands, Streams, Redirection, and Processes CSE 390a Lecture 2 Exploring Shell Commands, Streams, Redirection, and Processes slides created by Marty Stepp, modified by Jessica Miller & Ruth Anderson http://www.cs.washington.edu/390a/ 1 2 Lecture

More information

Files

Files http://www.cs.fsu.edu/~langley/cop3353-2013-1/reveal.js-2013-02-11/02.html?print-pdf 02/11/2013 10:55 AM Files A normal "flat" file is a collection of information. It's usually stored somewhere reasonably

More information

CS395T: Introduction to Scientific and Technical Computing

CS395T: Introduction to Scientific and Technical Computing CS395T: Introduction to Scientific and Technical Computing Instructors: Dr. Karl W. Schulz, Research Associate, TACC Dr. Bill Barth, Research Associate, TACC Outline Continue with Unix overview File attributes

More information

Finding files and directories (advanced), standard streams, piping

Finding files and directories (advanced), standard streams, piping Finding files and directories (advanced), standard streams, piping Laboratory of Genomics & Bioinformatics in Parasitology Department of Parasitology, ICB, USP Finding files or directories When you have

More information

Unix/Linux Primer. Taras V. Pogorelov and Mike Hallock School of Chemical Sciences, University of Illinois

Unix/Linux Primer. Taras V. Pogorelov and Mike Hallock School of Chemical Sciences, University of Illinois Unix/Linux Primer Taras V. Pogorelov and Mike Hallock School of Chemical Sciences, University of Illinois August 25, 2017 This primer is designed to introduce basic UNIX/Linux concepts and commands. No

More information

Lec 1 add-on: Linux Intro

Lec 1 add-on: Linux Intro Lec 1 add-on: Linux Intro Readings: - Unix Power Tools, Powers et al., O Reilly - Linux in a Nutshell, Siever et al., O Reilly Summary: - Linux File System - Users and Groups - Shell - Text Editors - Misc

More information

An Illustrated Guide to Shell Magic: Standard I/O & Redirection

An Illustrated Guide to Shell Magic: Standard I/O & Redirection An Illustrated Guide to Shell Magic: Standard I/O & Redirection Created by Brennen Bearnes Last updated on 2015-03-03 05:15:07 PM EST Guide Contents Guide Contents Overview Input & Output Standard I/O

More information

Assume that username is cse. The user s home directory will be /home/cse. You may remember what the relative pathname for users home directory is: ~

Assume that username is cse. The user s home directory will be /home/cse. You may remember what the relative pathname for users home directory is: ~ Introduction to Open Source Software Development Spring semester, 2017 School of Computer Science and Engineering, Pusan National University Joon-Seok Kim LINUX: COMMANDS Review Lab #1 2 Create Directories

More information

Introduction: What is Unix?

Introduction: What is Unix? Introduction Introduction: What is Unix? An operating system Developed at AT&T Bell Labs in the 1960 s Command Line Interpreter GUIs (Window systems) are now available Introduction: Unix vs. Linux Unix

More information

Linux Command Line Primer. By: Scott Marshall

Linux Command Line Primer. By: Scott Marshall Linux Command Line Primer By: Scott Marshall Draft: 10/21/2007 Table of Contents Topic Page(s) Preface 1 General Filesystem Background Information 2 General Filesystem Commands 2 Working with Files and

More information

- c list The list specifies character positions.

- c list The list specifies character positions. CUT(1) BSD General Commands Manual CUT(1)... 1 PASTE(1) BSD General Commands Manual PASTE(1)... 3 UNIQ(1) BSD General Commands Manual UNIQ(1)... 5 HEAD(1) BSD General Commands Manual HEAD(1)... 7 TAIL(1)

More information

Unix as a Platform Exercises + Solutions. Course Code: OS 01 UNXPLAT

Unix as a Platform Exercises + Solutions. Course Code: OS 01 UNXPLAT Unix as a Platform Exercises + Solutions Course Code: OS 01 UNXPLAT Working with Unix Most if not all of these will require some investigation in the man pages. That's the idea, to get them used to looking

More information

Using UNIX. -rwxr--r-- 1 root sys Sep 5 14:15 good_program

Using UNIX. -rwxr--r-- 1 root sys Sep 5 14:15 good_program Using UNIX. UNIX is mainly a command line interface. This means that you write the commands you want executed. In the beginning that will seem inferior to windows point-and-click, but in the long run the

More information

STATS Data Analysis using Python. Lecture 15: Advanced Command Line

STATS Data Analysis using Python. Lecture 15: Advanced Command Line STATS 700-002 Data Analysis using Python Lecture 15: Advanced Command Line Why UNIX/Linux? As a data scientist, you will spend most of your time dealing with data Data sets never arrive ready to analyze

More information

EECS2301. Lab 1 Winter 2016

EECS2301. Lab 1 Winter 2016 EECS2301 Lab 1 Winter 2016 Lab Objectives In this lab, you will be introduced to the Linux operating system. The basic commands will be presented in this lab. By the end of you alb, you will be asked to

More information

ECE 364 Software Engineering Tools Lab. Lecture 2 Bash II

ECE 364 Software Engineering Tools Lab. Lecture 2 Bash II ECE 364 Software Engineering Tools Lab Lecture 2 Bash II 1 Lecture 2 Summary Arrays I/O Redirection Pipes Quotes Capturing Command Output Commands: cat, head, tail, cut, paste, wc 2 Array Variables Declaring

More information

Unix as a Platform Exercises. Course Code: OS-01-UNXPLAT

Unix as a Platform Exercises. Course Code: OS-01-UNXPLAT Unix as a Platform Exercises Course Code: OS-01-UNXPLAT Working with Unix 1. Use the on-line manual page to determine the option for cat, which causes nonprintable characters to be displayed. Run the command

More information

CSC UNIX System, Spring 2015

CSC UNIX System, Spring 2015 CSC 352 - UNIX System, Spring 2015 Study guide for the CSC352 midterm exam (20% of grade). Dr. Dale E. Parson, http://faculty.kutztown.edu/parson We will have a midterm on March 19 on material we have

More information

Systems Programming/ C and UNIX

Systems Programming/ C and UNIX Systems Programming/ C and UNIX December 7-10, 2017 1/17 December 7-10, 2017 1 / 17 Outline 1 2 Using find 2/17 December 7-10, 2017 2 / 17 String Pattern Matching Tools Regular Expressions Simple Examples

More information

A shell can be used in one of two ways:

A shell can be used in one of two ways: Shell Scripting 1 A shell can be used in one of two ways: A command interpreter, used interactively A programming language, to write shell scripts (your own custom commands) 2 If we have a set of commands

More information

Review of Fundamentals. Todd Kelley CST8207 Todd Kelley 1

Review of Fundamentals. Todd Kelley CST8207 Todd Kelley 1 Review of Fundamentals Todd Kelley kelleyt@algonquincollege.com CST8207 Todd Kelley 1 The CST8207 course notes GPL the shell SSH (secure shell) the Course Linux Server RTFM vi general shell review 2 Linux

More information

22-Sep CSCI 2132 Software Development Lecture 8: Shells, Processes, and Job Control. Faculty of Computer Science, Dalhousie University

22-Sep CSCI 2132 Software Development Lecture 8: Shells, Processes, and Job Control. Faculty of Computer Science, Dalhousie University Lecture 8 p.1 Faculty of Computer Science, Dalhousie University CSCI 2132 Software Development Lecture 8: Shells, Processes, and Job Control 22-Sep-2017 Location: Goldberg CS 127 Time: 14:35 15:25 Instructor:

More information

Shells and Shell Programming

Shells and Shell Programming Shells and Shell Programming Shells A shell is a command line interpreter that is the interface between the user and the OS. The shell: analyzes each command determines what actions are to be performed

More information

Useful Unix Commands Cheat Sheet

Useful Unix Commands Cheat Sheet Useful Unix Commands Cheat Sheet The Chinese University of Hong Kong SIGSC Training (Fall 2016) FILE AND DIRECTORY pwd Return path to current directory. ls List directories and files here. ls dir List

More information

CS246 Spring14 Programming Paradigm Files, Pipes and Redirection

CS246 Spring14 Programming Paradigm Files, Pipes and Redirection 1 Files 1.1 File functions Opening Files : The function fopen opens a file and returns a FILE pointer. FILE *fopen( const char * filename, const char * mode ); The allowed modes for fopen are as follows

More information

5/8/2012. Creating and Changing Directories Chapter 7

5/8/2012. Creating and Changing Directories Chapter 7 Creating and Changing Directories Chapter 7 Types of files File systems concepts Using directories to create order. Managing files in directories. Using pathnames to manage files in directories. Managing

More information

File Commands. Objectives

File Commands. Objectives File Commands Chapter 2 SYS-ED/Computer Education Techniques, Inc. 2: 1 Objectives You will learn: Purpose and function of file commands. Interrelated usage of commands. SYS-ED/Computer Education Techniques,

More information

Files and Directories

Files and Directories CSCI 2132: Software Development Files and Directories Norbert Zeh Faculty of Computer Science Dalhousie University Winter 2019 Files and Directories Much of the operation of Unix and programs running on

More information

PESIT Bangalore South Campus

PESIT Bangalore South Campus INTERNAL ASSESSMENT TEST - 2 Date : 20/09/2016 Max Marks : 0 Subject & Code : Unix Shell Programming (15CS36) Section : 3 rd Sem ISE/CSE Name of faculty : Prof Ajoy Time : 11:30am to 1:00pm SOLUTIONS 1

More information

Utilities. September 8, 2015

Utilities. September 8, 2015 Utilities September 8, 2015 Useful ideas Listing files and display text and binary files Copy, move, and remove files Search, sort, print, compare files Using pipes Compression and archiving Your fellow

More information

Basic Linux Command Line Interface Guide

Basic Linux Command Line Interface Guide This basic Linux Command-Line Interface (CLI) Guide provides a general explanation of commonly used Bash shell commands for the Barracuda NG Firewall. You can access the command-line interface by connecting

More information

This is Lab Worksheet 3 - not an Assignment

This is Lab Worksheet 3 - not an Assignment This is Lab Worksheet 3 - not an Assignment This Lab Worksheet contains some practical examples that will prepare you to complete your Assignments. You do not have to hand in this Lab Worksheet. Make sure

More information

Crash Course in Unix. For more info check out the Unix man pages -orhttp://www.cs.rpi.edu/~hollingd/unix. -or- Unix in a Nutshell (an O Reilly book).

Crash Course in Unix. For more info check out the Unix man pages -orhttp://www.cs.rpi.edu/~hollingd/unix. -or- Unix in a Nutshell (an O Reilly book). Crash Course in Unix For more info check out the Unix man pages -orhttp://www.cs.rpi.edu/~hollingd/unix -or- Unix in a Nutshell (an O Reilly book). 1 Unix Accounts To access a Unix system you need to have

More information

Essentials for Scientific Computing: Bash Shell Scripting Day 3

Essentials for Scientific Computing: Bash Shell Scripting Day 3 Essentials for Scientific Computing: Bash Shell Scripting Day 3 Ershaad Ahamed TUE-CMS, JNCASR May 2012 1 Introduction In the previous sessions, you have been using basic commands in the shell. The bash

More information

Linux II and III. Douglas Scofield. Crea-ng directories and files 18/01/14. Evolu5onary Biology Centre, Uppsala University

Linux II and III. Douglas Scofield. Crea-ng directories and files 18/01/14. Evolu5onary Biology Centre, Uppsala University Linux II and III Douglas Scofield Evolu5onary Biology Centre, Uppsala University douglas.scofield@ebc.uu.se slides at Crea-ng directories and files mkdir 1 Crea-ng directories and files touch if file does

More information

Basic Unix Command. It is used to see the manual of the various command. It helps in selecting the correct options

Basic Unix Command. It is used to see the manual of the various command. It helps in selecting the correct options Basic Unix Command The Unix command has the following common pattern command_name options argument(s) Here we are trying to give some of the basic unix command in Unix Information Related man It is used

More information

Essentials for Scientific Computing: Stream editing with sed and awk

Essentials for Scientific Computing: Stream editing with sed and awk Essentials for Scientific Computing: Stream editing with sed and awk Ershaad Ahamed TUE-CMS, JNCASR May 2012 1 Stream Editing sed and awk are stream processing commands. What this means is that they are

More information

Introduction to UNIX. Logging in. Basic System Architecture 10/7/10. most systems have graphical login on Linux machines

Introduction to UNIX. Logging in. Basic System Architecture 10/7/10. most systems have graphical login on Linux machines Introduction to UNIX Logging in Basic system architecture Getting help Intro to shell (tcsh) Basic UNIX File Maintenance Intro to emacs I/O Redirection Shell scripts Logging in most systems have graphical

More information

Introduction to the Shell

Introduction to the Shell [Software Development] Introduction to the Shell Davide Balzarotti Eurecom Sophia Antipolis, France What a Linux Desktop Installation looks like What you need Few Words about the Graphic Interface Unlike

More information

UNIX System Programming Lecture 3: BASH Programming

UNIX System Programming Lecture 3: BASH Programming UNIX System Programming Outline Filesystems Redirection Shell Programming Reference BLP: Chapter 2 BFAQ: Bash FAQ BMAN: Bash man page BPRI: Bash Programming Introduction BABS: Advanced Bash Scripting Guide

More information

Introduction to Unix

Introduction to Unix Part 2: Looking into a file Introduction to Unix Now we want to see how the files are structured. Let's look into one. more $ more fe_03_06596.txt 0.59 1.92 A-f: hello 1.96 2.97 B-m: (( hello )) 2.95 3.98

More information

Unix Introduction to UNIX

Unix Introduction to UNIX Unix Introduction to UNIX Get Started Introduction The UNIX operating system Set of programs that act as a link between the computer and the user. Developed in 1969 by a group of AT&T employees Various

More information

Chapter 9. Shell and Kernel

Chapter 9. Shell and Kernel Chapter 9 Linux Shell 1 Shell and Kernel Shell and desktop enviroment provide user interface 2 1 Shell Shell is a Unix term for the interactive user interface with an operating system A shell usually implies

More information

Linux & Shell Programming 2014

Linux & Shell Programming 2014 Practical No : 1 Enrollment No: Group : A Practical Problem Write a date command to display date in following format: (Consider current date as 4 th January 2014) 1. dd/mm/yy hh:mm:ss 2. Today's date is:

More information

Unix Shell. Advanced Shell Tricks

Unix Shell. Advanced Shell Tricks The Advanced Shell Tricks Copyright The University of Southampton 2011 This work is licensed under the Creative Commons Attribution License See http://software-carpentry.org/license.html for more information.

More information

Cisco IOS Shell. Finding Feature Information. Prerequisites for Cisco IOS.sh. Last Updated: December 14, 2012

Cisco IOS Shell. Finding Feature Information. Prerequisites for Cisco IOS.sh. Last Updated: December 14, 2012 Cisco IOS Shell Last Updated: December 14, 2012 The Cisco IOS Shell (IOS.sh) feature provides shell scripting capability to the Cisco IOS command-lineinterface (CLI) environment. Cisco IOS.sh enhances

More information