Systems Programming/ C and UNIX

Size: px
Start display at page:

Download "Systems Programming/ C and UNIX"

Transcription

1 Systems Programming/ C and UNIX December 7-10, /17 December 7-10, / 17

2 Outline 1 2 Using find 2/17 December 7-10, / 17

3 String Pattern Matching Tools Regular Expressions Simple Examples of Quotes, Escapes, Alternatives, Sequences, and Wild Cards 3/17 December 7-10, / 17

4 String Pattern Matching Tools Three popular (and very useful) utilities let you search text for lines or strings that match a regular expression: grep is a very useful command to locate files with a particular characteristic. The name came from usage in ed, the old Unix text editor, where you wrote: global / regular expression / print emacs is the Free Software Foundation text editor used by many experts. It is more powerful and more complex than vi, the Unix editor and much more flexible than Word and similar editing applications. Emacs is especially useful when dealing with non-printing characters and editing data listings that result from running other Unix commands. sed (stream editor) is a filter used for reading a stream of text, identifying and replacing patterns, and writing the result. The file based version, ed is similar and still exists. 4/17 December 7-10, / 17

5 Regular Expressions In the context of grep, a regular expression is a pattern consisting of: Characters that must exactly match the text being searched.. which matches exactly one character of any sort. * is a postfix operator that is applied to the previous pattern element. The combination matches zero or more repetitions of the pattern element. Matching can be either unanchored or anchored. If the grep pattern is matched, the whole line is sent to stdout. A pattern can be anchored to either end of the line or be unanchored. If a pattern starts with ^ the match must start at the beginning of the line. If a pattern ends with $ the match must end at the end of the line. A match for an unanchored pattern can be anywhere on the line. 5/17 December 7-10, / 17

6 Grouping in Regular Expressions Alternatives and sets of characters can be specified: [ ] are used to group alternative characters or ranges of characters. A match is found if the input text matches any one of the alternatives. [a-z] means the set of characters from a to z. ( ) are used to surround parts of a pattern, with the meaning that whatever matches that part of the pattern is supposed to be captured and saved for reuse later in the pattern. This enables us to identify lines that have repeated parts. 6/17 December 7-10, / 17

7 Simple Examples of. The first line is used in the filter mode. The others are not. grep -n sock grep.txt > sock.out Find all lines in grep.txt that contain the word sock. Output the line number and the line to sock.out grep -i sock L8.tex Do a case-insensitive search. grep -r sock. Recursively search for lines containing sock in all the files in the current directory and all subdirectories. grep sock clientmary.c client.c You can list more than one file to search. grep sock../*.c You can use.. or specify files with a glob pattern, which will be expanded by the shell. 7/17 December 7-10, / 17

8 Quotes Use quotes if there are spaces embedded in your search pattern: grep sock L8.tex Find all lines containing the word sock (note the trailing space) in the file L8.tex grep "sock " L8.tex Double quotes are like using single quotes except for shell expansion rules. grep "sock s " L8.tex Two styles of quotes allow you to include a quote as part of a string. 8/17 December 7-10, / 17

9 Escape Characters and Options Sometimes you need escape characters: grep big\ cat L8.tex You can use backslash-space to put a space into a pattern. grep L8\. clientmary.c If you need to search for a period, the period must be preceded by a backslash (escape). Use -e when searching for multiple patterns. grep -v -e foo -e bar myfile.txt Use -v to search for lines that do NOT contain a pattern. This would find lines that contain foo but not bar grep -e foo -v -e bar myfile.txt 9/17 December 7-10, / 17

10 Alternatives and Sequences Sometimes you want to search for any character from a given set. To do this, put the set inside square brackets and quote the whole pattern. grep [qxz] test.txt Find lines that contain one of these three uncommon letters. grep -i [qxz][aeiou] test.txt Search for an unusual letter followed by a vowel. grep [v-z] test.txt Use a minus sign to define a range (in the local char set) of chars. grep -i [a-z][a-z] test.txt Search for two letter words preceded and followed by spaces. grep -i [a-z][0-9] * Case insensitive search for a letter followed by a digit. 10/17 December 7-10, / 17

11 Wild Cards Using *, you can search for zero, one, or more copies of anything. Any pattern that contains an * must be quoted; otherwise, the shell will think it is a glob pattern and try to expand it. grep c.t$ *.txt This matches cat, cot, and cut at the end of a line. grep c.*t *.txt This matches ct, cat, caat, caaat, cot, coat, coats,.... grep ^c*t *.txt This matches any line that starts with zero or more c s followed by a t. grep ac*t *.txt This matches at, act, action, acct, acctive, acccctoo, /17 December 7-10, / 17

12 Using find Using find Basic find Simple Examples of Using find Examples of find-combinations 12/17 December 7-10, / 17

13 Using find Basic find grep looks at the contents of a file; find looks at the meta data. The find command will do a recursive traversal of a directory, looking for files that match some criterion. find. -name "*.log" Search this directory for files with the extension.log. Display the relative pathname. find. -name "*.log" -ls Display the full directory listing, not just the pathname. find. -name "*.log" > logfiles.txt Redirect the listing to a file. Often used when you need to edit and format the information. find /sysprog/lecture -name "*.tex" You can use a pathname, relative or absolute. find.. -name "L*.pdf" You can search using a pattern. 13/17 December 7-10, / 17

14 Using find Examples of Using find Varied criteria can be used. find -empty Search for empty files and directories starting from my home directory. find -type f List all my files (not links or directories or... ). find -type l Search for symbolic (soft) links. find -type d Search for directories. find. -mtime -1 -type f List all the ordinary files that have been changed in the last day. find. -mmin -10 -type f List all the files that have been changed in the last 10 minutes. 14/17 December 7-10, / 17

15 Using find Using find Starting with my home directory, find all files bigger than 100*1024. find -type f -size +100k -ls Find all really big files. I tried +100M also. find -type f -size +1G -ls List all files that have more than one link. Use long form of listing. find. -links +1 -type f -ls Find all file names that link to the same Inode as the named file: find. -samefile./stu_sysprog/p2/test1/filec.dat 15/17 December 7-10, / 17

16 Using find Using find-combinations Basically, find() will traverse your file directory looking for anything you can figure out how to describe, then do anything you can find a command to do. Find all my log files and write their names to a file. find. -name "*.log" > logfiles.txt Search for files that end in build/debug or build/debug find. -iregex ".*build/[dd]ebug" Display the number of files in this directory tree, and the number of words and bytes in those file names. find. -type f wc Output: Find all the files in this directory tree with spaces in the path name: find. -type f grep " " 16/17 December 7-10, / 17

17 Using find Using find-combinations By combining find with other commands, you can do some fancy things: Starting here, find all files with names that ends in.tex. Then grep each of those files for the word socket. grep -i "socket" find. -type f -name "*.tex" Find all my directories that can be written by anybody find -perm +22 -type d Find all my files that can be written by anybody. For each one, use exec to call the chmod command to turn off the other write permission. The {} and + tell exec to run chmod on all the files identified by the find. find -perm +o=w -type f -exec chmod o-w {} + 17/17 December 7-10, / 17

UNIX files searching, and other interrogation techniques

UNIX files searching, and other interrogation techniques UNIX files searching, and other interrogation techniques Ways to examine the contents of files. How to find files when you don't know how their exact location. Ways of searching files for text patterns.

More information

ITST Searching, Extracting & Archiving Data

ITST Searching, Extracting & Archiving Data ITST 1136 - Searching, Extracting & Archiving Data Name: Step 1 Sign into a Pi UN = pi PW = raspberry Step 2 - Grep - One of the most useful and versatile commands in a Linux terminal environment is the

More information

File Commands. Objectives

File Commands. Objectives File Commands Chapter 2 SYS-ED/Computer Education Techniques, Inc. 2: 1 Objectives You will learn: Purpose and function of file commands. Interrelated usage of commands. SYS-ED/Computer Education Techniques,

More information

PESIT Bangalore South Campus

PESIT Bangalore South Campus INTERNAL ASSESSMENT TEST - 2 Date : 20/09/2016 Max Marks : 0 Subject & Code : Unix Shell Programming (15CS36) Section : 3 rd Sem ISE/CSE Name of faculty : Prof Ajoy Time : 11:30am to 1:00pm SOLUTIONS 1

More information

Lecture 3 Tonight we dine in shell. Hands-On Unix System Administration DeCal

Lecture 3 Tonight we dine in shell. Hands-On Unix System Administration DeCal Lecture 3 Tonight we dine in shell Hands-On Unix System Administration DeCal 2012-09-17 Review $1, $2,...; $@, $*, $#, $0, $? environment variables env, export $HOME, $PATH $PS1=n\[\e[0;31m\]\u\[\e[m\]@\[\e[1;34m\]\w

More information

CS 307: UNIX PROGRAMMING ENVIRONMENT FIND COMMAND

CS 307: UNIX PROGRAMMING ENVIRONMENT FIND COMMAND CS 307: UNIX PROGRAMMING ENVIRONMENT FIND COMMAND Prof. Michael J. Reale Fall 2014 Finding Files in a Directory Tree Suppose you want to find a file with a certain filename (or with a filename matching

More information

Table of contents. Our goal. Notes. Notes. Notes. Summer June 29, Our goal is to see how we can use Unix as a tool for developing programs

Table of contents. Our goal. Notes. Notes. Notes. Summer June 29, Our goal is to see how we can use Unix as a tool for developing programs Summer 2010 Department of Computer Science and Engineering York University Toronto June 29, 2010 1 / 36 Table of contents 1 2 3 4 2 / 36 Our goal Our goal is to see how we can use Unix as a tool for developing

More information

CSCI 2132 Software Development. Lecture 7: Wildcards and Regular Expressions

CSCI 2132 Software Development. Lecture 7: Wildcards and Regular Expressions CSCI 2132 Software Development Lecture 7: Wildcards and Regular Expressions Instructor: Vlado Keselj Faculty of Computer Science Dalhousie University 20-Sep-2017 (7) CSCI 2132 1 Previous Lecture Pipes

More information

find Command as Admin Security Tool

find Command as Admin Security Tool find Command as Admin Security Tool Dr. Bill Mihajlovic INCS-620 Operating Systems Security find Command find command searches for the file or files that meet certain condition. like: Certain name Certain

More information

Wildcards and Regular Expressions

Wildcards and Regular Expressions CSCI 2132: Software Development Wildcards and Regular Expressions Norbert Zeh Faculty of Computer Science Dalhousie University Winter 2019 Searching Problem: Find all files whose names match a certain

More information

5/20/2007. Touring Essential Programs

5/20/2007. Touring Essential Programs Touring Essential Programs Employing fundamental utilities. Managing input and output. Using special characters in the command-line. Managing user environment. Surveying elements of a functioning system.

More information

Module 8 Pipes, Redirection and REGEX

Module 8 Pipes, Redirection and REGEX Module 8 Pipes, Redirection and REGEX Exam Objective 3.2 Searching and Extracting Data from Files Objective Summary Piping and redirection Partial POSIX Command Line and Redirection Command Line Pipes

More information

CST Lab #5. Student Name: Student Number: Lab section:

CST Lab #5. Student Name: Student Number: Lab section: CST8177 - Lab #5 Student Name: Student Number: Lab section: Working with Regular Expressions (aka regex or RE) In-Lab Demo - List all the non-user accounts in /etc/passwd that use /sbin as their home directory.

More information

Basic Linux (Bash) Commands

Basic Linux (Bash) Commands Basic Linux (Bash) Commands Hint: Run commands in the emacs shell (emacs -nw, then M-x shell) instead of the terminal. It eases searching for and revising commands and navigating and copying-and-pasting

More information

Basics. I think that the later is better.

Basics.  I think that the later is better. Basics Before we take up shell scripting, let s review some of the basic features and syntax of the shell, specifically the major shells in the sh lineage. Command Editing If you like vi, put your shell

More information

C Shell Tutorial. Section 1

C Shell Tutorial. Section 1 C Shell Tutorial Goals: Section 1 Learn how to write a simple shell script and how to run it. Learn how to use local and global variables. About CSH The Barkley Unix C shell was originally written with

More information

Files (review) and Regular Expressions. Todd Kelley CST8207 Todd Kelley 1

Files (review) and Regular Expressions. Todd Kelley CST8207 Todd Kelley 1 Files (review) and Regular Expressions Todd Kelley kelleyt@algonquincollege.com CST8207 Todd Kelley 1 midterms (Feb 11 and April 1) Files and Permissions Regular Expressions 2 Sobel, Chapter 6 160_pathnames.html

More information

18-Sep CSCI 2132 Software Development Lecture 6: Links and Inodes. Faculty of Computer Science, Dalhousie University. Lecture 6 p.

18-Sep CSCI 2132 Software Development Lecture 6: Links and Inodes. Faculty of Computer Science, Dalhousie University. Lecture 6 p. Lecture 6 p.1 Faculty of Computer Science, Dalhousie University CSCI 2132 Software Development Lecture 6: Links and s 18-Sep-2017 Location: Goldberg CS 127 Time: 14:35 15:25 Instructor: Vlado Keselj Previous

More information

Chapter 1 - Introduction. September 8, 2016

Chapter 1 - Introduction. September 8, 2016 Chapter 1 - Introduction September 8, 2016 Introduction Overview of Linux/Unix Shells Commands: built-in, aliases, program invocations, alternation and iteration Finding more information: man, info Help

More information

Pattern Matching. An Introduction to File Globs and Regular Expressions

Pattern Matching. An Introduction to File Globs and Regular Expressions Pattern Matching An Introduction to File Globs and Regular Expressions Copyright 2006 2009 Stewart Weiss The danger that lies ahead Much to your disadvantage, there are two different forms of patterns

More information

Pattern Matching. An Introduction to File Globs and Regular Expressions. Adapted from Practical Unix and Programming Hunter College

Pattern Matching. An Introduction to File Globs and Regular Expressions. Adapted from Practical Unix and Programming Hunter College Pattern Matching An Introduction to File Globs and Regular Expressions Adapted from Practical Unix and Programming Hunter College Copyright 2006 2009 Stewart Weiss The danger that lies ahead Much to your

More information

5/8/2012. Exploring Utilities Chapter 5

5/8/2012. Exploring Utilities Chapter 5 Exploring Utilities Chapter 5 Examining the contents of files. Working with the cut and paste feature. Formatting output with the column utility. Searching for lines containing a target string with grep.

More information

A Brief Introduction to the Linux Shell for Data Science

A Brief Introduction to the Linux Shell for Data Science A Brief Introduction to the Linux Shell for Data Science Aris Anagnostopoulos 1 Introduction Here we will see a brief introduction of the Linux command line or shell as it is called. Linux is a Unix-like

More information

Files and Directories

Files and Directories CSCI 2132: Software Development Files and Directories Norbert Zeh Faculty of Computer Science Dalhousie University Winter 2019 Files and Directories Much of the operation of Unix and programs running on

More information

CSE 374 Midterm Exam 2/6/17 Sample Solution. Question 1. (12 points, 4 each) Suppose we have the following files and directories:

CSE 374 Midterm Exam 2/6/17 Sample Solution. Question 1. (12 points, 4 each) Suppose we have the following files and directories: Question 1. (12 points, 4 each) Suppose we have the following files and directories: $ pwd /home/user $ ls -l -rw-r--r-- 1 user uw 10 Feb 4 15:49 combine.sh drwxr-xr-x 2 user uw 2 Feb 4 15:51 hws -rw-r--r--

More information

Advanced training. Linux components Command shell. LiLux a.s.b.l.

Advanced training. Linux components Command shell. LiLux a.s.b.l. Advanced training Linux components Command shell LiLux a.s.b.l. alexw@linux.lu Kernel Interface between devices and hardware Monolithic kernel Micro kernel Supports dynamics loading of modules Support

More information

I/O and Shell Scripting

I/O and Shell Scripting I/O and Shell Scripting File Descriptors Redirecting Standard Error Shell Scripts Making a Shell Script Executable Specifying Which Shell Will Run a Script Comments in Shell Scripts File Descriptors Resources

More information

Regular Expressions 1

Regular Expressions 1 Regular Expressions 1 Basic Regular Expression Examples Extended Regular Expressions Extended Regular Expression Examples 2 phone number 3 digits, dash, 4 digits [[:digit:]][[:digit:]][[:digit:]]-[[:digit:]][[:digit:]][[:digit:]][[:digit:]]

More information

CSC UNIX System, Spring 2015

CSC UNIX System, Spring 2015 CSC 352 - UNIX System, Spring 2015 Study guide for the CSC352 midterm exam (20% of grade). Dr. Dale E. Parson, http://faculty.kutztown.edu/parson We will have a midterm on March 19 on material we have

More information

CS Unix Tools. Fall 2010 Lecture 5. Hussam Abu-Libdeh based on slides by David Slater. September 17, 2010

CS Unix Tools. Fall 2010 Lecture 5. Hussam Abu-Libdeh based on slides by David Slater. September 17, 2010 Fall 2010 Lecture 5 Hussam Abu-Libdeh based on slides by David Slater September 17, 2010 Reasons to use Unix Reason #42 to use Unix: Wizardry Mastery of Unix makes you a wizard need proof? here is the

More information

Unix as a Platform Exercises + Solutions. Course Code: OS 01 UNXPLAT

Unix as a Platform Exercises + Solutions. Course Code: OS 01 UNXPLAT Unix as a Platform Exercises + Solutions Course Code: OS 01 UNXPLAT Working with Unix Most if not all of these will require some investigation in the man pages. That's the idea, to get them used to looking

More information

CS Unix Tools & Scripting Lecture 7 Working with Stream

CS Unix Tools & Scripting Lecture 7 Working with Stream CS2043 - Unix Tools & Scripting Lecture 7 Working with Streams Spring 2015 1 February 4, 2015 1 based on slides by Hussam Abu-Libdeh, Bruno Abrahao and David Slater over the years Announcements Course

More information

Introduction to Unix CHAPTER 6. File Systems. Permissions

Introduction to Unix CHAPTER 6. File Systems. Permissions CHAPTER 6 Introduction to Unix The Unix operating system is an incredibly powerful and complex system that is ideal for running a distributed system such as ours, particularly since we use computers primarily

More information

Introduction To. Barry Grant

Introduction To. Barry Grant Introduction To Barry Grant bjgrant@umich.edu http://thegrantlab.org Working with Unix How do we actually use Unix? Inspecting text files less - visualize a text file: use arrow keys page down/page up

More information

A Brief Introduction to Unix

A Brief Introduction to Unix A Brief Introduction to Unix Sean Barag Drexel University March 30, 2011 Sean Barag (Drexel University) CS 265 - A Brief Introduction to Unix March 30, 2011 1 / 17 Outline 1 Directories

More information

Introduc)on to Linux Session 2 Files/Filesystems/Data. Pete Ruprecht Research Compu)ng Group University of Colorado Boulder

Introduc)on to Linux Session 2 Files/Filesystems/Data. Pete Ruprecht Research Compu)ng Group University of Colorado Boulder Introduc)on to Linux Session 2 Files/Filesystems/Data Pete Ruprecht Research Compu)ng Group University of Colorado Boulder www.rc.colorado.edu Outline LeHover from last week redirec)on Filesystem layout

More information

Regular Expressions. Todd Kelley CST8207 Todd Kelley 1

Regular Expressions. Todd Kelley CST8207 Todd Kelley 1 Regular Expressions Todd Kelley kelleyt@algonquincollege.com CST8207 Todd Kelley 1 POSIX character classes Some Regular Expression gotchas Regular Expression Resources Assignment 3 on Regular Expressions

More information

When talking about how to launch commands and other things that is to be typed into the terminal, the following syntax is used:

When talking about how to launch commands and other things that is to be typed into the terminal, the following syntax is used: Linux Tutorial How to read the examples When talking about how to launch commands and other things that is to be typed into the terminal, the following syntax is used: $ application file.txt

More information

Topic 4: Grep, Find & Sed

Topic 4: Grep, Find & Sed Topic 4: Grep, Find & Sed grep: a tool for searching for strings within files find: a tool for examining a directory tree sed: a tool for "batch editing" Associated topic: regular expressions 1 Motivation

More information

Part III. Shell Config. Tobias Neckel: Scripting with Bash and Python Compact Max-Planck, February 16-26,

Part III. Shell Config. Tobias Neckel: Scripting with Bash and Python Compact Max-Planck, February 16-26, Part III Shell Config Compact Course @ Max-Planck, February 16-26, 2015 33 Special Directories. current directory.. parent directory ~ own home directory ~user home directory of user ~- previous directory

More information

Grep and Shell Programming

Grep and Shell Programming Grep and Shell Programming Comp-206 : Introduction to Software Systems Lecture 7 Alexandre Denault Computer Science McGill University Fall 2006 Teacher's Assistants Michael Hawker Monday, 14h30 to 16h30

More information

ADVANCED LINUX SYSTEM ADMINISTRATION

ADVANCED LINUX SYSTEM ADMINISTRATION Lab Assignment 1 Corresponding to Topic 2, The Command Line L1 Main goals To get used to the command line. To gain basic skills with the system shell. To understand some of the basic tools of system administration.

More information

Introduction to UNIX. Introduction. Processes. ps command. The File System. Directory Structure. UNIX is an operating system (OS).

Introduction to UNIX. Introduction. Processes. ps command. The File System. Directory Structure. UNIX is an operating system (OS). Introduction Introduction to UNIX CSE 2031 Fall 2012 UNIX is an operating system (OS). Our goals: Learn how to use UNIX OS. Use UNIX tools for developing programs/ software, specifically shell programming.

More information

Introduction to UNIX. CSE 2031 Fall November 5, 2012

Introduction to UNIX. CSE 2031 Fall November 5, 2012 Introduction to UNIX CSE 2031 Fall 2012 November 5, 2012 Introduction UNIX is an operating system (OS). Our goals: Learn how to use UNIX OS. Use UNIX tools for developing programs/ software, specifically

More information

Unix/Linux Basics. Cpt S 223, Fall 2007 Copyright: Washington State University

Unix/Linux Basics. Cpt S 223, Fall 2007 Copyright: Washington State University Unix/Linux Basics 1 Some basics to remember Everything is case sensitive Eg., you can have two different files of the same name but different case in the same folder Console-driven (same as terminal )

More information

5/8/2012. Creating and Changing Directories Chapter 7

5/8/2012. Creating and Changing Directories Chapter 7 Creating and Changing Directories Chapter 7 Types of files File systems concepts Using directories to create order. Managing files in directories. Using pathnames to manage files in directories. Managing

More information

Introduction to Unix

Introduction to Unix Introduction to Unix Part 1: Navigating directories First we download the directory called "Fisher" from Carmen. This directory contains a sample from the Fisher corpus. The Fisher corpus is a collection

More information

Physics REU Unix Tutorial

Physics REU Unix Tutorial Physics REU Unix Tutorial What is unix? Unix is an operating system. In simple terms, its the set of programs that makes a computer work. It can be broken down into three parts. (1) kernel: The component

More information

Introduction to Unix

Introduction to Unix Part 2: Looking into a file Introduction to Unix Now we want to see how the files are structured. Let's look into one. more $ more fe_03_06596.txt 0.59 1.92 A-f: hello 1.96 2.97 B-m: (( hello )) 2.95 3.98

More information

Unix Internal Assessment-2 solution. Ans:There are two ways of starting a job in the background with the shell s & operator and the nohup command.

Unix Internal Assessment-2 solution. Ans:There are two ways of starting a job in the background with the shell s & operator and the nohup command. Unix Internal Assessment-2 solution 1 a.explain the mechanism of process creation. Ans: There are three distinct phases in the creation of a process and uses three important system calls viz., fork, exec,

More information

CSCI 2132 Software Development. Lecture 4: Files and Directories

CSCI 2132 Software Development. Lecture 4: Files and Directories CSCI 2132 Software Development Lecture 4: Files and Directories Instructor: Vlado Keselj Faculty of Computer Science Dalhousie University 12-Sep-2018 (4) CSCI 2132 1 Previous Lecture Some hardware concepts

More information

Scripting Languages Course 1. Diana Trandabăț

Scripting Languages Course 1. Diana Trandabăț Scripting Languages Course 1 Diana Trandabăț Master in Computational Linguistics - 1 st year 2017-2018 Today s lecture Introduction to scripting languages What is a script? What is a scripting language

More information

CSE 374 Midterm Exam Sample Solution 2/6/12

CSE 374 Midterm Exam Sample Solution 2/6/12 Question 1. (12 points) Suppose we have the following subdirectory structure inside the current directory: docs docs/friends docs/friends/birthdays.txt docs/friends/messages.txt docs/cse374 docs/cse374/notes.txt

More information

Lecture 18 Regular Expressions

Lecture 18 Regular Expressions Lecture 18 Regular Expressions In this lecture Background Text processing languages Pattern searches with grep Formal Languages and regular expressions Finite State Machines Regular Expression Grammer

More information

Essentials for Scientific Computing: Stream editing with sed and awk

Essentials for Scientific Computing: Stream editing with sed and awk Essentials for Scientific Computing: Stream editing with sed and awk Ershaad Ahamed TUE-CMS, JNCASR May 2012 1 Stream Editing sed and awk are stream processing commands. What this means is that they are

More information

Essentials for Scientific Computing: Bash Shell Scripting Day 3

Essentials for Scientific Computing: Bash Shell Scripting Day 3 Essentials for Scientific Computing: Bash Shell Scripting Day 3 Ershaad Ahamed TUE-CMS, JNCASR May 2012 1 Introduction In the previous sessions, you have been using basic commands in the shell. The bash

More information

Answers to Even-numbered Exercises

Answers to Even-numbered Exercises 11 Answers to Even-numbered Exercises 1. ewrite t propriate actions xists and the user does not have write permission to the le. Verify that the modied script works. 2. The special parameter "$@" is referenced

More information

Regular Expressions. Todd Kelley CST8207 Todd Kelley 1

Regular Expressions. Todd Kelley CST8207 Todd Kelley 1 Regular Expressions Todd Kelley kelleyt@algonquincollege.com CST8207 Todd Kelley 1 Our standard.bashrc and.bash_profile (or.profile) Our standard script header Regular Expressions 2 [ -z "${PS1-}" ] &&

More information

http://xkcd.com/208/ 1. Review of pipes 2. Regular expressions 3. sed 4. Editing Files 5. Shell loops 6. Shell scripts cat seqs.fa >0! TGCAGGTATATCTATTAGCAGGTTTAATTTTGCCTGCACTTGGTTGGGTACATTATTTTAAGTGTATTTGACAAG!

More information

Vi & Shell Scripting

Vi & Shell Scripting Vi & Shell Scripting Comp-206 : Introduction to Week 3 Joseph Vybihal Computer Science McGill University Announcements Sina Meraji's office hours Trottier 3rd floor open area Tuesday 1:30 2:30 PM Thursday

More information

AC109/AT109 UNIX & SHELL PROGRAMMING DEC 2014

AC109/AT109 UNIX & SHELL PROGRAMMING DEC 2014 Q.2 a. Explain the principal components: Kernel and Shell, of the UNIX operating system. Refer Page No. 22 from Textbook b. Explain absolute and relative pathnames with the help of examples. Refer Page

More information

FILESYSTEMS. Mmmm crunchy

FILESYSTEMS. Mmmm crunchy FILESYSTEMS Mmmm crunchy PURPOSE So all this data... How to organize? Whose job? Filesystems! PERMISSIONS Linux supports 3 main types of access on a file: read: View the contents write: Modify the contents

More information

Motivation (Scenarios) Topic 4: Grep, Find & Sed. Displaying File Names. grep

Motivation (Scenarios) Topic 4: Grep, Find & Sed. Displaying File Names. grep Topic 4: Grep, Find & Sed grep: a tool for searching for strings within files find: a tool for examining a directory tree sed: a tool for "batch editing" Associated topic: regular expressions Motivation

More information

Bash Script. CIRC Summer School 2015 Baowei Liu

Bash Script. CIRC Summer School 2015 Baowei Liu Bash Script CIRC Summer School 2015 Baowei Liu Filename Expansion / Globbing Expanding filenames containing special characters Wild cards *?, not include... Square brackets [set]: - Special characters:!

More information

Unix as a Platform Exercises. Course Code: OS-01-UNXPLAT

Unix as a Platform Exercises. Course Code: OS-01-UNXPLAT Unix as a Platform Exercises Course Code: OS-01-UNXPLAT Working with Unix 1. Use the on-line manual page to determine the option for cat, which causes nonprintable characters to be displayed. Run the command

More information

Unix Tools / Command Line

Unix Tools / Command Line Unix Tools / Command Line An Intro 1 Basic Commands / Utilities I expect you already know most of these: ls list directories common options: -l, -F, -a mkdir, rmdir make or remove a directory mv move/rename

More information

Useful Unix Commands Cheat Sheet

Useful Unix Commands Cheat Sheet Useful Unix Commands Cheat Sheet The Chinese University of Hong Kong SIGSC Training (Fall 2016) FILE AND DIRECTORY pwd Return path to current directory. ls List directories and files here. ls dir List

More information

Essential Linux Shell Commands

Essential Linux Shell Commands Essential Linux Shell Commands Special Characters Quoting and Escaping Change Directory Show Current Directory List Directory Contents Working with Files Working with Directories Special Characters There

More information

Appendix A GLOSSARY. SYS-ED/ Computer Education Techniques, Inc.

Appendix A GLOSSARY. SYS-ED/ Computer Education Techniques, Inc. Appendix A GLOSSARY SYS-ED/ Computer Education Techniques, Inc. $# Number of arguments passed to a script. $@ Holds the arguments; unlike $* it has the capability for separating the arguments. $* Holds

More information

Crash Course in Unix. For more info check out the Unix man pages -orhttp://www.cs.rpi.edu/~hollingd/unix. -or- Unix in a Nutshell (an O Reilly book).

Crash Course in Unix. For more info check out the Unix man pages -orhttp://www.cs.rpi.edu/~hollingd/unix. -or- Unix in a Nutshell (an O Reilly book). Crash Course in Unix For more info check out the Unix man pages -orhttp://www.cs.rpi.edu/~hollingd/unix -or- Unix in a Nutshell (an O Reilly book). 1 Unix Accounts To access a Unix system you need to have

More information

AOS Linux Tutorial. Introduction to Linux. Michael Havas Dept. of Atmospheric and Oceanic Sciences McGill University. September 15, 2011

AOS Linux Tutorial. Introduction to Linux. Michael Havas Dept. of Atmospheric and Oceanic Sciences McGill University. September 15, 2011 AOS Linux Tutorial Introduction to Linux Michael Havas Dept. of Atmospheric and Oceanic Sciences McGill University September 15, 2011 Outline 1 Introduction to Linux Benefits of Linux What Exactly is Linux?

More information

Unix basics exercise MBV-INFX410

Unix basics exercise MBV-INFX410 Unix basics exercise MBV-INFX410 In order to start this exercise, you need to be logged in on a UNIX computer with a terminal window open on your computer. It is best if you are logged in on freebee.abel.uio.no.

More information

Practical 4. Linux Commands: Working with Directories

Practical 4. Linux Commands: Working with Directories Practical 4 Linux Commands: Working with Directories 1. pwd: pwd stands for Print Working Directory. As the name states, command pwd prints the current working directory or simply the directory user is,

More information

Digital Humanities. Tutorial Regular Expressions. March 10, 2014

Digital Humanities. Tutorial Regular Expressions. March 10, 2014 Digital Humanities Tutorial Regular Expressions March 10, 2014 1 Introduction In this tutorial we will look at a powerful technique, called regular expressions, to search for specific patterns in corpora.

More information

Multiple Choice - 58 Questions - 10 of 15%

Multiple Choice - 58 Questions - 10 of 15% CST 8129 Ian Allen Fall 2005-1- 110 minutes Evaluation: Part I - 58 M/C Questions Name: Important Instructions 1. Read all the instructions and both sides (back and front) of all pages. 2. Manage your

More information

Operating Systems, Unix Files and Commands SEEM

Operating Systems, Unix Files and Commands SEEM Operating Systems, Unix Files and Commands SEEM 3460 1 Major Components of Operating Systems (OS) Process management Resource management CPU Memory Device File system Bootstrapping SEEM 3460 2 Programs

More information

Introduction to UNIX. Logging in. Basic System Architecture 10/7/10. most systems have graphical login on Linux machines

Introduction to UNIX. Logging in. Basic System Architecture 10/7/10. most systems have graphical login on Linux machines Introduction to UNIX Logging in Basic system architecture Getting help Intro to shell (tcsh) Basic UNIX File Maintenance Intro to emacs I/O Redirection Shell scripts Logging in most systems have graphical

More information

http://xkcd.com/208/ 1. Review of pipes 2. Regular expressions 3. sed 4. awk 5. Editing Files 6. Shell loops 7. Shell scripts cat seqs.fa >0! TGCAGGTATATCTATTAGCAGGTTTAATTTTGCCTGCACTTGGTTGGGTACATTATTTTAAGTGTATTTGACAAG!

More information

DATA 301 Introduction to Data Analytics Command Line. Dr. Ramon Lawrence University of British Columbia Okanagan

DATA 301 Introduction to Data Analytics Command Line. Dr. Ramon Lawrence University of British Columbia Okanagan DATA 301 Introduction to Data Analytics Command Line Dr. Ramon Lawrence University of British Columbia Okanagan ramon.lawrence@ubc.ca Why learn the Command Line? The command line is the text interface

More information

Why learn the Command Line? The command line is the text interface to the computer. DATA 301 Introduction to Data Analytics Command Line

Why learn the Command Line? The command line is the text interface to the computer. DATA 301 Introduction to Data Analytics Command Line DATA 301 Introduction to Data Analytics Command Line Why learn the Command Line? The command line is the text interface to the computer. DATA 301: Data Analytics (2) Understanding the command line allows

More information

COMP2100/2500 Lecture 16: Shell Programming I

COMP2100/2500 Lecture 16: Shell Programming I [ANU] [DCS] [COMP2100/2500] [Description] [Schedule] [Lectures] [Labs] [Homework] [Assignments] [COMP2500] [Assessment] [PSP] [Java] [Reading] [Help] COMP2100/2500 Lecture 16: Shell Programming I Summary

More information

Lec 1 add-on: Linux Intro

Lec 1 add-on: Linux Intro Lec 1 add-on: Linux Intro Readings: - Unix Power Tools, Powers et al., O Reilly - Linux in a Nutshell, Siever et al., O Reilly Summary: - Linux File System - Users and Groups - Shell - Text Editors - Misc

More information

Traditional Access Permissions

Traditional Access Permissions Traditional Access Permissions There are three types of users: - owner - group - other (aka world) A user may attempt to access an ordinary file in three ways: - read from - write to - execute Use ls lto

More information

Regular Expressions Explained

Regular Expressions Explained Found at: http://publish.ez.no/article/articleprint/11/ Regular Expressions Explained Author: Jan Borsodi Publishing date: 30.10.2000 18:02 This article will give you an introduction to the world of regular

More information

Week Overview. Simple filter commands: head, tail, cut, sort, tr, wc grep utility stdin, stdout, stderr Redirection and piping /dev/null file

Week Overview. Simple filter commands: head, tail, cut, sort, tr, wc grep utility stdin, stdout, stderr Redirection and piping /dev/null file ULI101 Week 05 Week Overview Simple filter commands: head, tail, cut, sort, tr, wc grep utility stdin, stdout, stderr Redirection and piping /dev/null file head and tail commands These commands display

More information

While Statement Examples. While Statement (35.15) Until Statement (35.15) Until Statement Example

While Statement Examples. While Statement (35.15) Until Statement (35.15) Until Statement Example While Statement (35.15) General form. The commands in the loop are performed while the condition is true. while condition one-or-more-commands While Statement Examples # process commands until a stop is

More information

Files

Files http://www.cs.fsu.edu/~langley/cop3353-2013-1/reveal.js-2013-02-11/02.html?print-pdf 02/11/2013 10:55 AM Files A normal "flat" file is a collection of information. It's usually stored somewhere reasonably

More information

GMI-Cmd.exe Reference Manual GMI Command Utility General Management Interface Foundation

GMI-Cmd.exe Reference Manual GMI Command Utility General Management Interface Foundation GMI-Cmd.exe Reference Manual GMI Command Utility General Management Interface Foundation http://www.gmi-foundation.org Program Description The "GMI-Cmd.exe" program is a standard part of the GMI program

More information

Shell Programming Overview

Shell Programming Overview Overview Shell programming is a way of taking several command line instructions that you would use in a Unix command prompt and incorporating them into one program. There are many versions of Unix. Some

More information

Chapter-3. Introduction to Unix: Fundamental Commands

Chapter-3. Introduction to Unix: Fundamental Commands Chapter-3 Introduction to Unix: Fundamental Commands What You Will Learn The fundamental commands of the Unix operating system. Everything told for Unix here is applicable to the Linux operating system

More information

User Commands find ( 1 )

User Commands find ( 1 ) NAME find find files SYNOPSIS /usr/bin/find path... expression /usr/xpg4/bin/find path... expression DESCRIPTION The find utility recursively descends the directory hierarchy for each path seeking files

More information

Traditional Access Permissions

Traditional Access Permissions Traditional Access Permissions There are three types of users: - owner - group - other (aka world) A user may attempt to access an ordinary file in three ways: - read from - write to - execute Use ls l

More information

Computer Systems and Architecture

Computer Systems and Architecture Computer Systems and Architecture Stephen Pauwels Regular Expressions Academic Year 2018-2019 Outline What is a Regular Expression? Tools Anchors, Character sets and Modifiers Advanced Regular Expressions

More information

Examples: Directory pathname: File pathname: /home/username/ics124/assignments/ /home/username/ops224/assignments/assn1.txt

Examples: Directory pathname: File pathname: /home/username/ics124/assignments/ /home/username/ops224/assignments/assn1.txt ULI101 Week 03 Week Overview Absolute and relative pathnames File name expansion Shell basics Command execution in detail Recalling and editing previous commands Quoting Pathnames A pathname is a list

More information

Perl and R Scripting for Biologists

Perl and R Scripting for Biologists Perl and R Scripting for Biologists Lukas Mueller PLBR 4092 Course overview Linux basics (today) Linux advanced (Aure, next week) Why Linux? Free open source operating system based on UNIX specifications

More information

Filesystem and common commands

Filesystem and common commands Filesystem and common commands Unix computing basics Campus-Booster ID : **XXXXX www.supinfo.com Copyright SUPINFO. All rights reserved Filesystem and common commands Your trainer Presenter s Name Title:

More information

COMS 6100 Class Notes 3

COMS 6100 Class Notes 3 COMS 6100 Class Notes 3 Daniel Solus September 1, 2016 1 General Remarks The class was split into two main sections. We finished our introduction to Linux commands by reviewing Linux commands I and II

More information

First of all, these notes will cover only a small subset of the available commands and utilities, and will cover most of those in a shallow fashion.

First of all, these notes will cover only a small subset of the available commands and utilities, and will cover most of those in a shallow fashion. Warnings 1 First of all, these notes will cover only a small subset of the available commands and utilities, and will cover most of those in a shallow fashion. Read the relevant material in Sobell! If

More information

A Big Step. Shell Scripts, I/O Redirection, Ownership and Permission Concepts, and Binary Numbers

A Big Step. Shell Scripts, I/O Redirection, Ownership and Permission Concepts, and Binary Numbers A Big Step Shell Scripts, I/O Redirection, Ownership and Permission Concepts, and Binary Numbers Copyright 2006 2009 Stewart Weiss What a shell really does Here is the scoop on shells. A shell is a program

More information

Computer Systems and Architecture

Computer Systems and Architecture Computer Systems and Architecture Regular Expressions Bart Meyers University of Antwerp August 29, 2012 Outline What? Tools Anchors, character sets and modifiers Advanced Regular expressions Exercises

More information