Awk & Regular Expressions
|
|
- Osborn Simpson
- 6 years ago
- Views:
Transcription
1 Awk & Regular Expressions CSCI-620 Dr. Bill Mihajlovic awk Text Editor awk, named after its developers Aho, Weinberger, and Kernighan. awk is UNIX utility. The awk command uses awk program to scan text files or standard input to: display specific data, change data format, and add text to existing data. 1
2 Awk awk is a pattern scanning and processing language. awk searches one or more input files to see if they contain lines that match specified patterns and then perform associated actions, such as writing the line to the standard output or incrementing a counter each time it finds a match. awk is a programming language which permits easy manipulation of structured data and the generation of formatted reports. # awk pattern {action} infile Awk Syntax Awk utility may receive instructions as the command line string of text or as a text read form a awk-file: # awk pattern {action} infile The awk utility performs the action on all lines that the pattern selects The pattern selects lines from the input file. Braces nust enclose the action so that awk can differentiate it from the pattern. 2
3 Awk If a program line does not contain a pattern, awk selects all lines in the input file. There are two rules which occur if either a pattern or action is ommited: # awk {action} infile # awk pattern { } infile If the program line does not contain an action, awk copies the selected lines to its standard output (this is usually the display, if you haven't redirected the output to another program or to a file). awk as Programming Language The capabilities of awk extend the idea of text editing into computation, making it possible to perform a variety of data processing tasks, including: analysis, extraction, and reporting of data. These are, indeed, the most common uses of awk. 3
4 Regular Expression RE Searching for: Exactly matching patterns or Closely matching patterns in the text is a common problem. Regular expressions make finding character patterns much easier. Regular expression MetaCharacters allows characters to take on a range of values. Regular Expression RE RE is a character pattern which can match numerous similar strings, because it can contain metacharacters that expand the scope of the search beyond a literal string. Metacharacters are special characters that represent more than their literal meanings. Quoting is the means of turning off the special meaning of metacharacters. 4
5 Editor REs Editor regular expressions are used in editors or shell commands to find character patterns within files. Typical editors and other programs that use meta characters are: vi, ex, edit, view, ed, red, sed,grep, egrep, and expr. Patterns An awk pattern is used to conditionally pass control to an action. An action only executes if its relevant pattern was matched. You can use a regular expression, enclosed within slashes, as a pattern. The ~ operator tests to see if a field or variable matches a regular expression. The!~ operator tests for no match. You can process arithmetic and character relational expressions with the following relational operators. 5
6 Patterns & Operators You can process arithmetic and character relational expressions with the following relational operators. Operator Meaning < less than <= less than or equal to == equal to!= not equal to >= greater than or equal to > greater than awk Operators You can combine any of the patterns using the Boolean operators (OR) or && (AND). The comma is the range operator. If you separate two patterns with a comma on a single awk progam line, awk selects a range of lines beginning with the first line that contains the first pattern. The last line awk selects is the next subsequent line that contains the second pattern. After awk finds the second pattern, it starts the process over by looking for the first pattern again. 6
7 Use of Patern-Matching Metacharacters Matching Filenames within directory files. File Name Generation (FNG) Matching strings within text. Editor regular expression Full regular expressions Awk regular expressions? Metacharacter The? Matches any one character but a dot. A dot must be matched explicitly. # echo? F A f b # 7
8 * Metacharacter The * matches any number of characters but a dot and whie space characters, (blank, tab, newline) # echo * F a f b F11 alexf a1 # ls -x a?* alexf a1 # Character Class Expressio Character class is a group of characters to be matched # ls -x f[abc,+123] fb f+ f1 f2 # ls -x f[!a-za-z0-9] f+ f- # Ranges can be included in character classes by listing by listing 2 characters to define range bounds separated by a dash. 8
9 Basic awk Command Format The basic format of this command consists of the awk command, the instructions enclosed in quotes and curly braces, and the name of the input file. If an input file is not specified, then standard input is used, for example, the keyboard. The following is a basic awk command. The output of the ls l command is piped to awk. For each line received by awk, the print action is executed, which prints the output to the screen. $ ls -l awk {print $0} awk Arguments When awk reads in a line it automatically breaks the line into fields. Each field is assigned a variable name. Spaces or tabs are used as the default delimiter between fields. The variable names assigned to fields are a dollar sign ($) followed by the number of the field, counting from left to right. The variable name $1 represents the contents of Field 1. The variable name $2 represents the contents of Field 2, and so on. The entire line is represented by the variable name $0. 9
10 awk Displays Specific Data To instruct awk to display specific data (for example, the file owner, file size, and file name), the fields variable names are used with the action. # ls -l awk {print $3 $5 $9} user154120dante user1368dante_1 user1176dat user1512dir1 user1512dir2 user1512dir3 user1512dir4 user1235file1 user1105file2 user1218file3 user1137file4 # awk Example Selecting Data fstats file contains the data for players PPG - points per game Consider this example: RPG - rebounds per game APG - assists per game $ cat fstats Smith Jones Davis Johnson Williams $ awk '$2 > 20 { print $1 }' fstats Smith Jones Davis This command says: "Read each line from fstats Johnson and if the second field is more than 20, print the first $ field." This awk command prints the names of the players who have more than 20 PPG. 10
11 awk Example Selecting Data fstats file contains the data for players PPG - points per game is $2 RPG - rebounds per game is $3 APG - assists per game is $4 $ cat fstats Smith Jones Davis Johnson Williams $ awk '($2 > 20 && $3 < 7) { print $1 }' fstats Smith Jones $ This command says "Read each line from fstats; if the second field is more than 20 and the third field is less than 7, print the first field." awk Example Selecting Data This command prints the names of players that begin with 'J' $ cat fstats This command prints the Smith names of players that start with Jones 'J' and have more than 5 Davis Johnson APG. Williams $ awk '$1 ~ /^J/ { print $1 } fstats Jones $ awk '($1 ~ /^J/ && $4 > 5) { print $1 }' fstats Jones $ 11
12 awk Operators BEGIN & END Two unique patterns, BEGIN and END, allow you to execute commands before awk starts its processing and after it finishes. The awk utility executes the actions associated with the BEGIN pattern before, and with the END pattern after, it processes all the files for input. awk Example BEGIN This command prints a header and all of the data. You can do things before or after all lines have been read with BEGIN and END! $ cat fstats Smith Jones Davis Johnson Williams $ awk 'BEGIN { print "Name PPG RPG APG" } { print }' fstats Name PPG RPG APG Smith Jones Davis Johnson Williams $ 12
13 awk Example END This command prints a header and all of the data. You can do things before or after all lines have been read with BEGIN and END! This command says: "Read each line from fstats, print the whole line, and after the last line, print That is all, folks!'." $ cat fstats Smith Jones Davis Johnson Williams $ awk '{ print } END { print "That is all, folks!" }' fstats Smith Jones Davis Johnson Williams That is all, folks! $ awk Example Math - awk even does math! This counts the number of players with more than 20 PPG The above command says "Read each line fstats; if the second field is more than maxppg, make maxppg the second field a make player the first field. After all lines been read, print the line 'player had t most PPG, with maxppg'." $ cat fstats Smith Jones Davis Johnson Williams $ awk '$2 > 20 { total = total + 1 } \ END { print total, "players had more than 20 PPG" }' fstats 4 players had more than 20 PPG $ awk '$2 > maxppg { maxppg = $2; player = $1 } END { print player, "had the most PPG, with", maxppg }' fstats Smith had the most PPG, with 26.4 $ 13
14 awk Example Math Read each line from fstats and add all total scores Running an awk program from a file Finally, you can store awk programs in files, so you do not have to re-enter long awk commands. For instance, if you wanted to run the previous command from a file, you would create a file (let's call it ex1.awk) containing the following: $ cat awk_prog { totrpg = totrpg + $3; count = count + 1 } END { print "Average RPG is", totrpg/count } $ awk -f ex1.awk fstats Average RPG is 7.34 $ This counts of players. This command says "Read each line from player.dat; add the third field to totrpg and add 1 to count. After all lines have been read, print the awk line 'Average Example RPG is totrpg / count'." Math - To calculate the average RPG for all players. awk even does math! $ cat fstats Smith Jones Davis Johnson Williams $ awk '$2 > 20 { total = total + 1 } \ END { print total, "players had more than 20 PPG" }' fstats 4 players had more than 20 PPG $ awk '{ totrpg = totrpg + $3; count = count + 1 } END { print "Average RPG is", totrpg/count }' player.dat Average RPG is 7.34 $ 14
15 Homework Homework Edit and save file fstats. Add two more lines with your name or the name of your fiend s name. Repeat all examples shown in the slide presentation. Enclose your screen shots. 15
16 Cygwin Shell and Awk Utility You may use Cygwin Linux shell to demonstrate awk examples. Place file fstats in the Cygwin home directory. Cygwin Shell and Awk Utility Check the version of your awk utiltiy. 16
17 Where is your Root Directory? Be careful with directories. Use your home directory. Place fstats File in your Home Directory 17
18 Awk in Janoshell Differs a Bit Different versions of awk utilitiy may have different option switch flags and may differ. However, all versions must perform the same regular expressions use. Place Inout File fstats in Administrator Directory 18
19 Do all Slide s Examples Do all slide presentation awk examples in both shells Cygwin and Janotech shell. Observe all differences. The End ==== 19
5/8/2012. Exploring Utilities Chapter 5
Exploring Utilities Chapter 5 Examining the contents of files. Working with the cut and paste feature. Formatting output with the column utility. Searching for lines containing a target string with grep.
More informationAwk A Pattern Scanning and Processing Language (Second Edition)
Awk A Pattern Scanning and Processing Language (Second Edition) Alfred V. Aho Brian W. Kernighan Peter J. Weinberger Bell Laboratories Murray Hill, New Jersey 07974 ABSTRACT Awk is a programming language
More informationCSCI 2132 Software Development. Lecture 7: Wildcards and Regular Expressions
CSCI 2132 Software Development Lecture 7: Wildcards and Regular Expressions Instructor: Vlado Keselj Faculty of Computer Science Dalhousie University 20-Sep-2017 (7) CSCI 2132 1 Previous Lecture Pipes
More informationAwk APattern Scanning and Processing Language (Second Edition)
Awk APattern Scanning and Processing Language (Second Edition) Alfred V. Aho Brian W. Kernighan Peter J. Weinberger Bell Laboratories Murray Hill, New Jersey 07974 ABSTRACT Awk is a programming language
More informationShell Programming Overview
Overview Shell programming is a way of taking several command line instructions that you would use in a Unix command prompt and incorporating them into one program. There are many versions of Unix. Some
More information5/20/2007. Touring Essential Programs
Touring Essential Programs Employing fundamental utilities. Managing input and output. Using special characters in the command-line. Managing user environment. Surveying elements of a functioning system.
More informationCommon File System Commands
Common File System Commands ls! List names of all files in current directory ls filenames! List only the named files ls -t! List in time order, most recent first ls -l! Long listing, more information.
More informationWildcards and Regular Expressions
CSCI 2132: Software Development Wildcards and Regular Expressions Norbert Zeh Faculty of Computer Science Dalhousie University Winter 2019 Searching Problem: Find all files whose names match a certain
More informationRegex, Sed, Awk. Arindam Fadikar. December 12, 2017
Regex, Sed, Awk Arindam Fadikar December 12, 2017 Why Regex Lots of text data. twitter data (social network data) government records web scrapping many more... Regex Regular Expressions or regex or regexp
More informationPattern Matching. An Introduction to File Globs and Regular Expressions
Pattern Matching An Introduction to File Globs and Regular Expressions Copyright 2006 2009 Stewart Weiss The danger that lies ahead Much to your disadvantage, there are two different forms of patterns
More informationLecture #13 AWK Part I (Chapter 6)
CS390 UNIX Programming Spring 2009 Page 1 Lecture #13 AWK Part I (Chapter 6) Background AWK is a pattern-scanning and processing language. Named after authors (Alfred Aho, Peter Weinberger, Brian Kernighan).
More informationfind Command as Admin Security Tool
find Command as Admin Security Tool Dr. Bill Mihajlovic INCS-620 Operating Systems Security find Command find command searches for the file or files that meet certain condition. like: Certain name Certain
More informationIntroduction to UNIX Part II
T H E U N I V E R S I T Y of T E X A S H E A L T H S C I E N C E C E N T E R A T H O U S T O N S C H O O L of H E A L T H I N F O R M A T I O N S C I E N C E S Introduction to UNIX Part II For students
More informationToday s Lecture. The Unix Shell. Unix Architecture (simplified) Lecture 3: Unix Shell, Pattern Matching, Regular Expressions
Lecture 3: Unix Shell, Pattern Matching, Regular Expressions Today s Lecture Review Lab 0 s info on the shell Discuss pattern matching Discuss regular expressions Kenneth M. Anderson Software Methods and
More informationCST Lab #5. Student Name: Student Number: Lab section:
CST8177 - Lab #5 Student Name: Student Number: Lab section: Working with Regular Expressions (aka regex or RE) In-Lab Demo - List all the non-user accounts in /etc/passwd that use /sbin as their home directory.
More informationsed Stream Editor Checks for address match, one line at a time, and performs instruction if address matched
Week11 sed & awk sed Stream Editor Checks for address match, one line at a time, and performs instruction if address matched Prints all lines to standard output by default (suppressed by -n option) Syntax:
More informationMastering Modern Linux by Paul S. Wang Appendix: Pattern Processing with awk
Mastering Modern Linux by Paul S. Wang Appendix: Pattern Processing with awk The awk program is a powerful yet simple filter. It processes its input one line at a time, applying user-specified awk pattern
More informationBashed One Too Many Times. Features of the Bash Shell St. Louis Unix Users Group Jeff Muse, Jan 14, 2009
Bashed One Too Many Times Features of the Bash Shell St. Louis Unix Users Group Jeff Muse, Jan 14, 2009 What is a Shell? The shell interprets commands and executes them It provides you with an environment
More informationCSCI 211 UNIX Lab. Shell Programming. Dr. Jiang Li. Jiang Li, Ph.D. Department of Computer Science
CSCI 211 UNIX Lab Shell Programming Dr. Jiang Li Why Shell Scripting Saves a lot of typing A shell script can run many commands at once A shell script can repeatedly run commands Help avoid mistakes Once
More informationAdvanced training. Linux components Command shell. LiLux a.s.b.l.
Advanced training Linux components Command shell LiLux a.s.b.l. alexw@linux.lu Kernel Interface between devices and hardware Monolithic kernel Micro kernel Supports dynamics loading of modules Support
More informationGetting to grips with Unix and the Linux family
Getting to grips with Unix and the Linux family David Chiappini, Giulio Pasqualetti, Tommaso Redaelli Torino, International Conference of Physics Students August 10, 2017 According to the booklet At this
More informationTable of contents. Our goal. Notes. Notes. Notes. Summer June 29, Our goal is to see how we can use Unix as a tool for developing programs
Summer 2010 Department of Computer Science and Engineering York University Toronto June 29, 2010 1 / 36 Table of contents 1 2 3 4 2 / 36 Our goal Our goal is to see how we can use Unix as a tool for developing
More information1 Lexical Considerations
Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science 6.035, Spring 2013 Handout Decaf Language Thursday, Feb 7 The project for the course is to write a compiler
More informationPattern Matching. An Introduction to File Globs and Regular Expressions. Adapted from Practical Unix and Programming Hunter College
Pattern Matching An Introduction to File Globs and Regular Expressions Adapted from Practical Unix and Programming Hunter College Copyright 2006 2009 Stewart Weiss The danger that lies ahead Much to your
More informationBasic Unix Command. It is used to see the manual of the various command. It helps in selecting the correct options
Basic Unix Command The Unix command has the following common pattern command_name options argument(s) Here we are trying to give some of the basic unix command in Unix Information Related man It is used
More informationITST Searching, Extracting & Archiving Data
ITST 1136 - Searching, Extracting & Archiving Data Name: Step 1 Sign into a Pi UN = pi PW = raspberry Step 2 - Grep - One of the most useful and versatile commands in a Linux terminal environment is the
More informationToday. Review. Unix as an OS case study Intro to Shell Scripting. What is an Operating System? What are its goals? How do we evaluate it?
Today Unix as an OS case study Intro to Shell Scripting Make sure the computer is in Linux If not, restart, holding down ALT key Login! Posted slides contain material not explicitly covered in class 1
More informationStd: XI CHAPTER-3 LINUX
Commands: General format: Command Option Argument Command: ls - Lists the contents of a file. Option: Begins with minus sign (-) ls a Lists including the hidden files. Argument refers to the name of a
More informationA A B U n i v e r s i t y
A A B U n i v e r s i t y Faculty of Computer Sciences O b j e c t O r i e n t e d P r o g r a m m i n g Week 4: Introduction to Classes and Objects Asst. Prof. Dr. M entor Hamiti mentor.hamiti@universitetiaab.com
More informationFall 2006 Shell programming, part 3. touch
touch Touch is usually a program, but it can be a shell built-in such as with busybox. The touch program by default changes the access and modification times for the files listed as arguments. If the file
More informationhttp://xkcd.com/208/ 1. Review of pipes 2. Regular expressions 3. sed 4. awk 5. Editing Files 6. Shell loops 7. Shell scripts cat seqs.fa >0! TGCAGGTATATCTATTAGCAGGTTTAATTTTGCCTGCACTTGGTTGGGTACATTATTTTAAGTGTATTTGACAAG!
More informationFILTERS USING REGULAR EXPRESSIONS grep and sed
FILTERS USING REGULAR EXPRESSIONS grep and sed We often need to search a file for a pattern, either to see the lines containing (or not containing) it or to have it replaced with something else. This chapter
More informationObjectives. In this chapter, you will:
Objectives In this chapter, you will: Become familiar with functions, special symbols, and identifiers in C++ Explore simple data types Discover how a program evaluates arithmetic expressions Learn about
More informationCSCI 2132 Software Development. Lecture 4: Files and Directories
CSCI 2132 Software Development Lecture 4: Files and Directories Instructor: Vlado Keselj Faculty of Computer Science Dalhousie University 12-Sep-2018 (4) CSCI 2132 1 Previous Lecture Some hardware concepts
More informationIntroduction of Linux
Introduction of Linux 阳 oslab2018_class1@163.com 寅 oslab2018_class2@163.com PART I Brief Introduction Basic Conceptions & Environment Install & Configure a Virtual Machine Basic Commands PART II Shell
More informationUNIX files searching, and other interrogation techniques
UNIX files searching, and other interrogation techniques Ways to examine the contents of files. How to find files when you don't know how their exact location. Ways of searching files for text patterns.
More informationA Brief Introduction to the Linux Shell for Data Science
A Brief Introduction to the Linux Shell for Data Science Aris Anagnostopoulos 1 Introduction Here we will see a brief introduction of the Linux command line or shell as it is called. Linux is a Unix-like
More informationShell Start-up and Configuration Files
ULI101 Week 10 Lesson Overview Shell Start-up and Configuration Files Shell History Alias Statement Shell Variables Introduction to Shell Scripting Positional Parameters echo and read Commands if and test
More informationBasic Shell Scripting Practice. HPC User Services LSU HPC & LON March 2018
Basic Shell Scripting Practice HPC User Services LSU HPC & LON sys-help@loni.org March 2018 Quotation Exercise 1. Print out your $LOGNAME 2. Print date 3. Print `who am i` 4. Print your current directory
More informationTypescript on LLVM Language Reference Manual
Typescript on LLVM Language Reference Manual Ratheet Pandya UNI: rp2707 COMS 4115 H01 (CVN) 1. Introduction 2. Lexical Conventions 2.1 Tokens 2.2 Comments 2.3 Identifiers 2.4 Reserved Keywords 2.5 String
More informationShells & Shell Programming (Part B)
Shells & Shell Programming (Part B) Software Tools EECS2031 Winter 2018 Manos Papagelis Thanks to Karen Reid and Alan J Rosenthal for material in these slides CONTROL STATEMENTS 2 Control Statements Conditional
More informationRegular Expressions. Regular expressions match input within a line Regular expressions are very different than shell meta-characters.
ULI101 Week 09 Week Overview Regular expressions basics Literal matching.wildcard Delimiters Character classes * repetition symbol Grouping Anchoring Search Search and replace in vi Regular Expressions
More informationBash Programming. Student Workbook
Student Workbook Bash Programming Published by ITCourseware, LLC, 7245 South Havana Street, Suite 100, Englewood, CO 80112 Contributing Authors: Julie Johnson, Rob Roselius Editor: Jeff Howell Special
More informationLexical Considerations
Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science 6.035, Spring 2010 Handout Decaf Language Tuesday, Feb 2 The project for the course is to write a compiler
More informationUnix for Developers grep, sed, awk
Unix for Developers grep, sed, Benedict Reuschling November 30, 2017 1 / 56 Overview In this part of the lecture we will look at grep, sed, and as tools for processing and analyzing of data. 2 / 56 grep
More informationData Analysis in Geophysics ESCI Bob Smalley Room 103 in 3892 (long building), x Tu/Th - 13:00-14:30 CERI MAC (or STUDENT) LAB
Data Analysis in Geophysics ESCI 7205 Bob Smalley Room 103 in 3892 (long building), x-4929 Tu/Th - 13:00-14:30 CERI MAC (or STUDENT) LAB Lab 21, 11/07/13 Computers make it easier to do a lot of things,
More informationUnix as a Platform Exercises + Solutions. Course Code: OS 01 UNXPLAT
Unix as a Platform Exercises + Solutions Course Code: OS 01 UNXPLAT Working with Unix Most if not all of these will require some investigation in the man pages. That's the idea, to get them used to looking
More information1. What statistic did the wc -l command show? (do man wc to get the answer) A. The number of bytes B. The number of lines C. The number of words
More Linux Commands 1 wc The Linux command for acquiring size statistics on a file is wc. This command provides the line count, word count and number of bytes in a file. Open up a terminal, make sure you
More informationCourse Outline. TERM EFFECTIVE: Fall 2016 CURRICULUM APPROVAL DATE: 11/23/2015
5055 Santa Teresa Blvd Gilroy, CA 95023 Course Outline COURSE: CSIS 49 DIVISION: 50 ALSO LISTED AS: TERM EFFECTIVE: Fall 2016 CURRICULUM APPROVAL DATE: 11/23/2015 SHORT TITLE: UNIX SHELL PROGRAM LONG TITLE:
More informationCS 25200: Systems Programming. Lecture 10: Shell Scripting in Bash
CS 25200: Systems Programming Lecture 10: Shell Scripting in Bash Dr. Jef Turkstra 2018 Dr. Jeffrey A. Turkstra 1 Lecture 10 Getting started with Bash Data types Reading and writing Control loops Decision
More informationPace University. Fundamental Concepts of CS121 1
Pace University Fundamental Concepts of CS121 1 Dr. Lixin Tao http://csis.pace.edu/~lixin Computer Science Department Pace University October 12, 2005 This document complements my tutorial Introduction
More informationLecture 2. Regular Expression Parsing Awk
Lecture 2 Regular Expression Parsing Awk Shell Quoting Shell Globing: file* and file? ls file\* (the backslash key escapes wildcards) Shell Special Characters ~ Home directory ` backtick (command substitution)
More informationLexical Considerations
Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science 6.035, Fall 2005 Handout 6 Decaf Language Wednesday, September 7 The project for the course is to write a
More informationLING 408/508: Computational Techniques for Linguists. Lecture 5
LING 408/508: Computational Techniques for Linguists Lecture 5 Last Time Installing Ubuntu 18.04 LTS on top of VirtualBox Your Homework 2: did everyone succeed? Ubuntu VirtualBox Host OS: MacOS or Windows
More informationRegular Expressions. Michael Wrzaczek Dept of Biosciences, Plant Biology Viikki Plant Science Centre (ViPS) University of Helsinki, Finland
Regular Expressions Michael Wrzaczek Dept of Biosciences, Plant Biology Viikki Plant Science Centre (ViPS) University of Helsinki, Finland November 11 th, 2015 Regular expressions provide a flexible way
More informationChapter 2: Basic Elements of C++
Chapter 2: Basic Elements of C++ Objectives In this chapter, you will: Become familiar with functions, special symbols, and identifiers in C++ Explore simple data types Discover how a program evaluates
More informationObjectives. Chapter 2: Basic Elements of C++ Introduction. Objectives (cont d.) A C++ Program (cont d.) A C++ Program
Objectives Chapter 2: Basic Elements of C++ In this chapter, you will: Become familiar with functions, special symbols, and identifiers in C++ Explore simple data types Discover how a program evaluates
More informationUNIX II:grep, awk, sed. October 30, 2017
UNIX II:grep, awk, sed October 30, 2017 File searching and manipulation In many cases, you might have a file in which you need to find specific entries (want to find each case of NaN in your datafile for
More information5/8/2012. Specifying Instructions to the Shell Chapter 8
An overview of shell. Execution of commands in a shell. Shell command-line expansion. Customizing the functioning of the shell. Employing advanced user features. Specifying Instructions to the Shell Chapter
More informationChapter 2: Basic Elements of C++ Objectives. Objectives (cont d.) A C++ Program. Introduction
Chapter 2: Basic Elements of C++ C++ Programming: From Problem Analysis to Program Design, Fifth Edition 1 Objectives In this chapter, you will: Become familiar with functions, special symbols, and identifiers
More informationBasics. I think that the later is better.
Basics Before we take up shell scripting, let s review some of the basic features and syntax of the shell, specifically the major shells in the sh lineage. Command Editing If you like vi, put your shell
More informationCS 307: UNIX PROGRAMMING ENVIRONMENT FIND COMMAND
CS 307: UNIX PROGRAMMING ENVIRONMENT FIND COMMAND Prof. Michael J. Reale Fall 2014 Finding Files in a Directory Tree Suppose you want to find a file with a certain filename (or with a filename matching
More informationShells and Shell Programming
Shells and Shell Programming 1 Shells A shell is a command line interpreter that is the interface between the user and the OS. The shell: analyzes each command determines what actions are to be performed
More informationIntroduction p. 1 Who Should Read This Book? p. 1 What You Need to Know Before Reading This Book p. 2 How This Book Is Organized p.
Introduction p. 1 Who Should Read This Book? p. 1 What You Need to Know Before Reading This Book p. 2 How This Book Is Organized p. 2 Conventions Used in This Book p. 2 Introduction to UNIX p. 5 An Overview
More informationIntroduction to Unix Week 3
Week 3 cat [file ] Display contents of files cat sample.file This is a sample file that i'll use to demo how the pr command is used. The pr command is useful in formatting various types of text files.
More informationRegular Expressions. Regular Expression Syntax in Python. Achtung!
1 Regular Expressions Lab Objective: Cleaning and formatting data are fundamental problems in data science. Regular expressions are an important tool for working with text carefully and eciently, and are
More informationUnix/Linux Primer. Taras V. Pogorelov and Mike Hallock School of Chemical Sciences, University of Illinois
Unix/Linux Primer Taras V. Pogorelov and Mike Hallock School of Chemical Sciences, University of Illinois August 25, 2017 This primer is designed to introduce basic UNIX/Linux concepts and commands. No
More informationWeek Overview. Simple filter commands: head, tail, cut, sort, tr, wc grep utility stdin, stdout, stderr Redirection and piping /dev/null file
ULI101 Week 05 Week Overview Simple filter commands: head, tail, cut, sort, tr, wc grep utility stdin, stdout, stderr Redirection and piping /dev/null file head and tail commands These commands display
More informationDr. Sarah Abraham University of Texas at Austin Computer Science Department. Regular Expressions. Elements of Graphics CS324e Spring 2017
Dr. Sarah Abraham University of Texas at Austin Computer Science Department Regular Expressions Elements of Graphics CS324e Spring 2017 What are Regular Expressions? Describe a set of strings based on
More informationAnswers to AWK problems. Shell-Programming. Future: Using loops to automate tasks. Download and Install: Python (Windows only.) R
Today s Class Answers to AWK problems Shell-Programming Using loops to automate tasks Future: Download and Install: Python (Windows only.) R Awk basics From the command line: $ awk '$1>20' filename Command
More informationLecture 18 Regular Expressions
Lecture 18 Regular Expressions In this lecture Background Text processing languages Pattern searches with grep Formal Languages and regular expressions Finite State Machines Regular Expression Grammer
More informationUNIX / LINUX - REGULAR EXPRESSIONS WITH SED
UNIX / LINUX - REGULAR EXPRESSIONS WITH SED http://www.tutorialspoint.com/unix/unix-regular-expressions.htm Copyright tutorialspoint.com Advertisements In this chapter, we will discuss in detail about
More informationLecture 5. Essential skills for bioinformatics: Unix/Linux
Lecture 5 Essential skills for bioinformatics: Unix/Linux UNIX DATA TOOLS Text processing with awk We have illustrated two ways awk can come in handy: Filtering data using rules that can combine regular
More informationShells and Shell Programming
Shells and Shell Programming Shells A shell is a command line interpreter that is the interface between the user and the OS. The shell: analyzes each command determines what actions are to be performed
More informationLanguage Reference Manual
TAPE: A File Handling Language Language Reference Manual Tianhua Fang (tf2377) Alexander Sato (as4628) Priscilla Wang (pyw2102) Edwin Chan (cc3919) Programming Languages and Translators COMSW 4115 Fall
More informationReview of Fundamentals
Review of Fundamentals 1 The shell vi General shell review 2 http://teaching.idallen.com/cst8207/14f/notes/120_shell_basics.html The shell is a program that is executed for us automatically when we log
More informationPESIT Bangalore South Campus
INTERNAL ASSESSMENT TEST - III Date : 09-11-2015 Marks : 0 Subject & Code : USP & 15CS36 Class : III ISE A & B Name of faculty : Prof. Ajoy Kumar Note: Solutions to ALL Questions Questions 1 a. Explain
More information1 The Var Shell (vsh)
CS 470G Project 1 The Var Shell Due Date: February 7, 2011 In this assignment, you will write a shell that allows the user to interactively execute Unix programs. Your shell, called the Var Shell (vsh),
More informationReading and manipulating files
Reading and manipulating files Goals By the end of this lesson you will be able to Read files without using text editors Access specific parts of files Count the number of words and lines in a file Sort
More informationSession: Shell Programming Topic: Advanced Commands
Lecture Session: Shell Programming Topic: Advanced Commands Daniel Chang Text File Processing Reading and filtering text files cut - Print specific columns from a text file awk - Print specific lines from
More informationModule 8 Pipes, Redirection and REGEX
Module 8 Pipes, Redirection and REGEX Exam Objective 3.2 Searching and Extracting Data from Files Objective Summary Piping and redirection Partial POSIX Command Line and Redirection Command Line Pipes
More informationEECS2301. Lab 1 Winter 2016
EECS2301 Lab 1 Winter 2016 Lab Objectives In this lab, you will be introduced to the Linux operating system. The basic commands will be presented in this lab. By the end of you alb, you will be asked to
More informationPractical 02. Bash & shell scripting
Practical 02 Bash & shell scripting 1 imac lab login: maclab password: 10khem 1.use the Finder to visually browse the file system (single click opens) 2.find the /Applications folder 3.open the Utilities
More informationQUESTION BANK ON UNIX & SHELL PROGRAMMING-502 (CORE PAPER-2)
BANK ON & SHELL PROGRAMMING-502 (CORE PAPER-2) TOPIC 1: VI-EDITOR MARKS YEAR 1. Explain set command of vi editor 2 2011oct 2. Explain the modes of vi editor. 7 2013mar/ 2013 oct 3. Explain vi editor 5
More informationOpen up a terminal, make sure you are in your home directory, and run the command.
More Linux Commands 0.1 wc The Linux command for acquiring size statistics on a file is wc. This command can provide information from line count, to bytes in a file. Open up a terminal, make sure you are
More informationOPERATING SYSTEMS LAB LAB # 6. I/O Redirection and Shell Programming. Shell Programming( I/O Redirection and if-else Statement)
P a g e 1 OPERATING SYSTEMS LAB LAB 6 I/O Redirection and Shell Programming Lab 6 Shell Programming( I/O Redirection and if-else Statement) P a g e 2 Redirection of Standard output/input i.e. Input - Output
More informationEssential Linux Shell Commands
Essential Linux Shell Commands Special Characters Quoting and Escaping Change Directory Show Current Directory List Directory Contents Working with Files Working with Directories Special Characters There
More informationCSC UNIX System, Spring 2015
CSC 352 - UNIX System, Spring 2015 Study guide for the CSC352 midterm exam (20% of grade). Dr. Dale E. Parson, http://faculty.kutztown.edu/parson We will have a midterm on March 19 on material we have
More informationThe e switch allows Perl to execute Perl statements at the command line instead of from a script.
CH02.DOC Page 5 Friday, January 26, 2001 2:35 PM 2 Perl Scripts 2.1 Perl at the Command Line Although most of your work with Perl will be done in scripts, Perl can also be executed at the command line
More informationUse of AWK / SED Prof. Navrati Saxena TA: Rochak Sachan
Use of AWK / SED Prof. Navrati Saxena TA: Rochak Sachan 1 CONTENTS 1. AWK 1.1 WHAT IS AWK? 1.2 GENERAL SYNTAX OF AWK 1.3 DOING ARITHMETIC WITH AWK 1.4 VAR. IN AWK 1.4 AWK MISCELLANEOUS 2. SED 2.1 WHAT
More informationhttp://xkcd.com/208/ 1. Review of pipes 2. Regular expressions 3. sed 4. Editing Files 5. Shell loops 6. Shell scripts cat seqs.fa >0! TGCAGGTATATCTATTAGCAGGTTTAATTTTGCCTGCACTTGGTTGGGTACATTATTTTAAGTGTATTTGACAAG!
More informationOperating System Interaction via bash
Operating System Interaction via bash bash, or the Bourne-Again Shell, is a popular operating system shell that is used by many platforms bash uses the command line interaction style generally accepted
More informationIB047. Unix Text Tools. Pavel Rychlý Mar 3.
Unix Text Tools pary@fi.muni.cz 2014 Mar 3 Unix Text Tools Tradition Unix has tools for text processing from the very beginning (1970s) Small, simple tools, each tool doing only one operation Pipe (pipeline):
More informationUNIX shell scripting
UNIX shell scripting EECS 2031 Summer 2014 Przemyslaw Pawluk June 17, 2014 What we will discuss today Introduction Control Structures User Input Homework Table of Contents Introduction Control Structures
More informationGBIL: Generic Binary Instrumentation Language. Language Reference Manual. By: Andrew Calvano. COMS W4115 Fall 2015 CVN
GBIL: Generic Binary Instrumentation Language Language Reference Manual By: Andrew Calvano COMS W4115 Fall 2015 CVN Table of Contents 1) Introduction 2) Lexical Conventions 1. Tokens 2. Whitespace 3. Comments
More informationUnix as a Platform Exercises. Course Code: OS-01-UNXPLAT
Unix as a Platform Exercises Course Code: OS-01-UNXPLAT Working with Unix 1. Use the on-line manual page to determine the option for cat, which causes nonprintable characters to be displayed. Run the command
More informationCOMS 6100 Class Notes 3
COMS 6100 Class Notes 3 Daniel Solus September 1, 2016 1 General Remarks The class was split into two main sections. We finished our introduction to Linux commands by reviewing Linux commands I and II
More informationYou must have a basic understanding of GNU/Linux operating system and shell scripting.
i About the Tutorial This tutorial takes you through AWK, one of the most prominent text-processing utility on GNU/Linux. It is very powerful and uses simple programming language. It can solve complex
More informationShell scripting Scripting and Computer Environment - Lecture 5
Shell scripting Scripting and Computer Environment - Lecture 5 Saurabh Barjatiya International Institute Of Information Technology, Hyderabad 28 July, 2011 Contents 1 Contents 1 Shell scripting with bash
More informationS E C T I O N O V E R V I E W
AN INTRODUCTION TO SHELLS S E C T I O N O V E R V I E W Continuing from last section, we are going to learn about the following concepts: understanding quotes and escapes; considering the importance of
More information