bash Cookbook Carl Albing, JP Vossen, and Cameron Newham O'REILLY* ULB Darmstadt Beijing Cambridge Farnham Koln Paris Sebastopol Taipei Tokyo

Transcription

1 bash Cookbook Carl Albing, JP Vossen, and Cameron Newham ULB Darmstadt O'REILLY* Beijing Cambridge Farnham Koln Paris Sebastopol Taipei Tokyo

2 Preface 1. Beginnina bash Decoding the Prompt Showing Where You Are Finding and Running Commands Getting Information About Files Showing All Hidden (dot) Files in the Current Directory Using Shell Quoting Using or Replacing Built-ins and External Commands Determining If You Are Running Interactively Setting bash As Your Default Shell Getting bash for Linux, Getting bash for xbsd Getting bash for Mac OS X Getting bash for Unix Getting bash for Windows Getting bash Without Getting bash Learning More About bash Documentation 2. Standard OutDut? Writing Output to the Terminal/Window Writing Output but Preserving Spacing Writing Output with More Formatting Control Writing Output Without the Newline Saving Output from a Command Saving Output to Other Files xiii

3 2.7 Saving Output from the Is Command Sending Both Output and Error Messages to Different Files Sending Both Output and Error Messages to the Same File Appending Rather Than Clobbering Output Using Just the Beginning or End of a File Skipping a Header in a File Throwing Output Away Saving or Grouping Output from Several Commands Connecting Two Programs by Using Output As Input Saving a Copy of Output Even While Using It As Input Connecting Two Programs by Using Output As Arguments Using Multiple Redirects on One Line Saving Output When Redirect Doesn't Seem to Work Swapping STDERR and STDOUT Keeping Files Safe from Accidental Overwriting Clobbering a File on Purpose Standard Input Getting Input from a File Keeping Your Data with Your Script Preventing Weird Behavior in a Here-Document Indenting Here-Documents Getting User Input Getting Yes or No Input Selecting from a List of Options Prompting for a Password "' Executina Commands Running Any Executable Telling If a Command Succeeded or Not Running Several Commands in Sequence Running Several Commands All at Once Deciding Whether a Command Succeeds " Using Fewer if Statements Running Long Jobs Unattended Displaying Error Messages When Failures Occur Running Commands from a Variable Running All Scripts in a Directory iv I

4 5. Basic Scrintina: Shell Variables 5.1 Documenting Your Script 5.2 Embedding Documentation in Shell Scripts 5.3 Promoting Script Readability 5.4 Separating Variable Names from Surrounding Text 5.5 Exporting Variables 5.6 Seeing All Variable Values 5.7 Using Parameters in a Shell Script 5.8 Looping Over Arguments Passed to a Script 5.9 Handling Parameters with Blanks 5.10 Handling Lists of Parameters with Blanks 5.11 Counting Arguments 5.12 Consuming Arguments 5.13 Getting Default Values 5.14 Setting Default Values 5.15 Using null As a Valid Default Value 5.16 Using More Than Just a Constant String for Default 5.17 Giving an Error Message for Unset Parameters 5.18 Changing Pieces of a String 5.19 Using Array Variables 6. Shell 1.ogic and Arithmetic ' Doing Arithmetic in Your Shell Script Branching on Conditions Testing for File Characteristics Testing for More Than One Thing Testing for String Characteristics Testing for Equal Testing with Pattern Matches Testing with Regular Expressions Changing Behavior with Redirections Looping for a While Looping with a read Looping with a Count Looping with Floating-Point Values Branching Many Ways Parsing Command-Line Arguments Creating Simple Menus v

5 6.17 Changing the Prompt on Simple Menus Creating a Simple RPN Calculator Creating a Command-Line Calculator Intermediate Shell Tools I Sifting Through Files for a String Getting Just the Filename from a Search Getting a Simple True/False from a Search Searching for Text While Ignoring Case Doing a Search in a Pipeline, Paring Down What the Search Finds Searching with More Complex Patterns Searching for an SSN Grepping Compressed Files " Keeping Some Output, Discarding the Rest Keeping Only a Portion of a Line of Output Reversing the Words on Each Line Summing a List of Numbers Counting String Values Showing Data As a Quick and Easy Histogram Showing a Paragraph of Text After a Found Phrase Intermediate Shell Tools II Sorting Your Output Sorting Numbers Sorting IP Addresses.,-> Cutting Out Parts of Your Output Removing Duplicate Lines Compressing Files Uncompressing Files Checking a tar Archive for Unique Directories Translating Characters Converting Uppercase to Lowercase ** Converting DOS Files to Linux Format Removing Smart Quotes Counting Lines, Words, or Characters in a File Rewrapping Paragraphs Doing More with less 181

6 9. Finding Files: find, locate, slocate Finding All Your MP3 Files Handling Filenames Containing Odd Characters Speeding Up Operations on Found Files Finding Files Across Symbolic Links Finding Files Irrespective of Case Finding Files by Date Finding Files by Type Finding Files by Size Finding Files by Content Finding Existing Files and Content Fast Finding a File Using a List of Possible Locations Additional Features for Scripting "Daemon-izing" Your Script Reusing Code with Includes and Sourcing Using Configuration Files in a Script Defining Functions Using Functions: Parameters and Return Values Trapping Interrupts Redefining Commands with alias Avoiding Aliases, Functions Working with Dates and Times Formatting Dates for Display ' Supplying a Default Date Automating Date Ranges Converting Dates and Times to Epoch Seconds Converting Epoch Seconds to Dates and Times Getting Yesterday or Tomorrow with Perl Figuring Out Date and Time Arithmetic Handling Time Zones, Daylight Saving Time, and Leap Years Using date and cfbn to Run a Script on the Nth Day End-User Tasks As Shell Scripts Starting Simple by Printing Dashes Viewing Photos in an Album Loading Your MP3 Player Burning a CD Comparing Two Documents 244 vii

7 13. Parsing and Similar Tasks Parsing Arguments for Your Shell Script Parsing Arguments with Your Own Error Messages Parsing Some HTML Parsing Output into an Array Parsing Output with a Function Call Parsing Text with a read Statement Parsing with read into an Array Getting Your Plurals Right Taking It One Character at a Time ' Cleaning Up an SVN Source Tree Setting Up a Database with MySQL Isolating Specific Fields in Data Updating Specific Fields in Data Files Trimming Whitespace Compressing Whitespace Processing Fixed-Length Records Processing Files with No Line Breaks Converting a Data File to CSV Parsing a CSV Data File Writing Secure Shell Scripts Avoiding Common Security Problems Avoiding Interpreter Spoofing Setting a Secure $PATH Clearing All Aliases ' : ' Clearing the Command Hash Preventing Core Dumps Setting a Secure $IFS Setting a Secure umask Finding World-Writable Directories in Your $PATH Adding the Current Directory to the $PATH Using Secure Temporary Files Validating Input Setting Permissions Leaking Passwords into the Process List Writing setuid or setgid Scripts Restricting Guest Users Using chroot Jails 303 viii I

8 14.18 Running Asa Non-root User Using sudo More Securely Using Passwords in Scripts Using SSH Without a Password Restricting SSH Commands Disconnecting Inactive Sessions Advanced Scripting Finding bash Portably for #! Setting a POSIX $PATH Developing Portable Shell Scripts Testing Scripts in VMware Using for Loops Portably Using echo Portably Splitting Output Only When Necessary Viewing Output in Hex Using bash Net-Redirection Finding My IP Address Getting Input from Another Machine Redirecting Output for the Life of a Script Working Around "argument list too long" Errors Logging to syslog from Your Script Sending from Your Script Automating a Process Using Phases Configuring and Customizing bash bash Startup Options Customizing Your Prompt Change Your $PATH Permanently Change Your $PATH Temporarily 362 > Setting Your SCDPATH Shortening or Changing Command Names Adjusting Shell Be"havior and Environment Adjusting readline Behavior Using.inputrc Keeping a Private Stash of Utilities by Adding /bin Using Secondary Prompts: $PS2, $PS3, $PS Synchronizing Shell History Between Sessions Setting Shell History Options 377

9 16.13 Creating a Better cd Command Creating and Changing into a New Directory in One Step Getting to the Bottom of Things Adding New Features to bash Using Loadable Built-ins Improving Programmable Completion Using Initialization Files Correctly Creating Self-Contained, Portable RC Files Getting Started with a Custom Configuration Housekeeping and Administrative Tasks Renaming Many Files Using GNU Texinfo and Info on Linux Unzipping Many ZIP Files Recovering Disconnected Sessions Using screen Sharing a Single bash Session Logging an Entire Session or Batch Job Clearing the Screen When You Log Out Capturing File Metadata for Recovery t Creating an Index of Many Files Using diff and patch Counting Differences in Files Removing or Renaming Files Named with Special Characters Prepending Data to a File Editing a File in Place Using sudo on a Group of Commands Finding Lines in One File But Not in the Other Keeping the Most Recent N Objects Grepping ps Output Without Also Getting the grep Process Itself Finding Out Whether a Process Is Running Adding a Prefix or Suffix to Output Numbering Lines Writing Sequences Emulating the DOS Pause Command Commifying Numbers Working Faster by Typing Less Moving Quickly Among Arbitrary Directories Repeating the Last Command Running Almost the Same Command 456 x

10 18.4 Substituting Across Word Boundaries Reusing Arguments Finishing Names for You Playing It Safe Tips and Traps: Common Goofs for Novices Forgetting to Set Execute Permissions Fixing "No such file or directory" Errors Forgetting That the Current Directory Is Not in the $PATH Naming Your Script Test Expecting to Change Exported Variables Forgetting Quotes Leads to "command not found" on Assignments Forgetting That Pattern Matching Alphabetizes Forgetting That Pipelines Make Subshells Making Your Terminal Sane Again Deleting Files Using an Empty Variable Seeing Odd Behavior from printf Testing bash Script Syntax Debugging Scripts Avoiding "command not found" When Using Functions Confusing Shell Wildcards and Regular Expressions 480 A. Reference Lists 482 bash Invocation 482 Prompt String Customizations 483 ANSI'Color Escape Sequences 484 Built-in Commands and Reserved Words 485 Built-in Shell Variables 487 set Options 491 shopt Options 492 Adjusting Shell Behavior Using set, shopt, and Environment Variables 494 Test Operators 505 I/O Redirection <* 506 echo Options and Escape Sequences 508 printf 509 Date and Time String Formatting with strftime 513 Pattern-Matching Characters 514 extglob Extended Pattern-Matching Operators 515 tr Escape Sequences 515 xi

11 Readline Init File Syntax 516 emacs Mode Commands 518 vi Control Mode Commands 520 Table of ASCII Values 522 B. Examples Included with bash Startup-Files Directory Examples 524 C. Command-Line Processing 532 Command-Line Processing Steps 532 D. Revision Control 538 CVS 539 Subversion 545 RCS 550 Other 557 E. Building bash from Source 559 Obtaining bash 559 Unpacking the Archive 559 What's in the Archive 560 Who Do I Turn To? 564 Index xii