Linux Command Line and Shell Scripting Bible

Linux Command Line and Shell Scripting Bible Richard Blum Wiley Publishing, Inc.

Linux Command Line and Shell Scripting Bible

Linux Command Line and Shell Scripting Bible Richard Blum Wiley Publishing, Inc.

Linux Command Line and Shell Scripting Bible Published by Wiley Publishing, Inc. 10475 Crosspoint Boulevard Indianapolis, IN 46256 www.wiley.com Copyright 2008 by Wiley Publishing, Inc., Indianapolis, Indiana Published simultaneously in Canada ISBN: 978-0-470-25128-7 Manufactured in the United States of America 10987654321 No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600. Requests to the Publisher for permission should be addressed to the Legal Department, Wiley Publishing, Inc., 10475 Crosspoint Blvd., Indianapolis, IN 46256, (317) 572-3447, fax (317) 572-4355, or online at http://www.wiley.com/go/permissions. Limit of Liability/Disclaimer of Warranty: The publisher and the author make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation warranties of fitness for a particular purpose. No warranty may be created or extended by sales or promotional materials. The advice and strategies contained herein may not be suitable for every situation. This work is sold with the understanding that the publisher is not engaged in rendering legal, accounting, or other professional services. If professional assistance is required, the services of a competent professional person should be sought. Neither the publisher nor the author shall be liable for damages arising herefrom. The fact that an organization or Website is referred to in this work as a citation and/or a potential source of further information does not mean that the author or the publisher endorses the information the organization or Website may provide or recommendations it may make. Further, readers should be aware that Internet Websites listed in this work may have changed or disappeared between when this work was written and when it is read. For general information on our other products and services or to obtain technical support, please contact our Customer Care Department within the U.S. at (800) 762-2974, outside the U.S. at (317) 572-3993 or fax (317) 572-4002. Library of Congress Cataloging-in-Publication Data is available from the publisher. Trademarks: Wiley, the Wiley logo, and related trade dress are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates, in the United States and other countries, and may not be used without written permission. Linux is a registered trademark of Linus Torvald. All other trademarks are the property of their respective owners. Wiley Publishing, Inc., is not associated with any product or vendor mentioned in this book. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may notbeavailableinelectronicbooks.

To all the people who ve helped form my education. Parents, relatives, teachers, coworkers, and even anonymous posters on the Internet. Always be prepared to accept education from wherever you find it. Always continue to learn new things. For the LORD gives wisdom, and from his mouth come knowledge and understanding. Proverbs 2:6 (NIV)

About the Author Richard Blum has worked in the IT industry for over 19 years as both a systems and network administrator. During this time he has administered Unix, Linux, Novell, and Microsoft servers, as well as helped design and maintain a 3500-user network utilizing Cisco switches and routers. He has used Linux servers and shell scripts to perform automated network monitoring, and has written shell scripts in just about every Unix shell environment. Rich has a bachelor of science degree in Electrical Engineering, and a master of science degree in Management, specializing in management information systems, from Purdue University. He is the author of several Linux books, including sendmail for Linux, Running qmail, Postfix, Open Source E-mail Security, Network Performance Open Source Toolkit, andprofessional Assembly Language Programming. He s also a coauthor of Professional Linux Programming and Linux For Dummies, 8th Edition. When he s not being a computer nerd, Rich plays bass guitar for his church worship band and enjoys spending time with his wife, Barbara, and their two daughters, Katie Jane and Jessica.

Credits Acquisitions Editor Jenny Watson Senior Development Editor Tom Dinse Technical Editor John Kennedy Production Editor Angela Smith Copy Editor Foxxe Editorial Services Editorial Manager Mary Beth Wakefield Production Manager Tim Tate Vice President and Executive Group Publisher Richard Swadley Vice President and Executive Publisher Joseph B. Wikert Project Coordinator, Cover Lynsey Stanford Proofreader Word One New York Indexer Melanie Belkin

Acknowledgments............................ xxv Introduction............................... xxvii Part I The Linux Command Line Chapter 1: Starting with Linux Shells...3 Chapter 2: Getting to the Shell...25 Chapter 3: Basic bash Shell Commands...59 Chapter 4: More bash Shell Commands...91 Chapter 5: Using Linux Environment Variables...123 Chapter 6: Understanding Linux File Permissions...147 Chapter 7: Working with Editors...171 Part II Shell Scripting Basics Chapter 8: Basic Script Building...201 Chapter 9: Using Structured Commands...229 Chapter 10: More Structured Commands...255 Chapter 11: Handling User Input...285 Chapter 12: Presenting Data...313 Chapter 13: Script Control...335 Part III Advanced Shell Scripting Chapter 14: Creating Functions...363 Chapter 15: Adding Color to Scripts...385 Chapter 16: Introducing sed and gawk...419 Chapter 17: Regular Expressions...447 Chapter 18: Advanced sed...473 Chapter 19: Advanced gawk...501 Part IV Alternative Linux Shells Chapter 20: The ash Shell...533 Chapter 21: The tcsh Shell...557 Chapter 22: The Korn Shell...587 Chapter 23: The zsh Shell...611 ix

Contents at a Glance Part V Advanced Topics Chapter 24: Using a Database...639 Chapter 25: Using the Web...673 Chapter 26: Using E-Mail...701 Chapter 27: Shell Scripts for Administrators...725 Appendix A: Quick Guide to bash Commands...749 Appendix B: Quick Guide to sed and gawk... 759 Appendix C: Comparing Shells...771 Index...777 x

Acknowledgments............................ xxv Introduction............................... xxvii Part I The Linux Command Line Chapter 1: Starting with Linux Shells.................... 3 What Is Linux?...3 Looking into the Linux kernel...4 The GNU utilities...12 The Linux desktop environment...14 Linux Distributions...20 Core Linux distributions...21 Specialized Linux distributions...22 The Linux LiveCD...23 Summary...24 Chapter 2: Getting to the Shell....................... 25 Terminal Emulation...25 Graphics capabilities...27 The keyboard...30 The terminfo Database...31 The Linux Console...35 The xterm Terminal...36 Command line parameters...37 The xterm main menu...38 The VT options menu...41 The VT fonts menu...43 The Konsole Terminal...45 Command line parameters...45 Sessions...45 The menu bar...48 The GNOME Terminal...52 The command line parameters...52 Tabs...53 The menu bar...54 Summary...58 xi

Contents Chapter 3: Basic bash Shell Commands.................. 59 Starting the Shell...59 The Shell Prompt...60 The bash Manual...63 Filesystem Navigation...64 The Linux filesystem...64 Traversing directories...66 File and Directory Listing...69 Basic listing...69 Modifying the information presented...71 The complete parameter list...72 Filtering listing output...74 File Handling...75 Creating files...75 Copying files...76 Linking files...79 Renaming files...80 Deleting files...81 Directory Handling...82 Creating directories...82 Deleting directories... 82 Viewing File Contents...83 Viewing file statistics...84 Viewing the file type...84 Viewing the whole file...85 Viewing parts of a file...89 Summary...90 Chapter 4: More bash Shell Commands.................. 91 Monitoring Programs...91 Peeking at the processes...91 Real-time process monitoring...98 Stopping processes...101 Monitoring Disk Space...104 Mounting media...104 Using the df command...108 Using the du command...109 Working with Data Files...110 Sorting data...110 Searching for data...114 Compressing data...116 Archiving data...120 Summary...121 xii

Contents Chapter 5: Using Linux Environment Variables.............. 123 What Are Environment Variables?...123 Global environment variables...124 Local environment variables...125 Setting Environment Variables...127 Setting local environment variables...127 Setting global environment variables...129 Removing Environment Variables...129 Default Shell Environment Variables...130 Setting the PATH Environment Variable...134 Locating System Environment Variables... 135 Login shell...136 Interactive shell...139 Non-interactive shell...141 Variable Arrays...142 Using Command Aliases...143 Summary...144 Chapter 6: Understanding Linux File Permissions............. 147 Linux Security...147 The /etc/passwd file...148 The /etc/shadow file...150 Adding a new user...150 Removing a user...153 Modifying a user...154 Using Linux Groups...157 The /etc/group file...157 Creating new groups...158 Modifying groups...159 Decoding File Permissions...160 Using file permission symbols...160 Default file permissions...161 Changing Security Settings... 163 Changing permissions...163 Changing ownership...165 Sharing Files...166 Summary...168 Chapter 7: Working with Editors..................... 171 The vim Editor...171 The basics of vim...172 Editing data...174 Copy and paste...174 Search and substitute...175 xiii