Join the p2p.wrox.com. Wrox Programmer to Programmer. Beginning. Python. Using Python 2.6 and Python 3.1. James Payne

Transcription

1 Join the p2p.wrox.com Wrox Programmer to Programmer Beginning Python Using Python 2.6 and Python 3.1 James Payne

2 Programmer to Programmer Get more out of wrox.com Interact Take an active role online by participating in our P2P p2p.wrox.com Wrox Online Library Hundreds of our books are available online through Books24x7.com Wrox Blox Download short informational pieces and code to keep you up to date and out of trouble! Join the Community Sign up for our free monthly newsletter at newsletter.wrox.com Browse Ready for more Wrox? We have books and e-books available on.net, SQL Server, Java, XML, Visual Basic, C#/ C++, and much more! Contact Us. We always like to get feedback from our readers. Have a book idea? Need community support? Let us know by ing wrox-partnerwithus@wrox.com

3 Beginning Python : Using Python 2.6 and Python 3.1 Introduction xxvii Part I: Dipping Your Toe into Python Chapter 1: Programming Basics and Strings Chapter 2: Numbers and Operators Chapter 3: Variables Names for Values Part II: Python Language and the Standard Library Chapter 4: Making Decisions Chapter 5: Functions Chapter 6: Classes and Objects Chapter 7: Organizing Programs Chapter 8: Files and Directories Chapter 9: Other Features of the Language Chapter 10: Building a Module Chapter 11: Text Processing Part III: Putting Python to Work Chapter 12: Testing Chapter 13: Writing a GUI with Python Chapter 14: Accessing Databases Chapter 15: Using Python for XML Chapter 16: Network Programming Continues

4 Chapter 17: Extension Programming with C Chapter 18: Numerical Programming Chapter 19: An Introduction to Django Chapter 20: Web Applications and Web Services Chapter 21: Integrating Java with Python Part IV: Appendices Appendix A: Answers to the Exercises Appendix B: Online Resources Appendix C: What s New in Python Appendix D: Glossary Index

5 Beginning Python

6

7 Beginning Python Using Python 2.6 and Python 3.1 James Payne Wiley Publishing, Inc.

8 Beginning Python : Using Python 2.6 and Python 3.1 Published by Wiley Publishing, Inc Crosspoint Boulevard Indianapolis, IN Copyright 2010 by Wiley Publishing, Inc., Indianapolis, Indiana Published simultaneously in Canada ISBN: Manufactured in the United States of America No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) , fax (978) Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) , fax (201) , or online at Limit of Liability/Disclaimer of Warranty: The publisher and the author make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation warranties of fitness for a particular purpose. No warranty may be created or extended by sales or promotional materials. The advice and strategies contained herein may not be suitable for every situation. This work is sold with the understanding that the publisher is not engaged in rendering legal, accounting, or other professional services. If professional assistance is required, the services of a competent professional person should be sought. Neither the publisher nor the author shall be liable for damages arising here from. The fact that an organization or Web site is referred to in this work as a citation and/or a potential source of further information does not mean that the author or the publisher endorses the information the organization or Web site may provide or recommendations it may make. Further, readers should be aware that Internet Web sites listed in this work may have changed or disappeared between when this work was written and when it is read. For general information on our other products and services please contact our Customer Care Department within the United States at (877) , outside the United States at (317) or fax (317) Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. Library of Congress Control Number: Trademarks: Wiley, the Wiley logo, Wrox, the Wrox logo, Wrox Programmer to Programmer, and related trade dress are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates, in the United States and other countries, and may not be used without written permission. Python is a registered trademark of Python Software Foundation. All other trademarks are the property of their respective owners. Wiley Publishing, Inc. is not associated with any product or vendor mentioned in this book.

9 To my patient and loving wife, Whitney, who believed in me before I did. And to my parents, Ronnie and Sharon Payne, who raised me to believe I could do anything I put my mind to, even when I told them I wanted to be Santa Claus. For my brother, Ron, who read my work even when it was bad, Dorjan, Eric, Clem, and Nick because they know things about me and will tell them if I don t include them.

10

11 About the Author James Payne (Margate, FL) is Editor-in-Chief of Developer Shed, Inc. and has been writing and programming since the age of seven years old. Proficient in many languages, he has written over 400 articles covering practically every major programming language. As a contractor, he develops proprietary software for the financial industry using Python and likes to dabble in Django in his spare time.

12

13 Executive Editor Carol Long Project Editor Ed Connor Technical Editor Chris McAvoy Production Editors Amy Weintraub and Tim Tate Copy Editor Kim Cofer Editorial Director Robyn B. Siesky Editorial Manager Mary Beth Wakefield Marketing Manager David Mayhew Credits Production Manager Tim Tate Vice President and Executive Group Publisher Richard Swadley Vice President and Executive Publisher Barry Pruett Associate Publisher Jim Minatel Project Coordinator, Cover Lynsey Stanford Proofreaders Scott Klemp and Kristy Eldredge, Word One Indexer Ron Strauss Cover Image istockphoto.com/frank_u

14

15 Acknowledgments I would like to acknowledge the gang at Developer Shed: Jack and Jay Kim, whose constant hatred is an inspiration, Charles Fagundes, who made me an editor and not just a writer, Keith Lee, who provided coding support, and a special thanks to Jenny Ruggieri, who got me the job that got me this book. Lastly, I d like to thank all of the people that worked on the previous editions of this book for laying the groundwork, and the Editors Carol Long, Jenny Watson, Ed Connor, and Chris McAvoy who helped me meet deadlines no matter how much I didn t want to. I would also like to acknowledge Guido Von Rossum, without whom there would be no language to write about.

16

17 Contents Introduction xxvii Part I: Dipping Your Toe into Python 1 Chapter 1: Programming Basics and Strings 3 How Programming is Different from Using a Computer 3 Programming is Consistency 4 Programming is Control 4 Programming Copes with Change 5 What All That Means Together 5 The First Steps 5 Installing Python 3.1 on Non-Windows Systems 6 Using the Python Shell 6 Beginning to Use Python Strings 7 What is a String? 7 Why the Quotes? 7 Why Three Types of Quotes? 8 Using the print() Function 8 Understanding Different Quotes 9 Putting Two Strings Together 11 Joining Strings with the Print() Function 12 Putting Strings Together in Different Ways 12 Summary 13 Exercises 14 Chapter 2: Numbers and Operators 15 Different Kinds of Numbers 15 Numbers in Python 16 Program Files 18 Using the Different Types 19 Basic Math 21 Some Surprises 23 Using Numbers 24 Order of Evaluation 24 Number Formats 25

18 Contents Mistakes Will Happen 26 Some Unusual Cases 27 Summary 28 Exercises 29 Chapter 3: Variables Names for Values 31 Referring to Data Using Names for Data 31 Changing Data Through Names 33 Copying Data 33 Names You Can t Use and Some Rules 34 Using More Built-in Types 34 Tuples Unchanging Sequences of Data 34 Lists Changeable Sequences of Data 37 Dictionaries Groupings of Data Indexed by Name 39 Treating a String Like a List 41 Special Types 42 Other Common Sequence Properties 43 Referencing the Last Elements 43 Ranges of Sequences 44 Growing Lists by Appending Sequences 45 Using Lists to Temporarily Store Data 45 Working with Sets 46 Summary 47 Exercises 48 Part II: Python Language and the Standard Library 49 Chapter 4: Making Decisions 51 Comparing Values Are They the Same? 51 Doing the Opposite Not Equal 53 Comparing Values Which One Is More? 54 More Than or Equal, Less Than or Equal 55 Reversing True and False 56 Looking for the Results of More Than One Comparison 56 How to Get Decisions Made 57 Repetition 60 How to Do Something Again and Again 60 Stopping the Repetition 62 Handling Errors 65 Trying Things Out 65 xvi

19 Contents Summary 67 Exercises 69 Chapter 5: Functions 71 Putting Your Program into Its Own File 71 Functions: Grouping Code under a Name 73 Choosing a Name 75 Describing a Function in the Function 75 The Same Name in Two Different Places 76 Making Notes to Yourself 78 Asking a Function to Use a Value You Provide 79 Checking Your Parameters 81 Setting a Default Value for a Parameter Just in Case 83 Calling Functions from within Other Functions 84 Functions Inside of Functions 86 Flagging an Error on Your Own Terms 87 Layers of Functions 88 How to Read Deeper Errors 88 Summary 89 Exercises 90 Chapter 6: Classes and Objects 93 Thinking About Programming 93 What is an Object? 93 Objects You Already Know 94 Looking Ahead: How You Want to Use Objects 95 Defining a Class 96 How Code Can Be Made into an Object 96 Objects and Their Scope 104 Summary 107 Exercises 108 Chapter 7: Organizing Programs 111 Modules 112 Importing a Module So That You Can Use It 112 Making a Module from Pre-existing Code 113 Using Modules Starting with the Command Line 115 Changing How Import Works Bringing in More 118 Packages 118 xvii

20 Contents Modules and Packages 120 Bringing Everything into the Current Scope 120 Re-importing Modules and Packages 121 Basics of Testing Your Modules and Packages 124 Summary 124 Exercises 125 Chapter 8: Files and Directories 127 File Objects 127 Writing Text Files 128 Appending Text to a File 129 Reading Text Files 130 File Exceptions 131 Paths and Directories 131 Exceptions in os 132 Paths 132 Directory Contents 135 Obtaining Information about Files 136 Renaming, Moving, Copying, and Removing Files 137 Example: Rotating Files 138 Creating and Removing Directories 140 Globbing 140 Summary 142 Exercises 142 Chapter 9: Other Features of the Language 143 Lambda and Filter: Short Anonymous Functions 143 Map: Short-Circuiting Loops 144 Decisions within Lists List Comprehension 145 Generating Iterators for Loops 146 Special String Substitution Using Dictionaries 148 Featured Modules 149 Getopt Getting Options from the Command Line 149 Using More Than One Process 152 Threads Doing Many Things in the Same Process 154 Summary 156 Exercises 156 xviii

21 Contents Chapter 10: Building a Module 157 Exploring Modules 157 Importing Modules 159 Finding Modules 159 Digging through Modules 160 Creating Modules and Packages 162 Working with Classes 163 Defining Object-Oriented Programming 163 Creating Classes 163 Extending Existing Classes 165 Finishing Your Modules 166 Defining Module-Specific Errors 166 Choosing What to Export 167 Documenting Your Modules 168 Testing Your Module 176 Running a Module as a Program 178 Creating a Whole Module 179 Installing Your Modules 183 Summary 187 Exercises 188 Chapter 11: Text Processing 189 Why Text Processing Is So Useful 189 Searching for Files 190 Clipping Logs 191 Sifting through Mail 192 Navigating the File System with the os Module 192 Working with Regular Expressions and the re Module 199 Summary 203 Exercises 204 Part III: Putting Python to Work 205 Chapter 12: Testing 207 Assertions 208 Test Cases and Test Suites 209 Test Fixtures 213 Putting It All Together with Extreme Programming 216 xix

22 Contents Implementing a Search Utility in Python 216 A More Powerful Python Search 222 Formal Testing in the Software Life Cycle 224 Summary 225 Chapter 13: Writing a GUI with Python 227 GUI Programming Toolkits for Python 228 Tkinter Introduction 229 Creating GUI Widgets with Tkinter 229 Resizing the Widget 230 Configuring Widget Options 231 Putting the Widgets to Work 231 Creating Layouts 232 Packing Order 233 Controlling Widget Appearances 233 Radio Buttons and Checkboxes 235 Dialog Boxes 236 Other Widget Types 237 Summary 238 Exercises 238 Chapter 14: Accessing Databases 239 Working with DBM Persistent Dictionaries 240 Choosing a DBM Module 240 Creating Persistent Dictionaries 241 Accessing Persistent Dictionaries 243 Deciding When to Use DBM and When to Use a Relational Database 245 Working with Relational Databases 245 Writing SQL Statements 247 Defining Tables 249 Setting Up a Database 250 Using the Python Database APIs 252 Downloading Modules 252 Creating Connections 253 Working with Cursors 253 Working with Transactions and Committing the Results 260 Examining Module Capabilities and Metadata 261 Handling Errors 261 Summary 262 Exercises 263 xx

23 Contents Chapter 15: Using Python for XML 265 What Is XML? 265 A Hierarchical Markup Language 265 A Family of Standards 267 What Is a Schema/DTD? 268 What Are Document Models For? 268 Do You Need One? 268 Document Type Definitions 268 An Example DTD 268 DTDs Aren t Exactly XML 270 Limitations of DTDs 270 Schemas 270 An Example Schema 270 Schemas Are Pure XML 271 Schemas Are Hierarchical 271 Other Advantages of Schemas 271 XPath 272 HTML as a Subset of XML 272 The HTML DTDs 273 HTMLParser 273 XML Libraries Available for Python 274 What Is SAX? 274 Stream-based 275 Event-driven 275 What Is DOM? 275 In-memory Access 275 Why Use SAX or DOM 275 Capability Trade-Offs 276 Memory Considerations 276 Speed Considerations 276 SAX and DOM Parsers Available for Python 276 xml.sax 276 xml.dom.minidom 277 Intro to XSLT 280 XSLT Is XML 280 Transformation and Formatting Language 280 Functional, Template-Driven 280 What Is lxml? 280 Element Classes 281 Adding Text to Elements 282 xxi

24 Contents xxii Parsing with lxml 283 Parsing Files 284 Summary 285 Exercises 285 Chapter 16: Network Programming 287 Understanding Protocols 289 Comparing Protocols and Programming Languages 289 The Internet Protocol Stack 290 A Little Bit About the Internet Protocol 292 Sending Internet 293 The File Format 294 MIME Messages 295 Sending Mail with SMTP and smtplib 303 Retrieving Internet 305 Parsing a Local Mail Spool with mailbox 305 Fetching Mail from a POP3 Server with poplib 307 Fetching Mail from an IMAP Server with imaplib 309 Secure POP3 and IMAP 313 Webmail Applications Are Not Applications 313 Socket Programming 314 Introduction to Sockets 314 Binding to an External Hostname 316 The Mirror Server 317 The Mirror Client 318 SocketServer 320 Multithreaded Servers 321 The Python Chat Server 322 Design of the Python Chat Server 323 The Python Chat Server Protocol 323 The Python Chat Client 329 Single-Threaded Multitasking with select 331 Other Topics 332 Miscellaneous Considerations for Protocol Design 333 The Peer-to-Peer Architecture 333 Summary 334 Exercises 335 Chapter 17: Extension Programming with C 337 Extension Module Outline 338 Building and Installing Extension Modules 340

25 Contents Passing Parameters from Python to C 342 Returning Values from C to Python 345 The LAME Project 346 The LAME Extension Module 350 Using Python Objects from C Code 363 Summary 366 Exercises 366 Chapter 18: Numerical Programming 367 Numbers in Python 368 Integers 368 Long Integers 369 Floating-point Numbers 369 Formatting Numbers 370 Characters as Numbers 373 Mathematics 374 Arithmetic 374 Built-in Math Functions 375 Complex Numbers 378 Arrays 380 The array Module 382 Summary 384 Exercises 384 Chapter 19: An Introduction to Django 387 What Are Frameworks and Why Would I Use One? 388 Other Features of Web Frameworks 388 Django How It All Began 389 Installing Django 389 Understanding Django s Architecture 390 Initial Project Setup 391 Creating a View 394 Working with Templates 396 Using Templates and Views 398 Models 401 Creating a Model: First Steps Configure the Database Settings 401 Creating a Model: Creating an Application 403 Working with Models: Installation 404 Summary 405 Exercises 406 xxiii

26 Contents Chapter 20: Web Applications and Web Services 407 xxiv REST: The Architecture of the Web 408 Characteristics of REST 409 REST Operations 410 HTTP: Real-World REST 411 The Visible Web Server 412 The HTTP Request 415 The HTTP Response 416 CGI: Turning Scripts into Web Applications 417 The Web Server Makes a Deal with the CGI Script 419 CGI s Special Environment Variables 420 Accepting User Input through HTML Forms 422 HTML Forms Limited Vocabulary 422 The cgi Module: Parsing HTML Forms 423 Safety When Accessing Form Values 423 Building a Wiki 428 The BittyWiki Core Library 429 The BittyWiki Web Interface 432 Web Services 441 How Web Services Work 442 REST Web Services 442 REST Quick Start: Finding Bargains on Amazon.com 443 Introducing WishListBargainFinder 445 Giving BittyWiki a REST API 448 Wiki Search-and-Replace Using the REST Web Service 451 XML-RPC 456 The XML-RPC Request 457 The XML-RPC Response 459 If Something Goes Wrong 459 Exposing the BittyWiki API through XML-RPC 460 Wiki Search-and-Replace Using the XML-RPC Web Service 463 SOAP 465 SOAP Quick Start 466 The SOAP Request 466 The SOAP Response 467 If Something Goes Wrong 468 Exposing a SOAP Interface to BittyWiki 468 Wiki Search-and-Replace Using the SOAP Web Service 470 Documenting Your Web Service API 472 Human-Readable API Documentation 473 The XML-RPC Introspection API 474 WSDL 475

27 Contents Choosing a Web Service Standard 478 Web Service Etiquette 479 For Consumers of Web Services 479 For Producers of Web Services 479 Using Web Applications as Web Services 480 Summary 480 Exercises 480 Chapter 21: Integrating Java with Python 481 Scripting within Java Applications 482 Comparing Python Implementations 483 Installing Jython 483 Running Jython 484 Running Jython Interactively 484 Running Jython Scripts 485 Controlling the jython Script 486 Making Executable Commands 487 Running Jython on Your Own 488 Packaging Jython-Based Applications 488 Integrating Java and Jython 489 Using Java Classes in Jython 489 Accessing Databases from Jython 494 Writing Java EE Servlets in Jython 500 Choosing Tools for Jython 506 Testing from Jython 506 Embedding the Jython Interpreter 507 Calling Jython Scripts from Java 508 Handling Differences between C-Python and Jython 510 Summary 511 Exercises 512 Part IV: Appendices 513 Appendix A: Answers to the Exercises 515 Appendix B: Online Resources 549 Appendix C: What s New in Python Appendix D: Glossary 559 Index 569 xxv

28

29 Introduction Welcome to Python 3.1! I ve been working with Python for about ten years now, and every new version has caused me to fall in love with the language all over again. Version 3.1 is no different. If you are new to Python, rest easy I ll guide you every step of the way. If, on the other hand, you are an old Python hand exploring the new version, the book is structured so that you can learn the new information you need, without wasting time on already-known information. I wanted to write this book because I love Python. I love it! And I want to share my love with you. And, maybe you ll grow to love it as I do. Who This Book Is For If you re computer-literate, and want to learn a fun programming language to better control your computer, this book is for you. If you are a system administrator who wants to learn a great language to help you better manage and configure systems and networks, this book is for you. If you already know Python, but are wondering what cool new features are available in version 3.1, this book is for you. In summary, this book is for anyone interested in exploring Python programming with the newest and most full-featured, easy-to-use version, 3.1. What This Book Covers This book is designed to cover Python 3.1. Python 3.1, released in 2009, is the latest major revision of the Python programming language. Since Python is a cross-platform language, the content and examples in the book are applicable in any platform (unless specified otherwise). When there is a choice to be made as to platform independence, the examples will be as cross-platform as possible. In addition, since Python 3.1 is relatively new, not all supporting libraries have been updated to work in Python 3.x. In those instances where this is the case and it is felt that the theory still needs to be expounded upon, Python 2.6 will be used in lieu of version 3.1.

30 Introduction How This Book Is Structured As might be expected from a Beginning book, the book begins with an introduction to the language. From there, you ll move through the core of the language, then move on to more advanced and specialized topics. The book is divided up into four parts. Part I Dipping Your Toe into Python The first part will allow you to, as the title suggests, dip your toe in. Programming Basics and Strings First you ll be introduced to Python. This chapter will explore what Python is, and why it is so useful and powerful. Also explored will be Python s history from its early development to the newest version, which is the focus of this book. You ll also learn about the scope of Python s reach, and all the different areas of application development in which Python plays a part. Finally, you ll learn to work with your first data type strings. Numbers and Operators This chapter will guide you through the basics of working with numbers and operators. You will learn the different types of numbers, how to perform simple and complex equations, and work with the various operators. You will also learn about order of precedence and formatting numbers. Variables Names for Values Ultimately, programming languages help you to manage different types of information in other words, data. An understanding of data types and how they are represented in Python is essential to programming in Python. This chapter will help you to understand the best ways to represent different data types in Python. Part II Python Language and the Standard Library Of course, the core piece of knowledge you need to use a language is to know the language itself, and familiarize yourself with its syntax and modules. This part will start small, with data types and variables, and gradually introduce additional concepts until you have all the information you need to develop fully functional Python programs. You ll want to read through these chapters sequentially each chapter builds on the information presented in the previous chapter. Making Decisions Ultimately, there will come a point when your program must make a decision do I take this path or that path? And what happens when I take that path? In this chapter, you will learn how to compare data, such as deciding if one value is greater than another, and use repetition to repeat repetitive tasks. xxviii