May 13, Mark Lutz Boulder, Colorado (303) [work] (303) [home]

Transcription

1 "Using Python": a Book Preview May 13, 1995 Mark Lutz Bouder, Coorado utz@kapre.com (303) [work] (303) [home] Introduction. This paper is a brief overview of the upcoming Python O'Reiy book I'm writing. I received a forma contract for the book in Apri. Roughy, the book wi be some 400 pages, and shoud be ready (at east in beta form) by fa, Athough we've spent quite a bit of time deveoping the current outine, everything mentioned here shoud be considered 'pre-apha' materia at best. In particuar, the outine is prone to frequent change (it wi probaby change by the time you read this!), and my editor and reviewers wi have a big infuence on the book's fina structure. So pease take a this with a reasonaby arge 'grain-of-sat'. Some of what gets covered in the book is sti open to input; in fact, this is one reason I'm making this information pubic right now. If you have suggestions, I'd be happy to isten. I may aso be asking for comments from experts in some of the domains the book addresses.

2 2 The Basic Approach. The goa of the book wi be to show how Python can be used for rea software deveopment tasks. It's not intended to be an exhaustive reference for every detai of the anguage. Instead, we want to show how Python can be appied to typica tasks faced by deveopers. Aong the way, we' present Python's major features, and show what it is about the anguage that makes it such a great too. In other words, this wi be a practica book for engineers, not a programming-anguage theory book. Of course, some theory wi be required, but that won't be the tone of the book. Some components of the approach: Organized by common roes, not anguage features Based on reaistic exampes, and incrementa improvements A very informa presentation (this is an O'Reiy book!) A practica book: not just a ist of syntax/semantic rues Detais on a need-to-know basis (to understand the exampes) Amost a "Python-by-exampe" book (but not reay...) Some of these goas may turn out to be difficut. For instance, "reaistic" exampes of ILU use, or WWW scripting might be too compex or arge for this book. Sti, we' try to avoid using overy-artificia code sampes, whenever possibe.

3 3 The Book's Structure. Essentiay, the book's structure just refects the approach we're taking. Because the book's goa is to present Python by its roes (not by its syntax or semantics), its structure refects ways that Python can be used in practice. Here's some genera notes; the outine beow gives specific detais. One section for each of Python's major roes Reaistic exampes of Python-at-work for each roe Language detais presented in the context of rea exampes A comprehensive index serves as a anguage reference Appendices for the most-common reference materia An optiona tutoria introduces basic anguage issues

4 4 The (current) Outine. The outine (as presented to O'Reiy editors, more or ess) appears beow. The first 2 sections (she and extension-anguage programming) are probaby the most critica, and the ongest, since they' aso cover much of the Python anguage, as an incidenta topic. The parts foowing these 2 are roughy seected popuar Python appication areas. Note: the outines of parts not yet written are sti very incompete. Summary... FOREWORD. PREFACE. 1. INTRODUCTION (15 pages). 2. A PYTHON TUTORIAL (50 pages; may be an appendix instead). 3. SHELL PROGRAMMING (50 pages). 4. EXTENSION-LANGUAGE PROGRAMMING (50 pages). 5. GUI PROGRAMMING (30 pages; may be moved to part 4). 6. DATA-STRUCTURE PROGRAMMING (30 pages). 7. WWW/INTERNET SCRIPTING TOOLS (30 pages). 8. LANGUAGE-BASED TOOLS (30 pages). 9. CLIENT/SERVER PROGRAMMING (30 pages). 10. KNOWLEDGE-BASED PROGRAMMING (30 pages). 11. CONCLUSION (10 pages). 12. APPENDICES (50 pages). Detais... FOREWORD. Guido; he' aso be a major reviewer for the book (in a his copious spare time :-). PREFACE. The book's structure, about the exampes (run on Linux), acknowedgments, etc. 1. INTRODUCTION (15 pages). Aready written. A short description of Python's roes, what peope ike about it, and a sneak-preview of some exampes we' ook at in the book. Here's the current subsections of the introduction:

5 5 Introduction So what's Python? What's a the excitement about? Some of Python's features What's Python good for? A "Sneak Preview" Data structures System administration GUI programming Extension-anguage work Concusion 2?. A PYTHON TUTORIAL (50 pages; here, or an appendix). Aready written. It's not yet cear whether the tutoria wi appear first (here) or as an appendix at the end (as optiona reading for beginners). There's different schoos-of-thought on this; my current idea is to put this first, to avoid cuttering up the 'roe' chapters that foow. On the other hand, it uses artificiay sma exampes to iustrate anguage features, and may move too sow as is. Stay tuned... ;-) Introduction Getting started Interactive use Simpe expressions and assignment Some usefu statements The 'if' seection Syntax rues The 'whie' oop The 'for' oop, part 1... Using buit-in data structures Lists range() Dictionaries Tupes Conversions Genera features Nesting Object references Generic operations Sicing Growing and shrinking Using Functions Basic concepts Functions are objects Arguments Recursion

6 6 Moduar design Goba and oca variabes Using Modues Basic concepts Running code in modues Modues are name-spaces Buit-in modues Reoading modues Modue export rues Compied modues Using casses Basic concepts Casses are name-spaces too Instances versus casses Inheritance, poymorphism, and encapsuation Constructors and destructors Subcass speciaization Operator overoading Inheritance versus composition Metacass protoco methods Impementing 'records' The stack cass Name scoping rues Program organization Exception handing The 'try' and 'raise' statements System exceptions Nesting exception handers Extending and embedding in C C extension modue format Running embedded strings Wrapping C++ casses Other integration ideas Using buit-in toos Python system structure Pre-coded functionaity: functions, types, modues Summary (each subsection has a summary too) Concusion 3. SHELL PROGRAMMING (50 pages). In progress. This section uses she-scripts to get started, and show basic concepts. Fie processing, and system-component access appears in the first exampe we ook at; these are critica topics to the target audience. The section starts with very simpe tasks, and deveops them into more sophisticated programs.

7 7 We start with a 6-ine text-fie packer; eventuay, we wind up with a simpe appication cass framework to encapsuate top-eve components. Aong the way, we introduce anguage-features needed to understand the exampes. Composed of mutipe chapters: Introduction Basic system administration 'Quick-and-dirty' fie packing scripts Improving on the pack scripts: step 1 Improving on the pack scripts: a functiona API A 'quick-and-dirty' mai-fie scanner Wrapping top-eve components The App cass Adding i/o stream redirection Adding interna (string) fies Adding interactive oops and menus Mixing mutipe Appication casses Testing the framework A menu-based appication pack, unpack, and mai-too scripts as appication objects Optimizing the mai-too by subcassing Adding fie backups to unpack An interactive interface for our she toos Summary/Concusion Compex topics, such as mutipe inheritance, wi appear in the context of rea program needs: to derive a subcass that supports both stdio redirection, and a command menu. We' aso introduce the debugger, using intentiona errors. Basic profier use comes up here too: we improve on a mai-too cass's performance be a factor of 3. More detais on this section appear beow. 4. EXTENSION-LANGUAGE PROGRAMMING (50 pages). An exampe of using Python as an embedded anguage, for on-site customization of a C/C++ ibraries and binaries. Deveops basic embedded ca functions, and shows extension modues and types by exampe. This section won't be an exhaustive reference for a run-time functions; the goa is just to show some ways that embedding/extending can be done. The section's exampe probems make use of both C->Python (embedding) and Python->C (extending) interfaces. Its current parts: Introduction The probem: on-site configurations Running embedded Python code As strings: expressions and statements As fies: by modue/object names As registered caabe objects As object methods

8 8 As UNIX scripts Conversions: sending inputs, getting resuts Error-handing Debugging and reoading embedded code An embedded-ca API A Python object API Caing C extensions from Python Extension modues Extension data-types Wrapping C++ casses Conversions: sending inputs, getting resuts Error-handing Code generation for integrating API's ('bgen') Buiding with Python Linking Python into an appication Linking extensions into Python Dynamic and static binding Shipping Python code: source, compied, frozen Concusion This section might aso present Guido's new 'bgen' too, which generates code needed to integrate a C API/ibrary into Python, by scanning header fies. It may aso buid some higher-eve embedded-ca functions (on top of existing ow-eve interfaces), uness these become a standard part of Python in the next reease. 5. GUI PROGRAMMING (30 pages) (might be moved to part 4). Using Tkinter: Python's object-oriented interface to TCL/Tk. Looks at some simpe GUI programs ('heo word', a simpe mai-fie browser), but defers to TCL/Tk documentation for more information. This section might be moved ahead to part 4 (between scripts and embedding); it wi have a powerfu visua impact. GUI options: Tk, X, MFC, Pesto/Fresco, portabiity,... The 'Heo word' program in Tkinter Basic use: widgets as functions Adding event caback handers Attaching widgets to frames Making GUI's by subcassing Pop-up diaog boxes Menus and too-bars Spawning new windows and processes A simpe text editor A GUI front-end for mai-too (part 3) Using GUI toos in embedded code (part 4) GuiApp: a subcass of the appication framework (part 3)

9 9 RAD: changing the GUI without stopping or rebuiding Tkinter as an extension exampe (part 4) Optiona: a GUI front-end for our she toos (part 3) Tkinter wi be used in ater chapters too. Rapid GUI deveopment is fast becoming one of Python's strongest points. Unfortunatey (fortunatey?), there's a ot of options right now; since Tk seems to be dominant in the UNIX pubic-domain arena (O'Reiy's market), it wi probaby get the most exposure. But I'm open to other opinions. 6. DATA-STRUCTURE PROGRAMMING (30 pages). Deveops a data-structure cass ibrary, to demonstrate more OOP in Python. The simpe framework wi hande sets, and wi grow to incude trees, graphs, etc. Aong the way, we' ook at the profier, for optimizing sets. Set-processing functions A simpe set cass (with a generic supercass..) Handing sequences genericay: reverse, permute,.. Optimizing set performance using dictionaries Adding trees and graphs The ReSet subcass: adding reationa agebra The GuiSet subcass: adding GUI dispay protoco The FieSet subcass: random access fies as sets Making objects persistent: 'picke' and 'sheve' SQL queries in Python Optiona: Database interfaces Aso ooks at persistent objects (a new feature of Python); if pressed for space, this is the most important topic here. May aso discuss other Python database interfaces: dbm, orace, sybase, etc. (optiona). 7. WWW/INTERNET SCRIPTING TOOLS (30 pages). Python 1.2 has a set of WWW toos bunded with it, as a standard part of the anguage. Given WWW's popuarity, the book wi devote a whoe part to Python's WWW connection. Under construction LANGUAGE-BASED TOOLS (30 pages). Buids a cacuator program, to show Python as a stand-aone anguage. Aso ooks at a command-ine interpreter, etc.

10 10 'cac': a simpe cacuator program (parser/evauator) Adding an unparser Adding an interpreter: cass instance trees A GUI front-end for 'cac' Moving the scanner to C Embedded Python + API's: versus home-grown anguages Optiona: a command-ine interpreter ('psh') Optiona: the 'guegen' parser Optiona: the Python parser modue Optiona: the regex/regsub modues 9. CLIENT/SERVER PROGRAMMING (30 pages). Aso known as 'distributed computing/objects'. Looks at pipes, threads, sockets. May aso buid up a simpe 'visua python' environment, using modues deveoped in earier chapters (a python server process, a GUI cient, text editor...), to tie-together some earier concepts into a compete system. Forking new Python processes: fork, pipes, sockets The 'cac' program as cient/server system (a GUI cient) ILU: anguage/machine-independent interfaces Optiona: A "Visua Python" environment This section wi aso give a brief overview of the Python/ILU system: roughy, a pubic-domain CORBA impementation, which supports Python, Modua-3, and Common-Lisp. This is becoming a popuar cient/server medium. 10. KNOWLEDGE-BASED PROGRAMMING (30 pages). Very optiona. Presents a simpified version of an expert-system she I deveoped ('Homes'), as an exampe of stand-aone Python programming. Incudes a simpe forward and backward chaining inference engine, simpified pattern matcher, and a rue index-tree cass. Very ikey to be cut; I ony incude it in case I wind up with some free time/space at the end. 11. CONCLUSION (10 pages). More on Python's roes in the deveopment cyce, now that we've shown how to use it. Hybrid systems, the deveopment botteneck, the Giigan Factor,...

11 APPENDICES (50 pages). The goa here isn't to incude a reference manua: that's what the index (and Python's standard documentation) is for. Instead, we' ony incude some of the more critica information not presented esewhere in the text. A Python tutoria (if not earier; see detais above) Common programming mistakes (from questions on the net) Common ibrary (buit-in) functions, modues, and exceptions Syntax definition (charts?) Some patform-dependencies (but not buid detais) A "peek under the hood": some design/internas notes Sources for more information (news, mai, ftp, WWW, PSA, etc.) Where to get the exampes in the book A quick-reference (a card?) Index Since the index serves as a reference, it must be compete. See above: the tutoria might appear earier.

12 12 More about the She programming section. This section opens with a rea-ife scenario: packing/unpacking text fies for transfer over a modem. The exampes are based on scripts I actuay needed to write from home; the mai-too aso represents a reaife need. The 'pack' program begins as a simpe 6-ine script: #!/usr/oca/bin/python import sys marker = '::::::' for name in sys.argv[1:]: input = open(name, 'r') print marker + name print input.read() which we' ook at ine-by-ine, since this is the first Python code in the book. 'pack' ater evoves to incude a functiona API, for use from other modues: #!/usr/oca/bin/python # use: pack_a('target', ['src', 'src']) # use: % pack src src... # use: % pack -b target src src... from sys import * from textpack import marker def pack_fie(name, output): input = open(name, 'r') output.write(marker + name + '\n') whie 1: ine = input.readine() if not ine: break output.write(ine) def pack_a(output, sources): if type(output) == type("string"): output = open(output, 'w') for name in sources: try: print 'packing:', name pack_fie(name, output) except: print 'error processing:', name; exit(1) if name == ' main ': try: if en(argv) >= 4 and argv[1] == '-b': pack_a(argv[2], argv[3:]) ese: pack_a(stdout, argv[1:]) except: print 'error opening output'; exit(1)

13 13 and finay, becomes an instance of the 'App' appication cass, inheriting input/output stream redirection, command ine processing, etc., after we've ooked at the framework: #!/usr/oca/bin/python # use: app.appca(packapp, args...) # use: % packapp.py -o target src src... from app import App from textpack import marker cass PackApp(App): def start(sef): if not sef.args: sef.exit('packapp.py [-o target]? src src...') ese: sef.setoutput() def run(sef): for name in sef.restargs(): try: sef.message('packing: ' + name) sef.pack_fie(name) except: sef.exit('error processing:' + name) def pack_fie(sef, name): sef.setinput(name) sef.write(marker + name + '\n') whie 1: ine = sef.readine() if not ine: break sef.write(ine) if name == ' main ': PackApp().main() Aong the way, each step in pack's evoution wi improve its functionaity in some way. Before this part is over, we' have introduced much of the Python anguage, indirecty. New anguage features in the exampes wi be discussed as needed. For exampe, object-oriented-programming comes up, as a means of wrapping top-eve components. The 'unpack' and 'mai-too' scripts simiary evove. For instance, 'unpack' starts out simpe: #!/usr/oca/bin/python from sys import * marker = '::::::' for ine in stdin.readines(): if ine[:6]!= marker: print ine[:-1] ese: stdout = open(ine[6:-1], 'w')

14 14 and then moves to a variant that doesn't have to oad the whoe packed fie into memory a at once: #!/usr/oca/bin/python import sys marker = '::::::' whie 1: ine = sys.stdin.readine() if not ine: break eif ine[:6]!= marker: print ine[:-1] ese: sys.stdout = open(ine[6:-1], 'w') and finay to an 'App' subcass. 'mai-too' is ony presented as a simpe script, and then as an 'App' instance. Mai-too extracts messages from a mai-box fie, based on simpe header-ine matching. Other book exampes are avaiabe on request; they' a be ftp'abe eventuay.