David J. Pine. Introduction to Python for Science & Engineering

Transcription

1 David J. Pine Introduction to Python for Science & Engineering

2 To Alex Pine who introduced me to Python

3 Contents Preface About the Author xi xv 1 Introduction Introduction to Python for Science and Engineering. 1 2 Launching Python Interacting with Python Installing Python on Your Computer The Spyder Window The IPython Pane Magic commands System shell commands Tab completion Recap of commands Interactive Python as a Calculator Binary arithmetic operations in Python Types of numbers Important note on integer division in Python Variables Names and the assignment operator Legal and recommended variable names Reserved words in Python Script Files and Programs First scripting example: The Editor pane Python Modules Python modules and functions: A first look Some NumPy functions Scripting Example Different ways of importing modules Getting Help: Documentation in IPython iii

4 iv Contents 2.10 Stand-alone IPython Writing Python scripts in a text editor Programming Errors Pyflakes Error checking Exercises Strings, Lists, Arrays, and Dictionaries Strings Lists Slicing lists The range function: Sequences of numbers Tuples Multidimensional lists and tuples NumPy Arrays Creating arrays (1-d) Mathematical operations with arrays Slicing and addressing arrays Fancy indexing: Boolean masks Multi-dimensional arrays and matrices Differences between lists and arrays Dictionaries Objects Exercises Input and Output Keyboard Input Screen Output Formatting output with str.format() Printing arrays File Input Reading data from a text file Reading data from an Excel file: CSV files File Output Writing data to a text file Writing data to a CSV file Exercises

5 Contents v 5 Conditionals and Loops Conditionals if, elif, and else statements Logical operators Loops for loops while loops Loops and array operations List Comprehensions Exercises Plotting An Interactive Session with PyPlot Basic Plotting Specifying line and symbol types and colors Error bars Setting plotting limits and excluding data Subplots Logarithmic Plots Semi-log plots Log-log plots More Advanced Graphical Output An alternative syntax for a grid of plots Plots with multiple axes Mathematics and Greek symbols The Structure of matplotlib: OOP and All That The backend layer The artist layer The PyPlot (scripting) layer Contour and Vector Field Plots Making a 2D grid of points Contour plots Streamline plots Three-Dimensional Plots Exercises Functions User-Defined Functions Looping over arrays in user-defined functions. 158

6 vi Contents Fast array processing for user-defined functions Functions with more than one input or output Positional and keyword arguments Variable number of arguments Passing function names and parameters as arguments Passing data (objects) to and from functions Variables and arrays created entirely within a function Passing lists and arrays to functions: Mutable and immutable objects Anonymous Functions: lambda Expressions NumPy Object Attributes: Methods and Instance Variables Example: Linear Least Squares Fitting Linear regression Linear regression with weighting: χ Exercises Curve Fitting Using Linear Regression for Fitting Nonlinear Functions Linear regression for fitting an exponential function Linear regression for fitting a power-law function Nonlinear Fitting Exercises Numerical Routines: SciPy and NumPy Special Functions Random Numbers Uniformly distributed random numbers Normally distributed random numbers Random distribution of integers Linear Algebra Basic computations in linear algebra Solving systems of linear equations Eigenvalue problems

7 Contents vii 9.4 Solving Nonlinear Equations Single equations of a single variable Solving systems of nonlinear equations Numerical Integration Single integrals Double integrals Solving ODEs Discrete (Fast) Fourier Transforms Continuous and discrete Fourier transforms The SciPy FFT library Exercises Data Manipulation and Analysis: Pandas Reading Data from Files Using Pandas Reading from Excel files saved as csv files Reading from text files Reading from an Excel file Dates and Times in Pandas Data Structures: Series and DataFrame Series DataFrame Getting Data from the Web Extracting Information from a DataFrame Plotting with Pandas Grouping and Aggregation The groupby method Iterating over groups Reformatting DataFrames Custom aggregation of DataFrames Exercises Animation Animating a Sequence of Images Simple image sequence Annotating and embellishing videos Animating Functions Animating for a fixed number of frames Animating until a condition is met Combining Videos with Animated Functions

8 viii Contents Using a single animation instance Combining multiple animation instances Exercises Python Classes and GUIs Defining and Using a Class The init () method Defining methods for a class Calling methods from within a class Updating instance variables Inheritance Graphical User Interfaces (GUIs) Event-driven programming PyQt A basic PyQt dialog Summary of PyQt5 classes used GUI summary A Installing Python 339 A.1 Installing Python A.1.1 Setting preferences A.1.2 Pyflakes A.1.3 Updating your Python installation A.2 Testing Your Installation of Python A.3 Installing FFmpeg for Saving Animations B Jupyter Notebooks 345 B.1 Launching a Jupyter Notebook B.2 Running Programs in a Jupyter Notebook B.3 Annotating a Jupyter Notebook B.3.1 Adding headings and text B.3.2 Comments with mathematical expressions B.4 Terminal commands in a Jupyter notebook B.5 Plotting in a Jupyter Notebook B.6 Editing and Rerunning a Notebook B.7 Quitting a Jupyter Notebook B.8 Working with an Existing Jupyter Notebook C Glossary 355

9 Contents ix D Python Resources 359 D.1 Python Programs and Data Files Introduced in This Text D.2 Web Resources D.3 Books Index 363

10

11 Preface The aim of this book is to provide science and engineering students a practical introduction to technical programming in Python. It grew out of notes I developed for various undergraduate physics courses I taught at NYU. While it has evolved considerably since I first put pen to paper, it retains its original purpose: to get students with no previous programming experience writing and running Python programs for scientific applications with a minimum of fuss. The approach is pedagogical and bottom up, which means starting with examples and extracting more general principles from that experience. This is in contrast to presenting the general principles first and then examples of how those general principles work. In my experience, the latter approach is satisfying only to the instructor. Much computer documentation takes a top-down approach, which is one of the reasons it s frequently difficult to read and understand. On the other hand, once examples have been seen, it s useful to extract the general ideas in order to develop the conceptual framework needed for further applications. In writing this text, I assume that the reader: has never programmed before; is not familiar with programming environments; is familiar with how to get around a Mac or PC at a very basic level; and is competent in basic algebra, and for Chapters 8 and 9, calculus, linear algebra, ordinary differential equations, and Fourier analysis. The other chapters, including 10 12, require only basic algebra skills. This book introduces, in some depth, four Python packages that are important for scientific applications: NumPy, short for Numerical Python, provides Python with a multidimensional array object (like a vector or matrix) that is at the center of virtually all fast numerical processing in scientific Python. xi

12 xii Introduction to Python for Science & Engineering It is both versatile and powerful, enabling fast numerical computation that, in some cases, approaches speeds close to those of a compiled language like C, C++, or Fortran. SciPy, short for Scientific Python, provides access through a Python interface to a very broad spectrum of scientific and numerical software written in C, C++, and Fortran. These include routines to numerically differentiate and integrate functions, solve differential equations, diagonalize matrices, take discrete Fourier transforms, perform least-squares fitting, as well as many other numerical tasks. matplotlib is a powerful plotting package written for Python and capable of producing publication-quality plots. While there are other Python plotting packages available, matplotlib is the most widely used and is the de facto standard. Pandas is a powerful package for manipulating and analyzing data formatted and labeled in a manner similar to a spreadsheet (think Excel). Pandas is very useful for handling data produced in experiments, and is particularly adept at manipulating large data sets in different ways. In addition, Chapter 12 provides a brief introduction to Python classes and to PyQt5, which provides Python routines for building graphical user interfaces (GUIs) that work on Macs, PCs, and Linux platforms. Chapters 1 7 provide the basic introduction to scientific Python and should be read in order. Chapters 8 12 do not depend on each other and, with a few mild caveats, can be read in any order. As the book s title implies, the text is focused on scientific uses of Python. Many of the topics that are of primary importance to computer scientists, such as object-oriented design, are of secondary importance here. Our focus is on learning how to harness Python s ability to perform scientific computations quickly and efficiently. The text shows the reader how to interact with Python using IPython, which stands for Interactive Python, through one of three different interfaces, all freely available on the web: Spyder, an integrated development environment, Jupyter Notebooks, and a simple IPython terminal. Chapter 2 provides an overview of Spyder and an introduction to IPython, which is a powerful interactive environment

13 Preface xiii tailored to scientific use of Python. Appendix B provides an introduction to Jupyter notebooks. Python 3 is used exclusively throughout the text with little reference to any version of Python 2. It s been nearly 10 years since Python 3 was introduced and there is little reason to write new code in Python 2; all the major Python packages have been updated to Python 3. Moreover, once Python 3 has been learned, it s a simple task to learn how Python 2 differs, which may be needed to deal with legacy code. There are many lucid web sites dedicated to this sometimes necessary but otherwise mind-numbing task. The scripts, programs, and data files introduced in this book are available at Finally, I would like to thank Étienne Ducrot, Wenhai Zheng, and Stefano Sacanna for providing some of the data and images used in Chapter 11, and Mingxin He and Wenhai Zheng for their critical reading of early versions of the text.

14

15 About the Author David Pine has taught physics and chemical engineering for over 30 years at four different institutions: Cornell University (as a graduate student), Haverford College, UCSB, and, at NYU, where he is a Professor of Physics, Mathematics, and Chemical & Biomolecular Engineering. He has taught a broad spectrum of courses, including numerical methods. He does research in experimental soft-matter physics, which is concerned with materials such as polymers, emulsions, and colloids. These materials constitute most of the material building blocks of biological organisms. xv

16

17 chapter 1 Introduction 1.1 Introduction to Python for Science and Engineering This book is meant to serve as an introduction to the Python programming language and its use for scientific computing. It s ok if you have never programmed a computer before. This book will teach you how to do it from the ground up. The Python programming language is useful for all kinds of scientific and engineering tasks. You can use it to analyze and plot data. You can also use it to numerically solve science and engineering problems that are difficult or even impossible to solve analytically. While we want to marshal Python s powers to address scientific problems, you should know that Python is a general purpose computer language that is widely used to address all kinds of computing tasks, from web applications to processing financial data on Wall Street and various scripting tasks for computer system management. Over the past decade it has been increasingly used by scientists and engineers for numerical computations, graphics, and as a wrapper for numerical software originally written in other languages, like Fortran and C. Python is similar to MATLAB, another computer language that is frequently used in science and engineering applications. Like MATLAB, Python is an interpreted language, meaning you can run your code without having to go through an extra step of compiling, as required for the C and Fortran programming languages. It is also a dynamically typed language, meaning you don t have to declare variables and set aside memory before using them. 1 Don t worry if you don t know exactly what these terms mean. Their primary significance for you is that you can write Python code, test, and use it quickly with a minimum of fuss. One advantage of Python compared to MATLAB is that it is free. It can be downloaded from the web and is available on all the standard computer platforms, including Windows, macos, and Linux. 1 Appendix C contains a glossary of terms you may find helpful. 1

18 2 Introduction to Python for Science & Engineering This also means that you can use Python without being tethered to the internet, as required for commercial software that is tied to a remote license server. Another advantage is Python s clean and simple syntax, including its implementation of object-oriented programming. This should not be discounted; Python s rich and elegant syntax renders a number of tasks that are difficult or arcane in other languages either simpler or more understandable in Python. An important disadvantage is that Python programs can be slower than compiled languages like C. For large-scale simulations and other demanding applications, there can be a considerable speed penalty in using Python. In these cases, C, C++, or Fortran is recommended, although intelligent use of Python s array processing tools contained in the NumPy module can greatly speed up Python code. Another disadvantage is that, compared to MATLAB, Python is less well documented. This stems from the fact that it is public open source software and thus is dependent on volunteers from the community of developers and users for documentation. The documentation is freely available on the web but is scattered among a number of different sites and can be terse. This manual will acquaint you with the most commonly used web sites. Search engines like Google can help you find others. You are not assumed to have had any previous programming experience. However, the purpose of this manual isn t to teach you the principles of computer programming; it s to provide a very practical guide to getting started with Python for scientific computing. Perhaps once you see some of the powerful tasks that you can accomplish with Python, you will be inspired to study computational science and engineering, as well as computer programming, in greater depth.