Python in Scientific Computing

Transcription

1 Jussi Enkovaara import sys, os try: from Bio.PDB import PDBParser biopython_installed = True except ImportError: biopython_installed = False Python in Scientific Computing June 10, 2014 Scientific computing in practice Aalto University default_bfactor = 0.0 default_occupancy = 1.0 default_segid = '' # default B-factor # default occupancy level # empty segment ID class EOF(Exception): def init (self): pass class FileCrawler: """ Crawl through a file reading back and forth without loading anything to memory. """ def init (self, filename): try: self. fp = open(filename) except IOError: raise ValueError, "Couldn't open file '%s' for reading." % filename self.tell = self. fp.tell self.seek = self. fp.seek def prevline(self): try: self.prev()

2 All material (C) 2014 by the authors. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License,

3 Agenda 9:00-9:45 NumPy fast array interface to Python 9:45-10:30 Exercises 10:30-10:45 Break 10:45-11:15 NumPy (continued) 11:15-12:15 Exercises 12:15-13:00 Lunch break 13:00-13:30 Matplotlib-package for visualization 13:30-14:00 Exercises 14:00-14:30 Scipy-package for scientific computing 14:30:15:00 Exercises 15:00-15:15 Break 15:15-15:45 Parallel programming with mpi4py 15:45-17:00 Exercises

4

5 NUMPY

6

7 Numpy fast array interface Standard Python is not well suitable for numerical computations lists are very flexible but also slow to process in numerical computations Numpy adds a new array data type static, multidimensional fast processing of arrays some linear algebra, random numbers Numpy arrays All elements of an array have the same type Array can have multiple dimensions The number of elements in the array is fixed, shape can be changed Python list vs. NumPy array Creating numpy arrays Python list Memory layout NumPy array Memory layout From a list: import numpy as np a = np.array((1, 2, 3, 4), float) a array([ 1., 2., 3., 4.]) list1 = [[1, 2, 3], [4,5,6]] mat = np.array(list1, complex) mat array([[ 1.+0.j, 2.+0.j, 3.+0.j], [ 4.+0.j, 5.+0.j, 6.+0.j]]) mat.shape (2, 3) mat.size 6 Creating numpy arrays More ways for creating arrays: import numpy as np a = np.arange(10) a array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) b = np.linspace(-4.5, 4.5, 5) b array([-4.5, -2.25, 0., 2.25, 4.5 ]) c = np.zeros((4, 6), float) c.shape (4, 6) d = np.ones((2, 4)) d array([[ 1., 1., 1., 1.], [ 1., 1., 1., 1.]]) Simple indexing: Slicing: Indexing and slicing arrays mat = np.array([[1, 2, 3], [4, 5, 6]]) mat[0,2] 3 mat[1,-2] 5 a = np.arange(5) a[2:] array([2, 3, 4]) a[:-1] array([0, 1, 2, 3]) a[1:3] = -1 a array([0, -1, -1, 3, 4]) Indexing and slicing arrays Slicing is possible over all dimensions: a = np.arange(10) a[1:7:2] array([1, 3, 5]) a = np.zeros((4, 4)) a[1:3, 1:3] = 2.0 a array([[ 0., 0., 0., 0.], [ 0., 2., 2., 0.], [ 0., 2., 2., 0.], [ 0., 0., 0., 0.]]) Views and copies of arrays Simple assignment creates references to arrays Slicing creates views to the arrays Use copy() for real copying of arrays example.py a = np.arange(10) b = a # reference, changing values in b changes a b = a.copy() # true copy c = a[1:4] # view, changing c changes elements [1:4] of a c = a[1:4].copy() # true copy of subarray

8 Array manipulation reshape : change the shape of array mat = np.array([[1, 2, 3], [4, 5, 6]]) mat array([[1, 2, 3], [4, 5, 6]]) mat.reshape(3,2) array([[1, 2], [3, 4], [5, 6]]) ravel : flatten array to 1-d mat.ravel() array([1, 2, 3, 4, 5, 6]) Array manipulation concatenate : join arrays together mat1 = np.array([[1, 2, 3], [4, 5, 6]]) mat2 = np.array([[7, 8, 9], [10, 11, 12]]) np.concatenate((mat1, mat2)) array([[ 1, 2, 3], [ 4, 5, 6], [ 7, 8, 9], [10, 11, 12]]) np.concatenate((mat1, mat2), axis=1) array([[ 1, 2, 3, 7, 8, 9], [ 4, 5, 6, 10, 11, 12]]) split : split array to N pieces np.split(mat1, 3, axis=1) [array([[1], [4]]), array([[2], [5]]), array([[3], [6]])] Array operations Most operations for numpy arrays are done elementwise +, -, *, /, ** a = np.array([1.0, 2.0, 3.0]) b = 2.0 a * b array([ 2., 4., 6.]) a + b array([ 3., 4., 5.]) a * a array([ 1., 4., 9.]) Array operations Numpy has special functions which can work with array arguments sin, cos, exp, sqrt, log,... import numpy, math a = numpy.linspace(-math.pi, math.pi, 8) a array([ , , , , , , , ]) numpy.sin(a) array([ e-16, e-01, e-01, e-01, e-01, e-01, e-01, e-16]) math.sin(a) Traceback (most recent call last): File "<stdin>", line 1, in? TypeError: only length-1 arrays can be converted to Python scalars Vectorized operations for loops in Python are slow Use vectorized operations when possible Example: difference example.py # brute force using a for loop arr = np.arange(1000) dif = np.zeros(999, int) for i in range(1, len(arr)): dif[i-1] = arr[i] - arr[i-1] # vectorized operation arr = np.arange(1000) dif = arr[1:] - arr[:-1] for loop is ~80 times slower! Broadcasting If array shapes are different, the smaller array may be broadcasted into a larger shape from numpy import array a = array([[1,2],[3,4],[5,6]], float) a array([[ 1., 2.], [ 3., 4.], [ 5., 6.]]) b = array([[7,11]], float) b array([[ 7., 11.]]) a * b array([[ 7., 22.], [ 21., 44.], [ 35., 66.]]) Advanced indexing Numpy arrays can be indexed also with other arrays (integer or boolean) x = np.arange(10,1,-1) x array([10, 9, 8, 7, 6, 5, 4, 3, 2]) x[np.array([3, 3, 1, 8])] array([7, 7, 9, 2]) Boolean mask arrays m = x > 7 m array([ True, True, True, False, False,... x[m] array([10, 9, 8]) Advanced indexing creates copies of arrays Masked arrays Sometimes datasets contain invalid data (faulty measurement, problem in simulation) Masked arrays provide a way to perform array operations neglecting invalid data Masked array support is provided by numpy.ma module

9 Masked arrays Masked arrays can be created by combining a regular numpy array and a boolean mask import numpy.ma as ma x = np.array([1, 2, 3, -1, 5]) m = x < 0 mx = ma.masked_array(x, mask=m) mx masked_array(data = [ ], mask = [False False False True False], fill_value = ) x.mean() 2.0 mx.mean() 2.75 I/O with Numpy Numpy provides functions for reading data from file and for writing data into the files Simple text files numpy.loadtxt numpy.savetxt Data in regular column layout Can deal with comments and different column delimiters Random numbers The module numpy.random provides several functions for constructing random arrays random: uniform random numbers normal: normal distribution poisson: Poisson distribution... import numpy.random as rnd rnd.random((2,2)) array([[ , ], [ , ]]) rnd.poisson(size=(2,2)) Polynomials Polynomial is defined by array of coefficients p p(x, N) = p[0] x N-1 + p[1] x N p[n-1] Least square fitting: numpy.polyfit Evaluating polynomials: numpy.polyval Roots of polynomial: numpy.roots... x = np.linspace(-4, 4, 7) y = x**2 + rnd.random(x.shape) p = np.polyfit(x, y, 2) p array([ , , ]) Linear algebra Numpy can calculate matrix and vector products efficiently: dot, vdot,... Eigenproblems: linalg.eig, linalg.eigvals, Linear systems and matrix inversion: linalg.solve, linalg.inv A = np.array(((2, 1), (1, 3))) B = np.array(((-2, 4.2), (4.2, 6))) C = np.dot(a, B) b = np.array((1, 2)) np.linalg.solve(c, b) # solve C x = b array([ , ]) Matrix multiplication C = A * B matrix dimension 200 pure python: 5.30 s naive C: 0.09 s numpy.dot: 0.01 s Numpy performance Summary Numpy provides a static array data structure Multidimensional arrays Fast mathematical operations for arrays Arrays can be broadcasted into same shapes Tools for linear algebra and random numbers

10

11 MATPLOTLIB

12

13 Matplotlib 2D plotting library for python Can be used in scripts and in interactive shell Publication quality in various hardcopy formats Easy things easy, hard things possible Some 3D functionality Matplotlib interfaces Simple command style functions similar to Matlab plot.py import pylab as pl... pl.plot(x, y) Powerful object oriented API for full control of plotting Basic concepts Figure: the main container of a plot Axes: the plotting area, a figure can contain multiple Axes graphical objects: lines, rectangles, text Command style functions are used for creating and manipulating figures, axes, lines,... The command style interface is stateful: track is kept about current figure and plotting area Simple plot plot : create a simple plot. Figure and axes are created if needed Multiple subplots Histograms subplot : create multiple axes in the figure and switch between subplots hist : create histogram Latex can be used with matplotlib Bar and pie charts 3D plots bar : bar charts pie : pie charts 3 D axes is created with Axes3D from the mpl_toolkits.mplot3d module

14 Summary of basic functions Simple plot: plot Multiple plots: subplot Histograms: hist Bar charts: bar Pie charts: pie Switch plotting on top of existing figure: hold Contour plots: contour, contourf Summary Matplotlib provides a simple command style interface for creating publication quality figures Interactive plotting and different output formats (.png,.pdf,.eps) Simple plots, multiplot figures, decorations Possible to use Latex in text

15 SCIPY

16

17 Scipy Scientific tools for Python Library overview Scipy is a Python package containing several tools for scientific computing Modules for: statistics, optimization, integration, interpolation linear algebra, Fourier transforms, signal and image processing ODE solvers, special functions... Vast package, reference guide is currently 975 pages Scipy is built on top of Numpy Clustering package (scipy.cluster) Constants (scipy.constants) Fourier transforms (scipy.fftpack) Integration and ODEs (scipy.integrate) Interpolation (scipy.interpolate) Input and output (scipy.io) Linear algebra (scipy.linalg) Maximum entropy models (scipy.maxentropy) Miscellaneous routines (scipy.misc) Multi-dimensional image processing (scipy.ndimage) Orthogonal distance regression (scipy.odr) Optimization and root finding (scipy.optimize) Signal processing (scipy.signal) Sparse matrices (scipy.sparse) Sparse linear algebra (scipy.sparse.linalg) Spatial algorithms and data structures (scipy.spatial) Special functions (scipy.special) Statistical functions (scipy.stats) Image Array Manipulation and Convolution (scipy.stsci) C/C++ integration (scipy.weave) Integration Routines for numerical integration single, double and triple integrals Function to integrate can be given by function object or by fixed samples integrate.py from scipy.integrate import simps, quad, inf x = np.linspace(0, 1, 20) y = np.exp(-x) int1 = simps(y, x) # integrate function given by samples def f(x): return exp(-x) Optimization Several classical optimization algorithms Quasi-Newton type optimizations Least squares fitting Simulated annealing General purpose root finding... from scipy.optimize import fmin int2 = quad(f, 0, 1) # integrate function object int3 = quad(f, 0, inf) # integrate up to infinity Special functions Scipy contains huge set of special functions Bessel functions Legendre functions Gamma functions... from scipy.special import jv, gamma Linear algebra Wider set of linear algebra operations than in Numpy decompositions, matrix exponentials Routines also for sparse matrices storage formats iterative algorithms sparse.py import numpy as np from scipy.sparse.linalg import LinearOperator, cg # "Sparse" matrix-vector product def mv(v): return np.array([ 2*v[0], 3*v[1]]) A = LinearOperator( (2,2), matvec=mv, dtype=float ) b = np.array((4.0, 1.0)) x = cg(a, b) # Solve linear equation Ax = b with conjugate gradient Summary Scipy is vast package of tools for scientific computing Uses lots of NumPy in the background Numerical integration, optimization, special functions, linear algebra,... Look Scipy documentation for finding tools for your needs!

18

19 PARALLEL PROGRAMMING WITH PYTHON USING MPI4PY

20

21 Outline Brief introduction to message passing interface (MPI) Python interface to MPI mpi4py Performance considerations Message passing interface MPI is an application programming interface (API) for communication between separate processes The most widely used approach for distributed parallel computing MPI programs are portable and scalable the same program can run on different types of computers, from PC's to supercomputers MPI is flexible and comprehensive large (over 120 procedures) concise (often only 6 procedures are needed) MPI standard defines C and Fortran interfaces mpi4py provides (an unofficial) Python interface Execution model in MPI Parallel program is launched as set of independent, identical processes Parallel program Process 1 Process 2... Process N All the processes contain the same program code and instructions Processes can reside in different nodes or even in different computers The way to launch parallel program is implementation dependent mpirun, mpiexec, aprun, poe,... When using Python, one launches N Python interpreters mpirun -np 32 python parallel_script.py MPI Concepts rank: id number given to process it is possible to query for rank processes can perform different tasks based on their rank mpi.py if (rank == 0): # do something elif (rank == 1): # do something else else: # all other processes do something different MPI Concepts Communicator: group containing process in mpi4py the basic object whose methods are called MPI_COMM_WORLD contains all the process (MPI.COMM_WORLD in mpi4py) Data model All variables and data structures are local to the process Processes can exchange data by sending and receiving messages Process 1 (rank 0) a = 1.0 b = 2.0 Process 2 (rank 1) a = -1.0 b = -2.0 MPI messages Process N (rank N-1) a = 6.0 b = Using mpi4py Basic methods of communicator object Get_size() Number of processes in communicator Get_rank() rank of this process mpi.py from mpi4py import MPI comm = MPI.COMM_WORLD # communicator object containing all processes size = comm.get_size() rank = comm.get_rank() Sending and receiving data Sending and receiving a dictionary mpi.py from mpi4py import MPI comm = MPI.COMM_WORLD # communicator object containing all processes rank = comm.get_rank() if rank == 0: data = {'a': 7, 'b': 3.14} comm.send(data, dest=1, tag=11) elif rank == 1: data = comm.recv(source=0, tag=11) print "I am rank %d in group of %d processes" % (rank, size)

22 Sending and receiving data Arbitrary Python objects can be communicated with the send and receive methods of communicator send(data, dest, tag) data Python object to send dest destination rank tag id given to the message recv(source, tag) source source rank tag id given to the message data is provided as return value Destination and source ranks as well as tags have to match Communicating NumPy arrays Arbitrary Python objects are converted to byte streams when sending Byte stream is converted back to Python object when receiving Conversions give overhead to communication (Contiguous) NumPy arrays can be communicated with very little overhead with upper case methods: Send(data, dest, tag) Recv(data, source, tag) Note the difference in receiving: the data array has to exist in the time of call Communicating NumPy arrays Sending and receiving a NumPy array mpi.py from mpi4py import MPI Ping-pong test mpi4py performance comm = MPI.COMM_WORLD rank = comm.get_rank() if rank == 0: data = numpy.arange(100, dtype=numpy.float) comm.send(data, dest=1, tag=13) elif rank == 1: data = numpy.empty(100, dtype=numpy.float) comm.recv(data, source=0, tag=13) Note the difference between upper/lower case! send/recv: general Python objects, slow Send/Recv: continuous arrays, fast Summary mpi4py provides Python interface to MPI MPI calls via communicator object Possible to communicate arbitrary Python objects NumPy arrays can be communicated with nearly same speed as from C/Fortran