Astronomical Data Analysis with Python

Size: px

Start display at page:

Download "Astronomical Data Analysis with Python"

Jeffrey Franklin
5 years ago
Views:

1 Astronomical Data Analysis with Python Lecture 8 Yogesh Wadadekar NCRA-TIFR July August 2010 Yogesh Wadadekar (NCRA-TIFR) Topical course 1 / 27

2 Slides available at: yogesh/python_course_2010/ Yogesh Wadadekar (NCRA-TIFR) Topical course 2 / 27

3 Comments on assignment Yogesh Wadadekar (NCRA-TIFR) Topical course 3 / 27

4 SciPy - superset of numpy constants: physical constants and conversion factors (since version 0.7.0[1]) cluster: hierarchical clustering, vector quantization, K-means fftpack: Discrete Fourier Transform algorithms integrate: numerical integration routines interpolate: interpolation tools io: data input and output lib: Python wrappers to external libraries linalg: linear algebra routines misc: miscellaneous utilities (e.g. image reading/writing) optimize: optimization algorithms including linear programming signal: signal processing tools sparse: sparse matrix and related algorithms spatial: KD-trees, nearest neighbors, distance functions Yogesh Wadadekar (NCRA-TIFR) Topical course 4 / 27

5 SciPy special: special functions stats: statistical functions weave: tool for writing C/C++ code as Python multiline strings ndimage: image processing Via RPy, SciPy can interface to the R statistical package R. Yogesh Wadadekar (NCRA-TIFR) Topical course 5 / 27

6 SciPy Cookbook This page hosts worked examples of commonly-done tasks. Some are introductory in nature, while others are quite advanced (these are at the bottom of the page). Yogesh Wadadekar (NCRA-TIFR) Topical course 6 / 27

7 power of matplotlib import numpy as np x, y = np.random.randn(2, ) H, xedges, yedges = np.histogram2d(x, y, bins=50) extent = [xedges[0], xedges[-1], yedges[0], yedges[-1]] imshow(h, extent=extent) Yogesh Wadadekar (NCRA-TIFR) Topical course 7 / 27

8 Three main reasons to extend Python 1 speed. 2 to leverage existing libraries that are not already wrapped. 3 to allow Python programs to access devices at absolute memory addresses, using C functions as an intermediary. If neither of these reasons apply, you can happily code only in Python. Yogesh Wadadekar (NCRA-TIFR) Topical course 8 / 27

9 Questions to answer before trying to extend Python is the compiled language code you need small enough to recode in Python. e.g. a Numerical recipes routine that requires perhaps 3-4 other routines should be easily rewritable in Python. Yogesh Wadadekar (NCRA-TIFR) Topical course 9 / 27

10 Questions to answer before trying to extend Python is the compiled language code you need small enough to recode in Python. e.g. a Numerical recipes routine that requires perhaps 3-4 other routines should be easily rewritable in Python. if the code is large, check whether it already has a Python interface. e.g. LAPACK Yogesh Wadadekar (NCRA-TIFR) Topical course 9 / 27

11 Questions to answer before trying to extend Python is the compiled language code you need small enough to recode in Python. e.g. a Numerical recipes routine that requires perhaps 3-4 other routines should be easily rewritable in Python. if the code is large, check whether it already has a Python interface. e.g. LAPACK if the code in question is in octave, idl, matlab, mathematica and you have a licensed copy of these software, it will be easier to use the interfaces to those languages. Yogesh Wadadekar (NCRA-TIFR) Topical course 9 / 27

12 Questions to answer before trying to extend Python is the compiled language code you need small enough to recode in Python. e.g. a Numerical recipes routine that requires perhaps 3-4 other routines should be easily rewritable in Python. if the code is large, check whether it already has a Python interface. e.g. LAPACK if the code in question is in octave, idl, matlab, mathematica and you have a licensed copy of these software, it will be easier to use the interfaces to those languages. Do you really need to pass arrays or are disk files a good way of exchanging data? Yogesh Wadadekar (NCRA-TIFR) Topical course 9 / 27

13 Questions to answer before trying to extend Python is the compiled language code you need small enough to recode in Python. e.g. a Numerical recipes routine that requires perhaps 3-4 other routines should be easily rewritable in Python. if the code is large, check whether it already has a Python interface. e.g. LAPACK if the code in question is in octave, idl, matlab, mathematica and you have a licensed copy of these software, it will be easier to use the interfaces to those languages. Do you really need to pass arrays or are disk files a good way of exchanging data? It is more efficient to wrap a set of related subroutines rather than just one (or a few). Can you collaborate to do this? Ask on astropy mailing list. Yogesh Wadadekar (NCRA-TIFR) Topical course 9 / 27

14 Numpy C extensions Yogesh Wadadekar (NCRA-TIFR) Topical course 10 / 27

15 SWIG SWIG is a software development tool that connects programs written in C and C++ with a variety of high-level programming languages. SWIG is used with different types of languages such as Perl, PHP, Python, Tcl, Ruby, C#, Common Lisp (CLISP, Allegro CL, CFFI, UFFI), Java, Lua, Modula-3, OCAML, Octave and R. Also several interpreted and compiled Scheme implementations (Guile, MzScheme, Chicken) are supported. SWIG is most commonly used to create high-level interpreted or compiled programming environments, user interfaces, and as a tool for testing and prototyping C/C++ software. Yogesh Wadadekar (NCRA-TIFR) Topical course 11 / 27

16 The example.c file Yogesh Wadadekar (NCRA-TIFR) Topical course 12 / 27

17 The example.i interface file Now, in order to add these files to your favorite language, you need to write an interface file which is the input to SWIG. An interface file for the C functions we want to wrap is needed. Yogesh Wadadekar (NCRA-TIFR) Topical course 13 / 27

18 Building the module $ swig -python example.i $ gcc -c example.c example_wrap.c -I/usr/include/python2.6 $ ld -shared example.o example_wrap.o -o _example.so Yogesh Wadadekar (NCRA-TIFR) Topical course 14 / 27

19 Dynamic typing gives way to static typing You can use an assert before the function call to the external function. Yogesh Wadadekar (NCRA-TIFR) Topical course 15 / 27

20 F2PY - Fortran to Python interface generator autogenerates interface files to allow Python to call a Fortran subroutine. Yogesh Wadadekar (NCRA-TIFR) Topical course 16 / 27

21 Installing f2py Recent versions of numpy already include f2py. So, if you installed numpy there is nothing more to do. Yogesh Wadadekar (NCRA-TIFR) Topical course 17 / 27

22 The Hello example C File hello.f subroutine foo (a) integer a print*, "Hello from Fortran!" print*, "a=",a end Of course, there may be multiple subroutines in hello.f, all of which will get wrapped in one go. Yogesh Wadadekar (NCRA-TIFR) Topical course 18 / 27

23 Build the module $ f2py -c -m hello hello.f doc are autocreated. Yogesh Wadadekar (NCRA-TIFR) Topical course 19 / 27

24 A more complicated example - fib3.f Calculate the first n Fibbonacci numbers. Yogesh Wadadekar (NCRA-TIFR) Topical course 20 / 27

25 If you can t edit the Fortran code you can work with signature files which are like the interface files in C. Yogesh Wadadekar (NCRA-TIFR) Topical course 21 / 27

26 Features of f2py F2PY automatically generates doc strings (and optionally LaTeX documentation) for extension modules. F2PY generated functions accept arbitrary (but sensible) Python objects as arguments. The F2PY interface automatically takes care of type-casting. most Fortran 90 features also work. Pyfort, a competitor to f2py is not as feature rich Yogesh Wadadekar (NCRA-TIFR) Topical course 22 / 27

27 Attributes and statements of f2py intent([ in inout out hide in,out inout,out c copy cache callback inplace aux ]) dimension(<dimspec>) common, parameter allocatable optional, required, external depend([<names>]) check([<c-booleanexpr>]) note(<latex text>) usercode, callstatement, callprotoargument, threadsafe, fortranname pymethoddef entry Yogesh Wadadekar (NCRA-TIFR) Topical course 23 / 27

28 Other options for extending Python SIP: lighter version of SWIG. Only works with C/C++. Originally written for PyQt ctypes: no interface code needed, can directly call compiled functions in a binary library file. Some features work only on Windows. Boost.python: good for wrapping entire libraries in C or C++ Pyrex: is a Python like language designed for writing Python extension modules. Almost any Python code is valid Pyrex code. Yogesh Wadadekar (NCRA-TIFR) Topical course 24 / 27

29 Extending Python to Java and C# Java: Jython is an implementation of Python for the JVM. Jython takes the Python programming language syntax and enables it to run on the Java platform. This allows seamless integration with the use of Java libraries and other Java-based applications. But CPython extensions don t work. C# - IronPython is an implementation of the Python programming language running under.net and Silverlight Yogesh Wadadekar (NCRA-TIFR) Topical course 25 / 27

30 My recommendations see if numpy C extensions will do the job for you. if you use Fortran (any flavour), use f2py. If you need to write fresh code (for speedy execution), write in Fortran and wrap with f2py. For C/C++ use SWIG or may SIP For Java - use Jython, for C# use IronPython For all other languages, SWIG is the richest option and please don t forget regression tests. Yogesh Wadadekar (NCRA-TIFR) Topical course 26 / 27

31 Feel free to contact me for any teething problems... Yogesh Wadadekar (NCRA-TIFR) Topical course 27 / 27

operator overloading algorithmic differentiation the class DifferentialNumber operator overloading

operator overloading algorithmic differentiation the class DifferentialNumber operator overloading operator overloading 1 Computing with Differential Numbers algorithmic differentiation the class DifferentialNumber operator overloading 2 Computing with Double Doubles the class DoubleDouble defining