Introduction. HPC Python. Cyrus Proctor January 23th, 2015

Transcription

1 Introduction HPC Python Cyrus Proctor January 23th, 2015

2 Why Python Easy! Nice, readable code Great for prototyping Many third party libraries Large community base Free! C. Proctor 2 Introduction

3 Python on Maverick Maverick User Guide python/2.7.6 You can install your own modules: python setup.py install --user python setup.py install --home=<dir> pip install --user module name You can use virtualenv C. Proctor 3 Introduction

4 Before We Begin XSEDE ssh gsissh -p 2222 maverick.tacc.xsede.org Local ssh -Y Python Exercises cp train00/python-hpc-2015.tar.gz. tar -xzf python-hpc-2015.tar.gz module load intel/ module load python/2.7.6 idev -t 2:00:00 -A python training C. Proctor 4 Introduction

5 Data Types Dynamic language, but also a strongly typed language Objects have a type, which is determined at runtime A variable is a value bound to a name: the value has a type, but the variable doesn t The interpreter keeps track of all variable types You can t do anything that s incompatible with the type of data you re working with: You can do string+string and it will concatenate the strings You can do integer+integer You can t do string+integer C. Proctor 5 Introduction

6 General Data Structures Python List Dynamic arrays Indexed structure Items: Python objects Items of different types Insertion and deletion at random positions C. Proctor 6 Introduction

7 General Data Structures Dictionary Associative arrays (key - value pairs) Indexed by key (string or number) Key: unique Value: any Python object Main operation: store a value with some key and extract the value given the key C. Proctor 7 Introduction

8 Python in HPC You ll hear that Python is slow If it s slow, why should you use it? If you already have a Python code, how can you fix it? You ll find help with Profiling! C. Proctor 8 Introduction

9 Profiling First Steps: examples/1 intro/interactive/prof.py Time your code 1 from math import sqrt 2 3 def hello (): 4 print " Hello world " 5 6 def sum (): 7 for i in range (10000) : 8 a = 1 9 b = 1 10 c = a+b def vector (): 13 a = [ 1., 2., 3., 4., 5., 6., 7.]* for i in a: 15 t = sqrt (i **2) 16 r = a. reverse () 17 s = a. sort () 18 print reduce ( lambda x, y: x + y, a) if name == main : 21 hello () 22 sum () 23 vector () Timing Output Use Linux built-in time 1 time python prof.py 2 Hello world real 0m2.557 s 6 user 0m2.524 s 7 sys 0m0.031 s C. Proctor 9 Introduction

10 Profiling python -m cprofile [-o output file] [-s sort order] script.py examples/1 intro/profiling.py 1 from math import sqrt 2 3 def hello (): 4 print " Hello world " 5 6 def mysum (): 7 for i in range (100000) : 8 a = 1 9 b = 1 10 c = a+b def vector (): 13 a = [ 1., 2., 3., 4., 5., 6., 7.]* for i in a: 15 t = sqrt (i **2) 16 r = a. reverse () 17 s = a. sort () 18 print reduce ( lambda x, y: x + y, a) if name == main : 21 hello () 22 mysum () 23 vector () Hello world function calls in seconds Ordered by: standard name ncalls tottime percall cumtime percall filename : lineno ( function ) prof.py :1( <module >) prof.py :12( vector ) prof.py :18( <lambda >) prof.py :3( hello ) prof.py :6( sum ) { math. sqrt } { method sort } { range } { reduce } C. Proctor 10 Introduction

11 Profiling Line-by-line timing and execution frequency with a profiler decorators to functions you want to profile Once the line profiler is installed, use kernprof -l -v prof.py 1 Hello world Wrote profile results to prof_dec.py. lprof 4 Timer unit : 1e -06 s Total time : s 8 File : prof_dec.py 9 Function : vector at line Line # Hits Time Per Hit % Time Line Contents 12 ============================================================== def vector (): a = [ 1., 2., 3., 4., 5., 6., 7.]* for i in a: t = sqrt (i **2) r = a. reverse () s = a. sort () print reduce ( lambda x, y: x + y, a) C. Proctor 11 Introduction

12 Profiling Let s Install the line profiler Tool pip install user line profiler Specific tools are installed to your /.local directory Change directory into examples/1 intro/interactive and try running python prof.py time python prof.py python -m cprofile -s cumulative prof.py python -m cprofile -s calls prof.py Decorate functions inside prof.py kernprof -l -v prof.py If you have profiled prof.py, profile some others in examples/1 intro! C. Proctor 12 Introduction

13 License c The University of Texas at Austin, 2015 This work is licensed under the Creative Commons Attribution Non-Commercial 3.0 Unported License. To view a copy of this license, visit When attributing this work, please use the following text: HPC Python, Texas Advanced Computing Center, Available under a Creative Commons Attribution Non-Commercial 3.0 Unported License.