LECTURE 9 The Standard Library Part 3

Transcription

1 LECTURE 9 The Standard Library Part 3

2 THE STANDARD LIBRARY In this lecture, we will briefly cover each of the remaining Standard Library modules that you absolutely must know about. Some of the remaining modules will be introduced later on when we cover specific applications domains in Python such as web development, data persistence, etc. os time argparse sys

3 OS SERVICES The os module provides a common interface for operating system dependent functionality. Most of the functions are actually implemented by platform-specific modules, but there is no need to explicitly call them as such.

4 OS SERVICES We ve already seen how the os module can be used to work with files. We know that there are built-in functions to open and close files but os extends file operations. os.rename(current_name, new_name) renames the file current_name to new_name. os.remove(filename) deletes an existing file named filename.

5 OS SERVICES There are also a number of directory services provided by the os module. os.listdir(dirname) lists all of the files in directory dirname. os.getcwd() returns the current directory. os.chdir(dirname) will change the current directory. >>> os.listdir("demos") ['frac.py', 'dogs.py', 'football.csv', 'csv_parser.py'] >>> os.listdir(".") ['lect5.py', 'demos', 'lect3.py', 'lect2.py'] >>> os.getcwd() '/home/faculty/carnahan/cis4930' >>> os.chdir(os.getcwd() + "/demos") >>> os.getcwd() '/home/faculty/carnahan/cis4930/demos >>> os.rename("dogs.py", "cats.py") >>> os.listdir(".") ['frac.py', 'football.csv', 'cats.py', 'csv_parser.py'] >>> os.remove("cats.py") >>> os.listdir(".") ['frac.py', 'football.csv', 'csv_parser.py']

6 OS SERVICES Use os.mkdir(dirname) and os.rmdir(dirname) to make and remove a single directory. Use os.makedirs(path/of/dirs) and os.removedirs(path/of/dirs) to make and remove a hierarchy of directories. Make sure directories are empty before removal! >>> os.makedirs("dir1/dir2/dir3") >>> os.listdir(".") ['frac.py', 'dir1', 'football.csv', 'csv_parser.py'] >>> f = open("dir1/dir2/dir3/test", "w") >>> f.write("hi!") >>> f.close() >>> for line in open("dir1/dir2/dir3/test", "r"):... print line... hi! >>> os.remove("dir1/dir2/dir3/test") >>> os.removedirs("dir1/dir2/dir3") >>> os.listdir(".") ['frac.py', 'football.csv', 'csv_parser.py']

7 OS SERVICES The os.walk(path) method will generate a tuple (dirpath, dirnames, filenames) for each directory found by traversing the directory tree rooted at path. >>> os.makedirs("dir1/dir2/dir3") >>> os.listdir(".") ['frac.py', 'dir1', 'football.csv', 'csv_parser.py'] >>> os.mkdir("dir1/dir2/dir4") >>> f = open("dir1/dir2/d2file", "w") >>> f = open("dir1/dir2/dir3/d3file", "w") >>> f = open("dir1/dir2/dir4/d4file", "w") >>> path = os.getcwd() >>> for (path, dirs, files) in os.walk(path):... print "Path: ", path... print "Directories: ", dirs... print "Files: ", files... print "---"

8 OS SERVICES The os.walk(path) method will generate a tuple (dirpath, dirnames, filenames) for each directory found by traversing the directory tree rooted at path. Path: /home/faculty/carnahan/cis4930/demos Directories: ['dir1'] Files: ['frac.py', 'football.csv', 'csv_parser.py'] --- Path: /home/faculty/carnahan/cis4930/demos/dir1 Directories: ['dir2'] Files: [] --- Path: /home/faculty/carnahan/cis4930/demos/dir1/dir2 Directories: ['dir4', 'dir3'] Files: ['d2file'] --- Path: /home/faculty/carnahan/cis4930/demos/dir1/dir2/dir4 Directories: [] Files: ['d4file'] --- Path: /home/faculty/carnahan/cis4930/demos/dir1/dir2/dir3 Directories: [] Files: ['d3file'] ---

9 OS SERVICES The os module includes an os.stat(path) method which will return file attributes related to the path provided (equivalent of stat() system call). Result is a stat structure which includes st_size: size of file in bytes. st_atime: time of most recent access. st_uid: user id of owner. st_nlink: number of hard links. >>> import os >>> stat_info = os.stat("football.csv") >>> stat_info posix.stat_result(st_mode=33216, st_ino= l, st_dev=20l, st_nlink=1, st_uid=87871, st_gid=300, st_size=648l, st_atime= , st_mtime= , st_ctime= ) >>> stat_info.st_mtime

10 OS SERVICES The os.system(cmd) function executes the argument cmd in a subshell. The return value is the exit status of the command. >>> os.system("ls") csv_parser.py dir1 football.csv frac.py 0 >>> os.system("touch newfile.txt") 0 >>> os.system("ls") csv_parser.py dir1 football.csv frac.py newfile.txt 0

11 OS SERVICES The os.exec(path, args) function will start a new process from path using the args as arguments, replacing the current one. Alternatives include os.execve(), os.execvp(), etc as usual. Arguments depend on version used. carnahan@diablo:~/cis4930/demos> python2.7 Python (default, Oct , 01:47:54) [GCC (Red Hat EL4)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import os >>> os.execvp("python2.7", ("python2.7", "csv_parser.py")) Aston_Villa has a minimum goal difference of 1 carnahan@diablo:~/cis4930/demos>

12 OS SERVICES Combine the os.exec*() functions with os.fork() and os.wait() to spawn processes from the current process. The former makes a copy of the current process, the latter waits for a child process to finish. Use os.spawn() on Windows. import os import sys pid = os.fork() if not pid: os.execvp( python2.7, ( python2.7, csv_parser.py )) os.wait()

13 TIME The time module includes implementations for a handful of time-related functions. The simplest and most common use of the time module is to retrieve the current time in seconds since 1/1/1970. >>> import time >>> time.time()

14 TIME The methods time.localtime(t) and time.gmtime(t) accept as an argument seconds since Unix time 0. Both return a time tuple (year, month, day, hour, minute, second, day of week, day of year, daylight savings flag). >>> t = time.time() >>> time.gmtime(t) # GMT time.struct_time(tm_year=2015, tm_mon=1, tm_mday=28, tm_hour=9, tm_min=24, tm_sec=17, tm_wday=2, tm_yday=28, tm_isdst=0) >>> lt = time.localtime(t) # Local Time >>> lt time.struct_time(tm_year=2015, tm_mon=1, tm_mday=28, tm_hour=4, tm_min=24, tm_sec=17, tm_wday=2, tm_yday=28, tm_isdst=0) >>> time.mktime(lt) # Only on localtime

15 TIME The time module also provides a number of functions for converting and formatting time tuples as strings. >>> t = time.time() >>> now = time.localtime(t) >>> print time.asctime(now) Wed Jan 28 04:27: >>> print time.strftime("%y/%m/%d %H:%M", now) 15/01/28 04:27 >>> print time.strftime("%m/%d/%y %H:%M", now) 01/28/15 04:27 >>> print time.strptime("28 Jan 15", "%d %b %y") time.struct_time(tm_year=2015, tm_mon=1, tm_mday=28, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=2, tm_yday=28, tm_isdst=-1) Do not use strptime on Windows it will typically not be defined.

16 TIME You can use time s methods to time the execution of a program. Use either time.time() for wall time or time.clock() for cpu time. Also, time.sleep(s) will cause the process to sleep for s seconds. On Windows, time.time() and time.clock() both measure the wall time. >>> import time >>> def sleep_func(t):... time.sleep(t)... >>> def cpu_time():... start = time.clock()... sleep_func(5)... print "CPU time:", time.clock() - start... >>> cpu_time() CPU time: 0.0 >>> def wall_time():... start = time.time()... sleep_func(5)... print "Wall time:", time.time() - start... >>> wall_time() Wall time:

17 ARGPARSE The argparse module is a very powerful tool for organizing and processing arguments to your python program. argtest.py import argparse carnahan@diablo:~>python2.7 argtest.py carnahan@diablo:~>python2.7 argtest.py --help usage: argtest.py [-h] parser = argparse.argumentparser() parser.parse_args() optional arguments: -h, --help show this help message and exit carnahan@diablo:~>python2.7 argtest.py spam usage: argtest.py [-h] argtest.py: error: unrecognized arguments: spam By default, argparse will specify actions for the help and h arguments. Namely, it ll tell the user their options.

18 ARGPARSE We can add positional arguments which are required. import argparse parser = argparse.argumentparser() parser.add_argument("name", help="enter a name to be greeted.", type=str) args = parser.parse_args() print "Hello, "+args.name carnahan@diablo:~>python2.7 argtest.py class Hello, class

19 ARGPARSE Now, let s add an optional argument. import argparse parser = argparse.argumentparser() parser.add_argument("name", help="enter a name to be greeted.", type=str) parser.add_argument("--bye", help="say goodbye instead of greet.", action="store_true") args = parser.parse_args() if not args.bye: print "Hello, "+args.name else: print "Goodbye, "+args.name

20 ARGPARSE argtest.py class Hello, class argtest.py class --bye Goodbye, class argtest.py usage: argtest.py [-h] [--bye] name argtest.py: error: too few arguments argtest.py -h usage: argtest.py [-h] [--bye] name positional arguments: name enter a name to be greeted. optional arguments: -h, --help show this help message and exit --bye say goodbye instead of greet.

21 SYS The sys module implements functions for accessing and manipulating the runtime environment. The most common way to use sys is to access arguments to a python program. testargs.py import sys for i in range(len(sys.argv)): print "sys.argv[" + str(i) + "] is " + sys.argv[i] carnahan@diablo:~>python testargs.py here are some arguments sys.argv[0] is testargs.py sys.argv[1] is here sys.argv[2] is are sys.argv[3] is some sys.argv[4] is arguments

22 SYS The path used to look for modules is stored in sys.path. You can manipulate it like a list. systest.py import sys print "path has", len(sys.path), "members" sys.path.insert(0, "samples") import sample carnahan@diablo:~>python systest.py path has 8 members Hello from the sample module! Traceback (most recent call last): File "systest.py", line 9, in? import math ImportError: No module named math sys.path = [] import math Use sys.builtin_module_names to see which modules are built into the interpreter. The sys module is one of them.

23 SYS The sys.modules dictionary contains all of the modules currently imported. >>> import sys >>> sys.modules.keys() ['copy_reg', 'sre_compile', '_sre', 'encodings', 'site', ' builtin ', 'sysconfig', ' main ', 'encodings.encodings', 'math', 'abc', 'posixpath', '_weakrefset', 'errno', 'encodings.codecs', 'sre_constants', 're', '_abcoll', 'types', '_codecs', 'encodings. builtin ', '_warnings', 'encodings.latin_1', 'genericpath', 'stat', 'zipimport', '_sysconfigdata', 'warnings', 'UserDict', 'sys', 'codecs', 'readline', 'os.path', 'signal', 'traceback', 'linecache', 'posix', 'encodings.aliases', 'exceptions', 'sre_parse', 'os', '_weakref'] Use sys.platform to determine the host platform. >>> sys.platform 'linux2'

24 SYS For profiling, use the sys.setprofile(profiler) function. The profiler function must be defined by you. It will profile every function or method called until you disable the profiler by passing None to sys.setprofile. For tracing, use sys.settrace(tracer) with a self-definer tracer. To exit a program before it terminates itself, use sys.exit(). It will raise a SystemExit exception which, if not caught, will end the program. You can also set a function object to sys.exitfunc to specify cleanup actions. Optional arguments: - 0 is considered a successful termination. - Some error message will print to stderr and return 1.

25 NEXT LECTURE We ll discuss development tools for Python including logging, testing, and debugging.