10. Brief Tour of the Standard Library
10.1 Operating System Interface
The os module
provides dozens of functions for interacting with the operating system:
>>> import os
>>> os.system('time 0:02')
>>> os.getcwd() # Return the current working directory
Be sure to use the "import os" style instead of "from os import *". This will keep os.open()
from shadowing the builtin open() function which operates much
The builtin dir() and help()
functions are useful as interactive aids for working with large modules like os:
>>> import os
<returns a list of all module functions>
<returns an extensive manual page created from the module's docstrings>
For daily file and directory management tasks, the shutil module provides a
higher level interface that is easier to use:
>>> import shutil
>>> shutil.copyfile('data.db', 'archive.db')
>>> shutil.move('/build/executables', 'installdir')
10.2 File Wildcards
module provides a function for making file lists from directory wildcard searches:
10.3 Command Line Arguments
Common utility scripts often invoke processing command line arguments. These arguments
are stored in the sys
module's argv attribute as a list. For instance the following output results
from running "python demo.py one two three" at the command
>>> import sys
>>> print sys.argv
['demo.py', 'one', 'two', 'three']
module processes sys.argv using the conventions of the Unix getopt() function.
More powerful and flexible command line processing is provided by the optparse module.
10.4 Error Output Redirection and Program Termination
module also has attributes for stdin, stdout, and stderr.
The latter is useful for emitting warnings and error messages to make them visible even
when stdout has been redirected:
>>> sys.stderr.write('Warning, log file not found starting a new one')
Warning, log file not found starting a new one
The most direct way to terminate a script is to use "sys.exit()".
10.5 String Pattern Matching
The re module
provides regular expression tools for advanced string processing. For complex matching and
manipulation, regular expressions offer succinct, optimized solutions:
>>> import re
>>> re.findall(r'\bf[a-z]*', 'which foot or hand fell fastest')
['foot', 'fell', 'fastest']
>>> re.sub(r'(\b[a-z]+) \1', r'\1', 'cat in the the hat')
'cat in the hat'
When only simple capabilities are needed, string methods are preferred because they are
easier to read and debug:
>>> 'tea for too'.replace('too', 'two')
'tea for two'
module gives access to the underlying C library functions for floating point math:
>>> import math
>>> math.cos(math.pi / 4.0)
>>> math.log(1024, 2)
module provides tools for making random selections:
>>> import random
>>> random.choice(['apple', 'pear', 'banana'])
>>> random.sample(xrange(100), 10) # sampling without replacement
[30, 83, 16, 4, 8, 81, 41, 50, 18, 33]
>>> random.random() # random float
>>> random.randrange(6) # random integer chosen from range(6)
10.7 Internet Access
There are a number of modules for accessing the internet and processing internet
protocols. Two of the simplest are urllib2 for retrieving data from urls and smtplib for sending mail:
>>> import urllib2
>>> for line in urllib2.urlopen('http://tycho.usno.navy.mil/cgi-bin/timer.pl'):
... if 'EST' in line: # look for Eastern Standard Time
... print line
<BR>Nov. 25, 09:43:32 PM EST
>>> import smtplib
>>> server = smtplib.SMTP('localhost')
>>> server.sendmail('email@example.com', 'firstname.lastname@example.org',
Beware the Ides of March.
10.8 Dates and Times
module supplies classes for manipulating dates and times in both simple and complex ways.
While date and time arithmetic is supported, the focus of the implementation is on
efficient member extraction for output formatting and manipulation. The module also
supports objects that are time zone aware.
# dates are easily constructed and formatted
>>> from datetime import date
>>> now = date.today()
datetime.date(2003, 12, 2)
>>> now.strftime("%m-%d-%y or %d%b %Y is a %A on the %d day of %B")
'12-02-03 or 02Dec 2003 is a Tuesday on the 02 day of December'
# dates support calendar arithmetic
>>> birthday = date(1964, 7, 31)
>>> age = now - birthday
10.9 Data Compression
Common data archiving and compression formats are directly supported by modules
gzip, bz2, zipfile, and tarfile.
>>> import zlib
>>> s = 'witch which has which witches wrist watch'
>>> t = zlib.compress(s)
'witch which has which witches wrist watch'
10.10 Performance Measurement
Some Python users develop a deep interest in knowing the relative performance between
different approaches to the same problem. Python provides a measurement tool that answers
those questions immediately.
For example, it may be tempting to use the tuple packing and unpacking feature instead
of the traditional approach to swapping arguments. The timeit module quickly
demonstrates that the traditional approach is faster:
>>> from timeit import Timer
>>> Timer('t=a; a=b; b=t', 'a=1; b=2').timeit()
>>> Timer('a,b = b,a', 'a=1; b=2').timeit()
In contrast to timeit's fine level of granularity, the profile and pstats modules provide tools for identifying time critical sections in
larger blocks of code.
10.11 Quality Control
One approach for developing high quality software is to write tests for each function
as it is developed and to run those tests frequently during the development process.
module provides a tool for scanning a module and validating tests embedded in a program's
docstrings. Test construction is as simple as cutting-and-pasting a typical call along
with its results into the docstring. This improves the documentation by providing the user
with an example and it allows the doctest module to make sure the code remains true to the
"""Computes the arithmetic mean of a list of numbers.
>>> print average([20, 30, 70])
return sum(values, 0.0) / len(values)
doctest.testmod() # automatically validate the embedded tests
module is not as effortless as the doctest module, but it allows a
more comprehensive set of tests to be maintained in a separate file:
self.assertEqual(average([20, 30, 70]), 40.0)
self.assertEqual(round(average([1, 5, 7]), 1), 4.3)
self.assertRaises(ZeroDivisionError, average, )
self.assertRaises(TypeError, average, 20, 30, 70)
unittest.main() # Calling from the command line invokes all tests
10.12 Batteries Included
Python has a ``batteries included'' philosophy. This is best seen through the
sophisticated and robust capabilities of its larger packages. For example:
* The xmlrpclib
modules make implementing remote procedure calls into an almost trivial task. Despite the
names, no direct knowledge or handling of XML is needed.
* The email
package is a library for managing email messages, including MIME and other RFC 2822-based
message documents. Unlike smtplib and poplib
which actually send and receive messages, the email package has a complete toolset for
building or decoding complex message structures (including attachments) and for
implementing internet encoding and header protocols.
* The xml.dom
packages provide robust support for parsing this popular data interchange format.
Likewise, the csv module supports direct reads and writes in a
common database format. Together, these modules and packages greatly simplify data
interchange between python applications and other tools.
* Internationalization is supported by a number of modules including gettext, locale, and the codecs package.