Learning Python: Unit 9, Built-ins (original) (raw)

���9. Built-in To���ols Overview

♦ Python's tool box

● Types and operations:�� lists, dictionaries, files, slices,�

● Functions:�� len, range, zip, getattr�

● Modules (Python and C):�� string, os, Tkinter, pickle,�

● Exceptions:�� IndexError, KeyError,�

● Attributes:�� __dict__, __name__,�

● Peripheral tools:�� NumPy, SWIG, Jython, PythonWin,�

Topics

♦ Debugging options

♦ Testing frameworks

♦ Timing and profiling Python programs

♦ Packaging Python programs

♦ Installation tools

♦ Development tools for larger projects

♦ Summary: Python tool set layers

But first, the secret handshake�

Debugging options

♦ �pdb� debugger: dbx-like command line interface

♦ Imported module, written in Python

♦ May also be run as script →� python -m pdb script.py

♦ Also see: �IDLE� Tkinter-based debugger GUI�

♦ See library manuals for pdbcommands and usage

Other error-handling tricks

● Error messages

● Top-level stack tracebacks

● Inserting print statements

● Outer exception handlers

defsafe(entry, *args):

��� try:

������� entry(*args)�������� # catch everything else (apply)

��� except:

������� import sys

������� print sys.exc_info()[0],sys.exc_info()[1]���# type, value

��

Debugging example

file: boom.py

def func(x, y):

��� return x / y

Session

>>> import boom

>>>boom.func(1, 0)

Traceback (innermost last):

� File "<stdin>", line 1, in ?

� File "boom.py", line 3, in func

��� return x / y

ZeroDivisionError: integer division or modulo

>>>

>>> import pdb

>>>pdb.run('boom.func(1, 0)')����# run/debug code

> (0)?()

(Pdb) b boom.func����������������� # set breakpoint

(Pdb) c��������������������������� # continue program

> boom.py(2)func()

->def func(x, y):

(Pdb) s��������������������������� # step 1 line

> boom.py(3)func()

-> return x / y

(Pdb) s

ZeroDivisionError: 'integer division or modulo'

> boom.py(3)func()

-> return x / y

(Pdb) where����������������������� # stack trace

� (1)?()

> boom.py(3)func()

-> return x / y

(Pdb) p y������������������������� # print variables

0

Inspecting name-spaces

♦ Python lookups use 3-scope rule: local, global, built-in

♦ locals(), globals(): return name-spaces as dictionaries

% python

>>> def func(x):

...���� a = 1

...���� print locals()��������������� # on function call

...���� print globals().keys()

...

>>> class klass:

...���� def __init__(self):

...�������� print locals()����������� # on instance creation

...�������� print globals().keys()

...���� print locals()��������������� # on class creation

...���� print globals().keys()

...

{'__init__': <function __init__ at 76ed00>}

['__builtins__', '__name__', 'func', '__doc__']

>>> func(1)

{'a': 2, 'x': 1}

['__builtins__', '__name__', 'func', 'klass', '__doc__']

>>> x = klass()

{'self': <klass instance at 76f8d0>, 'arg': None}

['__builtins__', '__name__', 'func', 'klass', '__doc__']

>>> def nester(L, M, N):

...���� class nested:�������������� # assigns class to name

...�������� def __init__(self):

...������������ pass

...�������� print locals()��������� # local=class global=mod

...�������� print globals().keys()� # no access to L/M/N!

...���� return nested���������������

...

>>> nester(1, 2, 3)

{'__init__': <function __init__ at 761e30>}

['__doc__', 'nester', '__name__', 'x', 'func', 'klass',...]

<class nested at 762960>

Dynamic coding tools

♦ apply �� [nowfunc(*args)] runs functions with argument tuples

♦ eval ���� evaluates a Python expression code-string

♦ exec ���� runs a Python statement code-string (3.X: exec())

♦ getattr fetches an object�s attribute by name string

♦ Supports run-time program construction

♦ Supports embedding Python in Python

Basic usage

>>> x = "2 ** 5"

>>> a = eval(x)

>>> a

32

>>> exec "print a / 2"�������� # 3.X: exec('print(a / 2)')

16

>>> def echo(a, b, c): print a, b, c

...

>>> apply(echo, (1, 2, 3))���� # now: echo(*(1, 2, 3))

1 2 3

>>> import string

>>> getattr(string, "uppercase")

'ABCDEFGHIJKLMNOPQRSTUVWXYZ'

>>> D = {}

>>> exec "import math\nx = math.pi" in D, D�� # 3.X exec()

>>> D['x']

3.14159265359

Example

♦ Import module by name, run function by name/args

♦ Runs:� [ message.printer(�sir�, �robin�) ]

♦ Also see:�newer __import__ function

♦ Preview: will revisit model to embed Python in C

file: message.py

def printer(str1, str2):

��� return 'brave', str1, str2

file: dynamic.py

def runFunction(moduleName, functionName, argsTuple):

��� exec 'import ' + moduleName

��� module = eval(moduleName)

��� function = getattr(module, functionName)

��� return function(*argsTuple)

if__name__ == '__main__':

��� from sys import argv

��� print runFunction(argv[1], argv[2], tuple(argv[3:]))

Command line

% python dynamic.py message printer sir robin

('brave', 'sir', 'robin')

Timing and profiling Python programs

♦ _time_module contains C library type tools

♦ Useful for performance tweaking (along with profiler)

♦ Warning: be careful to compare apples to apples!

→ See also newer �timeit� module: automated, portable timing tools

>>> import timeit

>>> min(timeit.repeat(stmt="[x ** 2 for x in range(1000)]", number=1000, repeat=5))

0.5062382371756811

c:\code> python -m timeit-n 1000 -r 5 "[x ** 2 for x in range(1000)]"

1000 loops, best of 5: 505 usecper loop

Example: inline stack alternatives

♦ List based stacks

['spam0', 'spam1', 'spam2']� ←top

● Use list in-place changes: append/del

● Lists resized on demand: grown in increments

♦ Tuple-pair based stacks

_top_→� ('spam2', ('spam1', ('spam0', None)))

● Use tuple packing/unpacking assignments

● Build a tree of 2-item tuples: (item, tree)

● Like Lisp �cons� cells/linked lists: avoids copies

file: testinline.py

#!/opt/local/bin/python

import time

from sys import argv, exit

numtests= 20

�

try:

��� pushes, pops = eval(argv[1]), eval(argv[2])

except:

��� print 'usage: testinline.py '; exit(1)

�

deftest(reps, func):

��� start_cpu �= time.clock()

��� for i in xrange(reps):���������������# call N times

������� x = func()

���return time.clock() - start_cpu

�

definline1():��������������������������� # builtin lists

��� x = []

��� for i in range(pushes): x.append('spam' + `i`)

��� for i in range(pops):�� del x[-1]

�

definline2():��������������������������� # builtin tuples

��� x = None

��� for i in range(pushes): x = ('spam' + `i`, x)

��� for i in range(pops):�� (top, x) = x

�

print 'lists: ', test(numtests, inline1)� # run 20 times

print 'tuples:', test(numtests, inline2)

�

Results (on an ancient machine�)

% testinline.py 500 500���� --20*(500 pushes + 500 pops)

lists:� 0.77��������������� --lists: append/del

tuples: 0.43��������������� --tuples: pack/unpack

% testinline.py 1000 1000�� --20K pushes + 20K pops

lists:� 1.54

tuples: 0.87

% testinline.py 200 200

lists:� 0.31

tuples: 0.17

% testinline.py 5000 5000

lists:� 7.72

tuples: 4.5

Related Modules

datetime

>>> from datetimeimport datetime, timedelta

>>> x = datetime(2004, 11, 21)

>>> y = datetime(2005, 3, 19)

>>>

>>> y - x

datetime.timedelta(118)

>>>

>>> x + timedelta(30)

datetime.datetime(2004, 12, 21, 0, 0)

profile

● Run as _script_or interactive, like pdb debugger

● As script → python -m profile script.py

● Warning: don�t run at IDLE prompt (too much data)!

● cProfile: more efficient version in 2.5+

● pstats to report on results later

● Optimizing: profile, time, shedskin/psyco, move to C

>>> import profile

>>> profile.run('import test1')

�������� 35 function calls in 0.027 CPU seconds

�� Ordered by: standard name

�� ncalls� tottime� percall� cumtime� percall filename:lineno(function)

������ 16���0.001��� 0.000��� 0.001���0.000 :0(range)

������� 1���0.005��� 0.005��� 0.005���0.005 :0(setprofile)

������� 1���0.002��� 0.002��� 0.022���0.022 :1(?)

������� 1���0.000��� 0.000��� 0.027���0.027 profile:0(import test1)

������� 0���0.000������������ 0.000��������� profile:0(profiler)

������� 1���0.000��� 0.000��� 0.020���0.020 test1.py:2(?)

������ 15���0.019��� 0.001��� 0.020���0.001 test1.py:2(myfun)

Example: timing iteration alternatives

File types and packaging options

Byte code

● Modules compiled to portable byte-code on import: .pyc

● compileall module forces imports, to make .pyc�s

● .pyo files created and run with �O command-line flag (removed in 3.5!)

Frozen Binaries

● Package byte-code + Python in an executable

● Don�t require Python to be installed

● Protect your program code

Other options

● Import hooks support zip files, decryption, etc.

● Pickler converts objects to/from text stream (serializer)

Development tools for larger projects

Testing tools in the standard library

See also third-party testing tools, such as Nose

Simplest convention

if __name__ == '__main__':

��� unit test code here...

doctest module

● automates interactive session

● regression test system

# file spams.py

"""

This module works as follows:

>>> spams(3)

'spamspamspam'

>>> shrubbery()

'spamspamspamspam!!!'

"""

def spams(N):

��� return 'spam' * N

def shrubbery():

��� return spams(4) + '!!!'

if __name__ == '__main__':

��� import doctest

��� doctest.testmod()

C:\Python25>python spams.py -v

Trying:

��� spams(3)

Expecting:

��� 'spamspamspam'

ok

Trying:

��� shrubbery()

Expecting:

��� 'spamspamspamspam!!!'

ok

2 items had no tests:

��� __main__.shrubbery

��� __main__.spams

1 items passed all tests:

�� 2 tests in __main__

2 tests in 3 items.

2 passed and 0 failed.

Test passed.

unittest module (PyUnit)

● class structure for unit test code

● See library manual: test suites, �

import random

import unittest

class TestSequenceFunctions(unittest.TestCase):

���

��� def setUp(self):

������� self.seq = range(10)

��� def testshuffle(self):

������� # make sure the shuffled sequence does not lose any elements

������� random.shuffle(self.seq)

������� self.seq.sort()

������� self.assertEqual(self.seq, range(10))

��� def testchoice(self):

������� element = random.choice(self.seq)

������� self.assert_(element in self.seq)

��� def testsample(self):

������� self.assertRaises(ValueError, random.sample, self.seq, 20)

������� for element in random.sample(self.seq, 5):

����������� self.assert_(element in self.seq)

if __name__ == '__main__':

��� unittest.main()

Distutils: auto-install/build utility

See also more recent �pip� installer regime, and the PyPI site

# setup.py

from distutils.coreimport setup

setup(name='foo',

�����version='1.0',

����� py_modules=['foo'],

����� )

# Usage

python setup.py sdist���������� # make source distribution

python setup.py install�������� # install the system

python setup.py bdist_wininst�� # make Windows exe installer

python setup.py bdist_rpm������ # make Linux RPM installer

python setup.py register������� # register with PyPIsite

# setup.py for C extension modules

from distutils.coreimport setup, Extension

setup(name='foo',

�����version='1.0',

����� ext_modules=[Extension('foo', ['foo.c'])],

����� )

Summary: Python tool-set layers

♦ Built-ins

● Lists, dictionaries, strings, library modules, etc.

● High-level tools for simple, fast programming

������

♦ Python extensions

● Functions, classes, modules

● For adding extra features, and new object types

������

♦ C extensions

● C modules, C types

● For integrating external systems, optimizing components, customization

Lab Session 7

Click here to go to lab exercises

Click here to go to exercise solutions

Click here to go to solution source files

Click here to go to lecture example files