functools – Tools for Manipulating Functions¶
| Purpose: | Functions that operate on other functions. |
|---|---|
| Available In: | 2.5 and later |
The functools module provides tools for working with functions and other callable objects, to adapt or extend them for new purposes without completely rewriting them.
Decorators¶
The primary tool supplied by the functools module is the class partial, which can be used to “wrap” a callable object with default arguments. The resulting object is itself callable, and can be treated as though it is the original function. It takes all of the same arguments as the original, and can be invoked with extra positional or named arguments as well.
partial¶
This example shows two simple partial objects for the function myfunc(). Notice that show_details() prints the func, args, and keywords attributes of the partial object.
import functools
def myfunc(a, b=2):
"""Docstring for myfunc()."""
print '\tcalled myfunc with:', (a, b)
return
def show_details(name, f, is_partial=False):
"""Show details of a callable object."""
print '%s:' % name
print '\tobject:', f
if not is_partial:
print '\t__name__:', f.__name__
print '\t__doc__', repr(f.__doc__)
if is_partial:
print '\tfunc:', f.func
print '\targs:', f.args
print '\tkeywords:', f.keywords
return
show_details('myfunc', myfunc)
myfunc('a', 3)
print
p1 = functools.partial(myfunc, b=4)
show_details('partial with named default', p1, True)
p1('default a')
p1('override b', b=5)
print
p2 = functools.partial(myfunc, 'default a', b=99)
show_details('partial with defaults', p2, True)
p2()
p2(b='override b')
print
print 'Insufficient arguments:'
p1()
At the end of the example, the first partial created is invoked without passing a value for a, causing an exception.
$ python functools_partial.py
myfunc:
object: <function myfunc at 0x100468c08>
__name__: myfunc
__doc__ 'Docstring for myfunc().'
called myfunc with: ('a', 3)
partial with named default:
object: <functools.partial object at 0x10046b050>
__doc__ 'partial(func, *args, **keywords) - new function with partial
application\n of the given arguments and keywords.\n'
func: <function myfunc at 0x100468c08>
args: ()
keywords: {'b': 4}
called myfunc with: ('default a', 4)
called myfunc with: ('override b', 5)
partial with defaults:
object: <functools.partial object at 0x10046b0a8>
__doc__ 'partial(func, *args, **keywords) - new function with partial
application\n of the given arguments and keywords.\n'
func: <function myfunc at 0x100468c08>
args: ('default a',)
keywords: {'b': 99}
called myfunc with: ('default a', 99)
called myfunc with: ('default a', 'override b')
Insufficient arguments:
Traceback (most recent call last):
File "functools_partial.py", line 49, in <module>
p1()
TypeError: myfunc() takes at least 1 argument (1 given)
update_wrapper¶
The partial object does not have __name__ or __doc__ attributes by default, and without those attributes decorated functions are more difficult to debug. Using update_wrapper(), copies or adds attributes from the original function to the partial object.
import functools
def myfunc(a, b=2):
"""Docstring for myfunc()."""
print '\tcalled myfunc with:', (a, b)
return
def show_details(name, f):
"""Show details of a callable object."""
print '%s:' % name
print '\tobject:', f
print '\t__name__:',
try:
print f.__name__
except AttributeError:
print '(no __name__)'
print '\t__doc__', repr(f.__doc__)
print
return
show_details('myfunc', myfunc)
p1 = functools.partial(myfunc, b=4)
show_details('raw wrapper', p1)
print 'Updating wrapper:'
print '\tassign:', functools.WRAPPER_ASSIGNMENTS
print '\tupdate:', functools.WRAPPER_UPDATES
print
functools.update_wrapper(p1, myfunc)
show_details('updated wrapper', p1)
The attributes added to the wrapper are defined in functools.WRAPPER_ASSIGNMENTS, while functools.WRAPPER_UPDATES lists values to be modified.
$ python functools_update_wrapper.py
myfunc:
object: <function myfunc at 0x100468c80>
__name__: myfunc
__doc__ 'Docstring for myfunc().'
raw wrapper:
object: <functools.partial object at 0x10046c0a8>
__name__: (no __name__)
__doc__ 'partial(func, *args, **keywords) - new function with partial
application\n of the given arguments and keywords.\n'
Updating wrapper:
assign: ('__module__', '__name__', '__doc__')
update: ('__dict__',)
updated wrapper:
object: <functools.partial object at 0x10046c0a8>
__name__: myfunc
__doc__ 'Docstring for myfunc().'
Other Callables¶
Partials work with any callable object, not just standalone functions.
import functools
class MyClass(object):
"""Demonstration class for functools"""
def meth1(self, a, b=2):
"""Docstring for meth1()."""
print '\tcalled meth1 with:', (self, a, b)
return
def meth2(self, c, d=5):
"""Docstring for meth2"""
print '\tcalled meth2 with:', (self, c, d)
return
wrapped_meth2 = functools.partial(meth2, 'wrapped c')
functools.update_wrapper(wrapped_meth2, meth2)
def __call__(self, e, f=6):
"""Docstring for MyClass.__call__"""
print '\tcalled object with:', (self, e, f)
return
def show_details(name, f):
"""Show details of a callable object."""
print '%s:' % name
print '\tobject:', f
print '\t__name__:',
try:
print f.__name__
except AttributeError:
print '(no __name__)'
print '\t__doc__', repr(f.__doc__)
return
o = MyClass()
show_details('meth1 straight', o.meth1)
o.meth1('no default for a', b=3)
print
p1 = functools.partial(o.meth1, b=4)
functools.update_wrapper(p1, o.meth1)
show_details('meth1 wrapper', p1)
p1('a goes here')
print
show_details('meth2', o.meth2)
o.meth2('no default for c', d=6)
print
show_details('wrapped meth2', o.wrapped_meth2)
o.wrapped_meth2('no default for c', d=6)
print
show_details('instance', o)
o('no default for e')
print
p2 = functools.partial(o, f=7)
show_details('instance wrapper', p2)
p2('e goes here')
This example creates partials from an instance, and methods of an instance.
$ python functools_method.py
meth1 straight:
object: <bound method MyClass.meth1 of <__main__.MyClass object at
0x10046a3d0>>
__name__: meth1
__doc__ 'Docstring for meth1().'
called meth1 with: (<__main__.MyClass object at 0x10046a3d0>, 'no d
efault for a', 3)
meth1 wrapper:
object: <functools.partial object at 0x10046c158>
__name__: meth1
__doc__ 'Docstring for meth1().'
called meth1 with: (<__main__.MyClass object at 0x10046a3d0>, 'a go
es here', 4)
meth2:
object: <bound method MyClass.meth2 of <__main__.MyClass object at
0x10046a3d0>>
__name__: meth2
__doc__ 'Docstring for meth2'
called meth2 with: (<__main__.MyClass object at 0x10046a3d0>, 'no d
efault for c', 6)
wrapped meth2:
object: <functools.partial object at 0x10046c0a8>
__name__: meth2
__doc__ 'Docstring for meth2'
called meth2 with: ('wrapped c', 'no default for c', 6)
instance:
object: <__main__.MyClass object at 0x10046a3d0>
__name__: (no __name__)
__doc__ 'Demonstration class for functools'
called object with: (<__main__.MyClass object at 0x10046a3d0>, 'no
default for e', 6)
instance wrapper:
object: <functools.partial object at 0x10046c1b0>
__name__: (no __name__)
__doc__ 'partial(func, *args, **keywords) - new function with parti
al application\n of the given arguments and keywords.\n'
called object with: (<__main__.MyClass object at 0x10046a3d0>, 'e g
oes here', 7)
wraps¶
Updating the properties of a wrapped callable is especially useful when used in a decorator, since the transformed function ends up with properties of the original, “bare”, function.
import functools
def show_details(name, f):
"""Show details of a callable object."""
print '%s:' % name
print '\tobject:', f
print '\t__name__:',
try:
print f.__name__
except AttributeError:
print '(no __name__)'
print '\t__doc__', repr(f.__doc__)
print
return
def simple_decorator(f):
@functools.wraps(f)
def decorated(a='decorated defaults', b=1):
print '\tdecorated:', (a, b)
print '\t',
f(a, b=b)
return
return decorated
def myfunc(a, b=2):
print '\tmyfunc:', (a,b)
return
show_details('myfunc', myfunc)
myfunc('unwrapped, default b')
myfunc('unwrapped, passing b', 3)
print
wrapped_myfunc = simple_decorator(myfunc)
show_details('wrapped_myfunc', wrapped_myfunc)
wrapped_myfunc()
wrapped_myfunc('args to decorated', 4)
functools provides a decorator, wraps(), which applies update_wrapper() to the decorated function.
$ python functools_wraps.py
myfunc:
object: <function myfunc at 0x10046c050>
__name__: myfunc
__doc__ None
myfunc: ('unwrapped, default b', 2)
myfunc: ('unwrapped, passing b', 3)
wrapped_myfunc:
object: <function myfunc at 0x10046c0c8>
__name__: myfunc
__doc__ None
decorated: ('decorated defaults', 1)
myfunc: ('decorated defaults', 1)
decorated: ('args to decorated', 4)
myfunc: ('args to decorated', 4)
Comparison¶
Under Python 2, classes can define a __cmp__() method that returns -1, 0, or 1 based on whether the object is less than, equal to, or greater than the item being compared. Python 2.1 introduces the rich comparison methods API, __lt__(), __le__(), __eq__(), __ne__(), __gt__(), and __ge__(), which perform a single comparison operation and return a boolean value. Python 3 deprecated __cmp__() in favor of these new methods, so functools provides tools to make it easier to write Python 2 classes that comply with the new comparison requirements in Python 3.
Rich Comparison¶
The rich comparison API is designed to allow classes with complex comparisons to implement each test in the most efficient way possible. However, for classes where comparison is relatively simple, there is no point in manually creating each of the rich comparison methods. The total_ordering() class decorator takes a class that provides some of the methods, and adds the rest of them.
import functools
import inspect
from pprint import pprint
@functools.total_ordering
class MyObject(object):
def __init__(self, val):
self.val = val
def __eq__(self, other):
print ' testing __eq__(%s, %s)' % (self.val, other.val)
return self.val == other.val
def __gt__(self, other):
print ' testing __gt__(%s, %s)' % (self.val, other.val)
return self.val > other.val
print 'Methods:\n'
pprint(inspect.getmembers(MyObject, inspect.ismethod))
a = MyObject(1)
b = MyObject(2)
print '\nComparisons:'
for expr in [ 'a < b', 'a <= b', 'a == b', 'a >= b', 'a > b' ]:
print '\n%-6s:' % expr
result = eval(expr)
print ' result of %s: %s' % (expr, result)
The class must provide an implmentation of __eq__() and any one of the other rich comparison methods. The decorator adds implementations of the other methods that work by using the comparisons provided.
$ python functools_total_ordering.py
Methods:
[('__eq__', <unbound method MyObject.__eq__>),
('__ge__', <unbound method MyObject.__ge__>),
('__gt__', <unbound method MyObject.__gt__>),
('__init__', <unbound method MyObject.__init__>),
('__le__', <unbound method MyObject.__le__>),
('__lt__', <unbound method MyObject.__lt__>)]
Comparisons:
a < b :
testing __gt__(1, 2)
testing __eq__(1, 2)
result of a < b: True
a <= b:
testing __gt__(1, 2)
result of a <= b: True
a == b:
testing __eq__(1, 2)
result of a == b: False
a >= b:
testing __gt__(1, 2)
testing __eq__(1, 2)
result of a >= b: False
a > b :
testing __gt__(1, 2)
result of a > b: False
Collation Order¶
Since old-style comparison functions are deprecated in Python 3, the cmp argument to functions like sort() are also no longer supported. Python 2 programs that use comparison functions can use cmp_to_key() to convert them to a function that returns a collation key, which is used to determine the position in the final sequence.
import functools
class MyObject(object):
def __init__(self, val):
self.val = val
def __str__(self):
return 'MyObject(%s)' % self.val
def compare_obj(a, b):
"""Old-style comparison function.
"""
print 'comparing %s and %s' % (a, b)
return cmp(a.val, b.val)
# Make a key function using cmp_to_key()
get_key = functools.cmp_to_key(compare_obj)
def get_key_wrapper(o):
"""Wrapper function for get_key to allow for print statements.
"""
new_key = get_key(o)
print 'key_wrapper(%s) -> %s' % (o, new_key)
return new_key
objs = [ MyObject(x) for x in xrange(5, 0, -1) ]
for o in sorted(objs, key=get_key_wrapper):
print o
Note
Normally cmp_to_key() would be used directly, but in this example an extra wrapper function is introduced to print out more information as the key function is being called.
The output shows that sorted() starts by calling get_key_wrapper() for each item in the sequence to produce a key. The keys returned by cmp_to_key() are instances of a class defined in functools that implements the rich comparison API based on the return value of the provided old-style comparison function. After all of the keys are created, the sequence is sorted by comparing the keys.
$ python functools_cmp_to_key.py
key_wrapper(MyObject(5)) -> <functools.K object at 0x100466558>
key_wrapper(MyObject(4)) -> <functools.K object at 0x100466590>
key_wrapper(MyObject(3)) -> <functools.K object at 0x1004665c8>
key_wrapper(MyObject(2)) -> <functools.K object at 0x100466600>
key_wrapper(MyObject(1)) -> <functools.K object at 0x100466638>
comparing MyObject(4) and MyObject(5)
comparing MyObject(3) and MyObject(4)
comparing MyObject(2) and MyObject(3)
comparing MyObject(1) and MyObject(2)
MyObject(1)
MyObject(2)
MyObject(3)
MyObject(4)
MyObject(5)
See also
- functools
- The standard library documentation for this module.
- Rich comparison methods
- Description of the rich comparison methods from the Python Reference Guide.
