Navigation

contextlib – Context manager utilities

Purpose:Utilities for creating and working with context managers.
Available In:2.5 and later

The contextlib module contains utilities for working with context managers and the with statement.

Note

Context managers are tied to the with statement. Since with is officially part of Python 2.6, you have to import it from __future__ before using contextlib in Python 2.5.

Context Manager API

A context manager is responsible for a resource within a code block, possibly creating it when the block is entered and then cleaning it up after the block is exited. For example, files support the context manager API to make it easy to ensure they are closed after all reading or writing is done.

with open('/tmp/pymotw.txt', 'wt') as f:
    f.write('contents go here')
# file is automatically closed

A context manager is enabled by the with statement, and the API involves two methods. The __enter__() method is run when execution flow enters the code block inside the with. It returns an object to be used within the context. When execution flow leaves the with block, the __exit__() method of the context manager is called to clean up any resources being used.

class Context(object):

    def __init__(self):
        print '__init__()'

    def __enter__(self):
        print '__enter__()'
        return self
    
    def __exit__(self, exc_type, exc_val, exc_tb):
        print '__exit__()'
        
with Context():
    print 'Doing work in the context'

Combining a context manager and the with statement is a more compact way of writing a try:finally block, since the context manager’s __exit__() method is always called, even if an exception is raised.

$ python contextlib_api.py

__init__()
__enter__()
Doing work in the context
__exit__()

__enter__() can return any object to be associated with a name specified in the as clause of the with statement. In this example, the Context returns an object that uses the open context.

class WithinContext(object):

    def __init__(self, context):
        print 'WithinContext.__init__(%s)' % context
        
    def do_something(self):
        print 'WithinContext.do_something()'

    def __del__(self):
        print 'WithinContext.__del__'
        

class Context(object):

    def __init__(self):
        print 'Context.__init__()'

    def __enter__(self):
        print 'Context.__enter__()'
        return WithinContext(self)

    def __exit__(self, exc_type, exc_val, exc_tb):
        print 'Context.__exit__()'
    
with Context() as c:
    c.do_something()

It can be a little confusing, but the value associated with the variable c is the object returned by __enter__() and not the Context instance created in the with statement.

$ python contextlib_api_other_object.py

Context.__init__()
Context.__enter__()
WithinContext.__init__(<__main__.Context object at 0x10045f6d0>)
WithinContext.do_something()
Context.__exit__()
WithinContext.__del__

The __exit__() method receives arguments containing details of any exception raised in the with block.

class Context(object):

    def __init__(self, handle_error):
        print '__init__(%s)' % handle_error
        self.handle_error = handle_error

    def __enter__(self):
        print '__enter__()'
        return self
    
    def __exit__(self, exc_type, exc_val, exc_tb):
        print '__exit__(%s, %s, %s)' % (exc_type, exc_val, exc_tb)
        return self.handle_error
        
with Context(True):
    raise RuntimeError('error message handled')

print

with Context(False):
    raise RuntimeError('error message propagated')

If the context manager can handle the exception, __exit__() should return a true value to indicate that the exception does not need to be propagated. Returning false causes the exception to be re-raised after __exit__() returns.

$ python contextlib_api_error.py

__init__(True)
__enter__()
__exit__(<type 'exceptions.RuntimeError'>, error message handled, <traceback object at 0x10046a5f0>)

__init__(False)
__enter__()
__exit__(<type 'exceptions.RuntimeError'>, error message propagated, <traceback object at 0x10046a680>)
Traceback (most recent call last):
  File "contextlib_api_error.py", line 30, in <module>
    raise RuntimeError('error message propagated')
RuntimeError: error message propagated

From Generator to Context Manager

Creating context managers the traditional way, by writing a class with __enter__() and __exit__() methods, is not difficult. But sometimes it is more overhead than you need just to manage a trivial bit of context. In those sorts of situations, you can use the contextmanager() decorator to convert a generator function into a context manager.

import contextlib

@contextlib.contextmanager
def make_context():
    print '  entering'
    try:
        yield {}
    except RuntimeError, err:
        print '  ERROR:', err
    finally:
        print '  exiting'

print 'Normal:'
with make_context() as value:
    print '  inside with statement:', value

print
print 'Handled error:'
with make_context() as value:
    raise RuntimeError('showing example of handling an error')

print
print 'Unhandled error:'
with make_context() as value:
    raise ValueError('this exception is not handled')

The generator should initialize the context, yield exactly one time, then clean up the context. The value yielded, if any, is bound to the variable in the as clause of the with statement. Exceptions from within the with block are re-raised inside the generator, so they can be handled there.

$ python contextlib_contextmanager.py

Normal:
  entering
  inside with statement: {}
  exiting

Handled error:
  entering
  ERROR: showing example of handling an error
  exiting

Unhandled error:
  entering
  exiting
Traceback (most recent call last):
  File "contextlib_contextmanager.py", line 34, in <module>
    raise ValueError('this exception is not handled')
ValueError: this exception is not handled

Nesting Contexts

At times it is necessary to manage multiple contexts simultaneously (such as when copying data between input and output file handles, for example). It is possible to nest with statements one inside another. If the outer contexts do not need their own separate block, though, this adds to the indention level without giving any real benefit. Using nested() nests the contexts using a single with statement.

import contextlib

@contextlib.contextmanager
def make_context(name):
    print 'entering:', name
    yield name
    print 'exiting :', name

with contextlib.nested(make_context('A'), make_context('B'), make_context('C')) as (A, B, C):
    print 'inside with statement:', A, B, C

Notice that the contexts are exited in the reverse order in which they are entered.

$ python contextlib_nested.py

entering: A
entering: B
entering: C
inside with statement: A B C
exiting : C
exiting : B
exiting : A

In Python 2.7 and later, nested() is deprecated because the with statement supports nesting directly.

import contextlib

@contextlib.contextmanager
def make_context(name):
    print 'entering:', name
    yield name
    print 'exiting :', name

with make_context('A') as A, make_context('B') as B, make_context('C') as C:
    print 'inside with statement:', A, B, C

Each context manager and optional as clause are separated by a comma (,). The effect is similar to using nested(), but avoids some of the edge-cases around error handling that nested() could not implement correctly.

$ python contextlib_nested_with.py

entering: A
entering: B
entering: C
inside with statement: A B C
exiting : C
exiting : B
exiting : A

Closing Open Handles

The file class supports the context manager API directly, but some other objects that represent open handles do not. The example given in the standard library documentation for contextlib is the object returned from urllib.urlopen(). There are other legacy classes that use a close() method but do not support the context manager API. To ensure that a handle is closed, use closing() to create a context manager for it.

import contextlib

class Door(object):
    def __init__(self):
        print '  __init__()'
    def close(self):
        print '  close()'

print 'Normal Example:'
with contextlib.closing(Door()) as door:
    print '  inside with statement'

print
print 'Error handling example:'
try:
    with contextlib.closing(Door()) as door:
        print '  raising from inside with statement'
        raise RuntimeError('error message')
except Exception, err:
    print '  Had an error:', err

The handle is closed whether there is an error in the with block or not.

$ python contextlib_closing.py

Normal Example:
  __init__()
  inside with statement
  close()

Error handling example:
  __init__()
  raising from inside with statement
  close()
  Had an error: error message

See also

contextlib
The standard library documentation for this module.
PEP 343
The with statement.
Context Manager Types
Description of the context manager API from the standard library documentation.
With Statement Context Managers
Description of the context manager API from the Python Reference Guide.

Navigation