contextlib —- Utilities for with-statement contexts

源代码Lib/contextlib.py

此模块为涉及 with 语句的常见任务提供了实用的程序。更多信息请参见 上下文管理器类型 和 with 语句上下文管理器。

工具

提供的函数和类：

• class contextlib.AbstractContextManager
• An abstract base class for classes that implementobject.enter() and object.exit(). A defaultimplementation for object.enter() is provided which returnsself while object.exit() is an abstract method which by defaultreturns None. See also the definition of 上下文管理器类型.

3.6 新版功能.

• class contextlib.AbstractAsyncContextManager
• An abstract base class for classes that implementobject.aenter() and object.aexit(). A defaultimplementation for object.aenter() is provided which returnsself while object.aexit() is an abstract method which by defaultreturns None. See also the definition of异步上下文管理器.

3.7 新版功能.

• @contextlib.contextmanager
• This function is a decorator that can be used to define a factoryfunction for with statement context managers, without needing tocreate a class or separate enter() and exit() methods.

While many objects natively support use in with statements, sometimes aresource needs to be managed that isn't a context manager in its own right,and doesn't implement a close() method for use with contextlib.closing

An abstract example would be the following to ensure correct resourcemanagement:

from contextlib import contextmanager
 
@contextmanager
def managed_resource(*args, **kwds):
    # Code to acquire resource, e.g.:
    resource = acquire_resource(*args, **kwds)
    try:
        yield resource
    finally:
        # Code to release resource, e.g.:
        release_resource(resource)
 
>>> with managed_resource(timeout=3600) as resource:
...     # Resource is released at the end of this block,
...     # even if code in the block raises an exception

被装饰的函数在被调用时，必须返回一个 generator-iterator。这个迭代器必须只 yield 一个值出来，这个值会被用在 with 语句中，绑定到 as 后面的变量，如果给定了的话。

At the point where the generator yields, the block nested in the withstatement is executed. The generator is then resumed after the block is exited.If an unhandled exception occurs in the block, it is reraised inside thegenerator at the point where the yield occurred. Thus, you can use atry…except…finally statement to trapthe error (if any), or ensure that some cleanup takes place. If an exception istrapped merely in order to log it or to perform some action (rather than tosuppress it entirely), the generator must reraise that exception. Otherwise thegenerator context manager will indicate to the with statement thatthe exception has been handled, and execution will resume with the statementimmediately following the with statement.

contextmanager() uses ContextDecorator so the context managersit creates can be used as decorators as well as in with statements.When used as a decorator, a new generator instance is implicitly created oneach function call (this allows the otherwise "one-shot" context managerscreated by contextmanager() to meet the requirement that contextmanagers support multiple invocations in order to be used as decorators).

在 3.2 版更改: Use of ContextDecorator.

• @contextlib.asynccontextmanager
• Similar to contextmanager(), but creates anasynchronous context manager.

This function is a decorator that can be used to define a factoryfunction for async with statement asynchronous context managers,without needing to create a class or separate aenter() andaexit() methods. It must be applied to an asynchronousgenerator function.

A simple example:

from contextlib import asynccontextmanager
 
@asynccontextmanager
async def get_connection():
    conn = await acquire_db_connection()
    try:
        yield conn
    finally:
        await release_db_connection(conn)
 
async def get_all_users():
    async with get_connection() as conn:
        return conn.query('SELECT ...')

3.7 新版功能.

• contextlib.closing(thing)
• Return a context manager that closes thing upon completion of the block. Thisis basically equivalent to:

from contextlib import contextmanager
 
@contextmanager
def closing(thing):
    try:
        yield thing
    finally:
        thing.close()

And lets you write code like this:

from contextlib import closing
from urllib.request import urlopen
 
with closing(urlopen('http://www.python.org')) as page:
    for line in page:
        print(line)

without needing to explicitly close page. Even if an error occurs,page.close() will be called when the with block is exited.

• contextlib.nullcontext(enter_result=None)
• Return a context manager that returns enter_result from enter, butotherwise does nothing. It is intended to be used as a stand-in for anoptional context manager, for example:

def myfunction(arg, ignore_exceptions=False):
    if ignore_exceptions:
        # Use suppress to ignore all exceptions.
        cm = contextlib.suppress(Exception)
    else:
        # Do not ignore any exceptions, cm has no effect.
        cm = contextlib.nullcontext()
    with cm:
        # Do something

An example using enter_result:

def process_file(file_or_path):
    if isinstance(file_or_path, str):
        # If string, open file
        cm = open(file_or_path)
    else:
        # Caller is responsible for closing file
        cm = nullcontext(file_or_path)
 
    with cm as file:
        # Perform processing on the file

3.7 新版功能.

• contextlib.suppress(*exceptions)
• Return a context manager that suppresses any of the specified exceptionsif they occur in the body of a with statement and then resumes executionwith the first statement following the end of the with statement.

As with any other mechanism that completely suppresses exceptions, thiscontext manager should be used only to cover very specific errors wheresilently continuing with program execution is known to be the rightthing to do.

例如

from contextlib import suppress
 
with suppress(FileNotFoundError):
    os.remove('somefile.tmp')
 
with suppress(FileNotFoundError):
    os.remove('someotherfile.tmp')

This code is equivalent to:

try:
    os.remove('somefile.tmp')
except FileNotFoundError:
    pass
 
try:
    os.remove('someotherfile.tmp')
except FileNotFoundError:
    pass

This context manager is reentrant.

3.4 新版功能.

• contextlib.redirectstdout(_new_target)
• Context manager for temporarily redirecting sys.stdout toanother file or file-like object.

This tool adds flexibility to existing functions or classes whose outputis hardwired to stdout.

For example, the output of help() normally is sent to sys.stdout.You can capture that output in a string by redirecting the output to anio.StringIO object:

f = io.StringIO()
with redirect_stdout(f):
    help(pow)
s = f.getvalue()

To send the output of help() to a file on disk, redirect the outputto a regular file:

with open('help.txt', 'w') as f:
    with redirect_stdout(f):
        help(pow)

To send the output of help() to sys.stderr:

with redirect_stdout(sys.stderr):
    help(pow)

Note that the global side effect on sys.stdout means that thiscontext manager is not suitable for use in library code and most threadedapplications. It also has no effect on the output of subprocesses.However, it is still a useful approach for many utility scripts.

This context manager is reentrant.

3.4 新版功能.

• contextlib.redirectstderr(_new_target)
• Similar to redirect_stdout() but redirectingsys.stderr to another file or file-like object.

This context manager is reentrant.

3.5 新版功能.

• class contextlib.ContextDecorator
• A base class that enables a context manager to also be used as a decorator.

Context managers inheriting from ContextDecorator have to implemententer and exit as normal. exit retains its optionalexception handling even when used as a decorator.

ContextDecorator is used by contextmanager(), so you get thisfunctionality automatically.

ContextDecorator 的示例:

from contextlib import ContextDecorator
 
class mycontext(ContextDecorator):
    def __enter__(self):
        print('Starting')
        return self
 
    def __exit__(self, *exc):
        print('Finishing')
        return False
 
>>> @mycontext()
... def function():
...     print('The bit in the middle')
...
>>> function()
Starting
The bit in the middle
Finishing
 
>>> with mycontext():
...     print('The bit in the middle')
...
Starting
The bit in the middle
Finishing

This change is just syntactic sugar for any construct of the following form:

def f():
    with cm():
        # Do stuff

ContextDecorator lets you instead write:

@cm()def f():
# Do stuff

It makes it clear that the cm applies to the whole function, rather thanjust a piece of it (and saving an indentation level is nice, too).

Existing context managers that already have a base class can be extended byusing ContextDecorator as a mixin class:

from contextlib import ContextDecorator
 
class mycontext(ContextBaseClass, ContextDecorator):
    def __enter__(self):
        return self
 
    def __exit__(self, *exc):
        return False

注解

As the decorated function must be able to be called multiple times, theunderlying context manager must support use in multiple withstatements. If this is not the case, then the original construct with theexplicit with statement inside the function should be used.

3.2 新版功能.

• class contextlib.ExitStack
• A context manager that is designed to make it easy to programmaticallycombine other context managers and cleanup functions, especially thosethat are optional or otherwise driven by input data.

For example, a set of files may easily be handled in a single withstatement as follows:

with ExitStack() as stack:
    files = [stack.enter_context(open(fname)) for fname in filenames]
    # All opened files will automatically be closed at the end of
    # the with statement, even if attempts to open files later
    # in the list raise an exception

Each instance maintains a stack of registered callbacks that are called inreverse order when the instance is closed (either explicitly or implicitlyat the end of a with statement). Note that callbacks are _not_invoked implicitly when the context stack instance is garbage collected.

This stack model is used so that context managers that acquire theirresources in their init method (such as file objects) can behandled correctly.

Since registered callbacks are invoked in the reverse order ofregistration, this ends up behaving as if multiple nested withstatements had been used with the registered set of callbacks. This evenextends to exception handling - if an inner callback suppresses or replacesan exception, then outer callbacks will be passed arguments based on thatupdated state.

This is a relatively low level API that takes care of the details ofcorrectly unwinding the stack of exit callbacks. It provides a suitablefoundation for higher level context managers that manipulate the exitstack in application specific ways.

3.3 新版功能.

• entercontext(_cm)
• Enters a new context manager and adds its exit() method tothe callback stack. The return value is the result of the contextmanager's own enter() method.

These context managers may suppress exceptions just as they normallywould if used directly as part of a with statement.

• push(exit)
• Adds a context manager's exit() method to the callback stack.

As enter is not invoked, this method can be used to coverpart of an enter() implementation with a context manager's ownexit() method.

If passed an object that is not a context manager, this method assumesit is a callback with the same signature as a context manager'sexit() method and adds it directly to the callback stack.

By returning true values, these callbacks can suppress exceptions thesame way context manager exit() methods can.

The passed in object is returned from the function, allowing thismethod to be used as a function decorator.

• callback(callback, *args, **kwds)
• Accepts an arbitrary callback function and arguments and adds it tothe callback stack.

Unlike the other methods, callbacks added this way cannot suppressexceptions (as they are never passed the exception details).

The passed in callback is returned from the function, allowing thismethod to be used as a function decorator.

• pop_all()
• Transfers the callback stack to a fresh ExitStack instanceand returns it. No callbacks are invoked by this operation - instead,they will now be invoked when the new stack is closed (eitherexplicitly or implicitly at the end of a with statement).

For example, a group of files can be opened as an "all or nothing"operation as follows:

with ExitStack() as stack:
    files = [stack.enter_context(open(fname)) for fname in filenames]
    # Hold onto the close method, but don't call it yet.
    close_files = stack.pop_all().close
    # If opening any file fails, all previously opened files will be
    # closed automatically. If all files are opened successfully,
    # they will remain open even after the with statement ends.
    # close_files() can then be invoked explicitly to close them all.

• close()
• Immediately unwinds the callback stack, invoking callbacks in thereverse order of registration. For any context managers and exitcallbacks registered, the arguments passed in will indicate that noexception occurred.

• class contextlib.AsyncExitStack
• An asynchronous context manager, similarto ExitStack, that supports combining both synchronous andasynchronous context managers, as well as having coroutines forcleanup logic.

The close() method is not implemented, aclose() must be usedinstead.

• enterasync_context(_cm)

• Similar to enter_context() but expects an asynchronous contextmanager.

• pushasync_exit(_exit)

• Similar to push() but expects either an asynchronous context manageror a coroutine function.

• pushasync_callback(_callback, *args, **kwds)

• Similar to callback() but expects a coroutine function.

• aclose()

• Similar to close() but properly handles awaitables.

Continuing the example for asynccontextmanager():

async with AsyncExitStack() as stack:
    connections = [await stack.enter_async_context(get_connection())
        for i in range(5)]
    # All opened connections will automatically be released at the end of
    # the async with statement, even if attempts to open a connection
    # later in the list raise an exception.

3.7 新版功能.

例子和配方

This section describes some examples and recipes for making effective use ofthe tools provided by contextlib.

Supporting a variable number of context managers

The primary use case for ExitStack is the one given in the classdocumentation: supporting a variable number of context managers and othercleanup operations in a single with statement. The variabilitymay come from the number of context managers needed being driven by userinput (such as opening a user specified collection of files), or fromsome of the context managers being optional:

with ExitStack() as stack:
    for resource in resources:
        stack.enter_context(resource)
    if need_special_resource():
        special = acquire_special_resource()
        stack.callback(release_special_resource, special)
    # Perform operations that use the acquired resources

As shown, ExitStack also makes it quite easy to use withstatements to manage arbitrary resources that don't natively support thecontext management protocol.

Catching exceptions from enter methods

It is occasionally desirable to catch exceptions from an entermethod implementation, without inadvertently catching exceptions fromthe with statement body or the context manager's exitmethod. By using ExitStack the steps in the context managementprotocol can be separated slightly in order to allow this:

stack = ExitStack()
try:
    x = stack.enter_context(cm)
except Exception:
    # handle __enter__ exception
else:
    with stack:
        # Handle normal case

Actually needing to do this is likely to indicate that the underlying APIshould be providing a direct resource management interface for use withtry/except/finally statements, but notall APIs are well designed in that regard. When a context manager is theonly resource management API provided, then ExitStack can make iteasier to handle various situations that can't be handled directly in awith statement.

Cleaning up in an enter implementation

As noted in the documentation of ExitStack.push(), thismethod can be useful in cleaning up an already allocated resource if latersteps in the enter() implementation fail.

Here's an example of doing this for a context manager that accepts resourceacquisition and release functions, along with an optional validation function,and maps them to the context management protocol:

from contextlib import contextmanager, AbstractContextManager, ExitStack
 
class ResourceManager(AbstractContextManager):
 
    def __init__(self, acquire_resource, release_resource, check_resource_ok=None):
        self.acquire_resource = acquire_resource
        self.release_resource = release_resource
        if check_resource_ok is None:
            def check_resource_ok(resource):
                return True
        self.check_resource_ok = check_resource_ok
 
    @contextmanager
    def _cleanup_on_error(self):
        with ExitStack() as stack:
            stack.push(self)
            yield
            # The validation check passed and didn't raise an exception
            # Accordingly, we want to keep the resource, and pass it
            # back to our caller
            stack.pop_all()
 
    def __enter__(self):
        resource = self.acquire_resource()
        with self._cleanup_on_error():
            if not self.check_resource_ok(resource):
                msg = "Failed validation for {!r}"
                raise RuntimeError(msg.format(resource))
        return resource
 
    def __exit__(self, *exc_details):
        # We don't need to duplicate any of our resource release logic
        self.release_resource()

Replacing any use of try-finally and flag variables

A pattern you will sometimes see is a try-finally statement with a flagvariable to indicate whether or not the body of the finally clause shouldbe executed. In its simplest form (that can't already be handled just byusing an except clause instead), it looks something like this:

cleanup_needed = True
try:
    result = perform_operation()
    if result:
        cleanup_needed = False
finally:
    if cleanup_needed:
        cleanup_resources()

As with any try statement based code, this can cause problems fordevelopment and review, because the setup code and the cleanup code can endup being separated by arbitrarily long sections of code.

ExitStack makes it possible to instead register a callback forexecution at the end of a with statement, and then later decide to skipexecuting that callback:

from contextlib import ExitStack
 
with ExitStack() as stack:
    stack.callback(cleanup_resources)
    result = perform_operation()
    if result:
        stack.pop_all()

This allows the intended cleanup up behaviour to be made explicit up front,rather than requiring a separate flag variable.

If a particular application uses this pattern a lot, it can be simplifiedeven further by means of a small helper class:

from contextlib import ExitStack
 
class Callback(ExitStack):
    def __init__(self, callback, *args, **kwds):
        super(Callback, self).__init__()
        self.callback(callback, *args, **kwds)
 
    def cancel(self):
        self.pop_all()
 
with Callback(cleanup_resources) as cb:
    result = perform_operation()
    if result:
        cb.cancel()

If the resource cleanup isn't already neatly bundled into a standalonefunction, then it is still possible to use the decorator form ofExitStack.callback() to declare the resource cleanup inadvance:

from contextlib import ExitStack
 
with ExitStack() as stack:
    @stack.callback
    def cleanup_resources():
        ...
    result = perform_operation()
    if result:
        stack.pop_all()

Due to the way the decorator protocol works, a callback functiondeclared this way cannot take any parameters. Instead, any resources tobe released must be accessed as closure variables.

Using a context manager as a function decorator

ContextDecorator makes it possible to use a context manager inboth an ordinary with statement and also as a function decorator.

For example, it is sometimes useful to wrap functions or groups of statementswith a logger that can track the time of entry and time of exit. Rather thanwriting both a function decorator and a context manager for the task,inheriting from ContextDecorator provides both capabilities in asingle definition:

from contextlib import ContextDecorator
import logging
 
logging.basicConfig(level=logging.INFO)
 
class track_entry_and_exit(ContextDecorator):
    def __init__(self, name):
        self.name = name
 
    def __enter__(self):
        logging.info('Entering: %s', self.name)
 
    def __exit__(self, exc_type, exc, exc_tb):
        logging.info('Exiting: %s', self.name)

Instances of this class can be used as both a context manager:

with track_entry_and_exit('widget loader'):
    print('Some time consuming activity goes here')
    load_widget()

And also as a function decorator:

@track_entry_and_exit('widget loader')def activity():    print('Some time consuming activity goes here')    load_widget()

Note that there is one additional limitation when using context managersas function decorators: there's no way to access the return value ofenter(). If that value is needed, then it is still necessary to usean explicit with statement.

参见

• PEP 343 - "with" 语句
• Python with 语句的规范描述、背景和示例。

Single use, reusable and reentrant context managers

Most context managers are written in a way that means they can only beused effectively in a with statement once. These single usecontext managers must be created afresh each time they're used -attempting to use them a second time will trigger an exception orotherwise not work correctly.

This common limitation means that it is generally advisable to createcontext managers directly in the header of the with statementwhere they are used (as shown in all of the usage examples above).

Files are an example of effectively single use context managers, sincethe first with statement will close the file, preventing anyfurther IO operations using that file object.

Context managers created using contextmanager() are also single usecontext managers, and will complain about the underlying generator failingto yield if an attempt is made to use them a second time:

>>> from contextlib import contextmanager
>>> @contextmanager
... def singleuse():
...     print("Before")
...     yield
...     print("After")
...
>>> cm = singleuse()
>>> with cm:
...     pass
...
Before
After
>>> with cm:
...     pass
...
Traceback (most recent call last):
    ...
RuntimeError: generator didn't yield

Reentrant context managers

More sophisticated context managers may be "reentrant". These contextmanagers can not only be used in multiple with statements,but may also be used inside a with statement that is alreadyusing the same context manager.

threading.RLock is an example of a reentrant context manager, as aresuppress() and redirect_stdout(). Here's a very simple example ofreentrant use:

>>> from contextlib import redirect_stdout
>>> from io import StringIO
>>> stream = StringIO()
>>> write_to_stream = redirect_stdout(stream)
>>> with write_to_stream:
...     print("This is written to the stream rather than stdout")
...     with write_to_stream:
...         print("This is also written to the stream")
...
>>> print("This is written directly to stdout")
This is written directly to stdout
>>> print(stream.getvalue())
This is written to the stream rather than stdout
This is also written to the stream

Real world examples of reentrancy are more likely to involve multiplefunctions calling each other and hence be far more complicated than thisexample.

Note also that being reentrant is not the same thing as being thread safe.redirect_stdout(), for example, is definitely not thread safe, as itmakes a global modification to the system state by binding sys.stdoutto a different stream.

Reusable context managers

Distinct from both single use and reentrant context managers are "reusable"context managers (or, to be completely explicit, "reusable, but notreentrant" context managers, since reentrant context managers are alsoreusable). These context managers support being used multiple times, butwill fail (or otherwise not work correctly) if the specific context managerinstance has already been used in a containing with statement.

threading.Lock is an example of a reusable, but not reentrant,context manager (for a reentrant lock, it is necessary to usethreading.RLock instead).

Another example of a reusable, but not reentrant, context manager isExitStack, as it invokes all currently registered callbackswhen leaving any with statement, regardless of where those callbackswere added:

>>> from contextlib import ExitStack
>>> stack = ExitStack()
>>> with stack:
...     stack.callback(print, "Callback: from first context")
...     print("Leaving first context")
...
Leaving first context
Callback: from first context
>>> with stack:
...     stack.callback(print, "Callback: from second context")
...     print("Leaving second context")
...
Leaving second context
Callback: from second context
>>> with stack:
...     stack.callback(print, "Callback: from outer context")
...     with stack:
...         stack.callback(print, "Callback: from inner context")
...         print("Leaving inner context")
...     print("Leaving outer context")
...
Leaving inner context
Callback: from inner context
Callback: from outer context
Leaving outer context

As the output from the example shows, reusing a single stack object acrossmultiple with statements works correctly, but attempting to nest themwill cause the stack to be cleared at the end of the innermost withstatement, which is unlikely to be desirable behaviour.

Using separate ExitStack instances instead of reusing a singleinstance avoids that problem:

>>> from contextlib import ExitStack
>>> with ExitStack() as outer_stack:
...     outer_stack.callback(print, "Callback: from outer context")
...     with ExitStack() as inner_stack:
...         inner_stack.callback(print, "Callback: from inner context")
...         print("Leaving inner context")
...     print("Leaving outer context")
...
Leaving inner context
Callback: from inner context
Leaving outer context
Callback: from outer context