Title: Reliable Acquisition/Release Pairs
Author: Michael Hudson <email@example.com>,
Paul Moore <firstname.lastname@example.org>
Type: Standards Track
It would be nice to have a less typing-intense way of writing:
This PEP proposes a piece of syntax (a 'with' block) and a
"small-i" interface that generalizes the above.
This PEP is rejected in favor of PEP 343.
One of the advantages of Python's exception handling philosophy is
that it makes it harder to do the "wrong" thing (e.g. failing to
check the return value of some system call). Currently, this does
not apply to resource cleanup. The current syntax for acquisition
and release of a resource (for example, a lock) is
This syntax separates the acquisition and release by a (possibly
large) block of code, which makes it difficult to confirm "at a
glance" that the code manages the resource correctly. Another
common error is to code the "acquire" call within the try block,
which incorrectly releases the lock if the acquire fails.
Basic Syntax and Semantics
The syntax of a 'with' statement is as follows::
'with' [ var '=' ] expr ':'
This statement is defined as being equivalent to the following
sequence of statements:
var = expr
if hasattr(var, "__enter__"):
(The presence of an __exit__ method is *not* checked like that of
__enter__ to ensure that using inappropriate objects in with:
statements gives an error).
If the variable is omitted, an unnamed object is allocated on the
stack. In that case, the suite has no access to the unnamed object.
A number of potential extensions to the basic syntax have been
discussed on the Python Developers list. None of these extensions
are included in the solution proposed by this PEP. In many cases,
the arguments are nearly equally strong in both directions. In
such cases, the PEP has always chosen simplicity, simply because
where extra power is needed, the existing try block is available.
One proposal was for allowing multiple expressions within one
'with' statement. The __enter__ methods would be called left to
right, and the __exit__ methods right to left. The advantage of
doing so is that where more than one resource is being managed,
nested 'with' statements will result in code drifting towards the
right margin. The solution to this problem is the same as for any
other deep nesting - factor out some of the code into a separate
function. Furthermore, the question of what happens if one of the
__exit__ methods raises an exception (should the other __exit__
methods be called?) needs to be addressed.
An extension to the protocol to include an optional __except__
handler, which is called when an exception is raised, and which
can handle or re-raise the exception, has been suggested. It is
not at all clear that the semantics of this extension can be made
precise and understandable. For example, should the equivalent
code be try ... except ... else if an exception handler is
defined, and try ... finally if not? How can this be determined
at compile time, in general? The alternative is to define the
code as expanding to a try ... except inside a try ... finally.
But this may not do the right thing in real life.
The only use case identified for exception handling is with
transactional processing (commit on a clean finish, and rollback
on an exception). This is probably just as easy to handle with a
conventional try ... except ... else block, and so the PEP does
not include any support for exception handlers.
There is a potential race condition in the code specified as
equivalent to the with statement. For example, if a
KeyboardInterrupt exception is raised between the completion of
the __enter__ method call and the start of the try block, the
__exit__ method will not be called. This can lead to resource
leaks, or to deadlocks. [XXX Guido has stated that he cares about
this sort of race condition, and intends to write some C magic to
handle them. The implementation of the 'with' statement should
Should existing classes (for example, file-like objects and locks)
gain appropriate __enter__ and __exit__ methods? The obvious
reason in favour is convenience (no adapter needed). The argument
against is that if built-in files have this but (say) StringIO
does not, then code that uses "with" on a file object can't be
reused with a StringIO object. So __exit__ = close becomes a part
of the "file-like object" protocol, which user-defined classes may
need to support.
The __enter__ hook may be unnecessary - for many use cases, an
adapter class is needed and in that case, the work done by the
__enter__ hook can just as easily be done in the __init__ hook.
If a way of controlling object lifetimes explicitly was available,
the function of the __exit__ hook could be taken over by the
existing __del__ hook. An email exchange with a proponent of
this approach left one of the authors even more convinced that
it isn't the right idea...
It has been suggested that the "__exit__" method be called
"close", or that a "close" method should be considered if no
__exit__ method is found, to increase the "out-of-the-box utility"
of the "with ..." construct.
There are some similarities in concept between 'with ...' blocks
and generators, which have led to proposals that for loops could
implement the with block functionality. While neat on some
levels, we think that for loops should stick to being loops.
IEXEC: Holger Krekel -- generalised approach with XML-like syntax
(no URL found...)
Holger has much more far-reaching ideas about "execution monitors"
that are informed about details of control flow in the monitored
block. While interesting, these ideas could change the language
in deep and subtle ways and as such belong to a different PEP.
Any Smalltalk/Ruby anonymous block style extension obviously
subsumes this one.
PEP 319 is in the same area, but did not win support when aired on
This PEP proposes a new keyword, so the __future__ game will need
to be played.
Cost of Adoption
Those who claim the language is getting larger and more
complicated have something else to complain about. It's something
else to teach.
For the proposal to be useful, many file-like and lock-like
classes in the standard library and other code will have to have
__exit__ = close
or similar added.
Cost of Non-Adoption
Writing correct code continues to be more effort than writing
There are various python-list and python-dev discussions that
could be mentioned here.
 Off-list conversation between Michael Hudson and Bill Soudan
(made public with permission)
 Samuele Pedroni on python-dev
 Thread on python-dev with subject
[Python-Dev] pre-PEP: Resource-Release Support for Generators
This document has been placed in the public domain.