Source

python-peps / pep-3134.txt

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
PEP: 3134
Title: Exception Chaining and Embedded Tracebacks
Version: $Revision$
Last-Modified: $Date$
Author: Ka-Ping Yee
Status: Final
Type: Standards Track
Content-Type: text/plain
Created: 12-May-2005
Python-Version: 3.0
Post-History:


Numbering Note

    This PEP started its life as PEP 344.  Since it is now targeted
    for Python 3000, it has been moved into the 3xxx space.


Abstract

    This PEP proposes three standard attributes on exception instances:
    the '__context__' attribute for implicitly chained exceptions, the
    '__cause__' attribute for explicitly chained exceptions, and the
    '__traceback__' attribute for the traceback.  A new "raise ... from"
    statement sets the '__cause__' attribute.


Motivation

    During the handling of one exception (exception A), it is possible
    that another exception (exception B) may occur.  In today's Python
    (version 2.4), if this happens, exception B is propagated outward
    and exception A is lost.  In order to debug the problem, it is
    useful to know about both exceptions.  The '__context__' attribute
    retains this information automatically.

    Sometimes it can be useful for an exception handler to intentionally
    re-raise an exception, either to provide extra information or to
    translate an exception to another type.  The '__cause__' attribute
    provides an explicit way to record the direct cause of an exception.

    In today's Python implementation, exceptions are composed of three
    parts: the type, the value, and the traceback.  The 'sys' module,
    exposes the current exception in three parallel variables, exc_type,
    exc_value, and exc_traceback, the sys.exc_info() function returns a
    tuple of these three parts, and the 'raise' statement has a
    three-argument form accepting these three parts.  Manipulating
    exceptions often requires passing these three things in parallel,
    which can be tedious and error-prone.  Additionally, the 'except'
    statement can only provide access to the value, not the traceback.
    Adding the '__traceback__' attribute to exception values makes all
    the exception information accessible from a single place.


History

    Raymond Hettinger [1] raised the issue of masked exceptions on
    Python-Dev in January 2003 and proposed a PyErr_FormatAppend()
    function that C modules could use to augment the currently active
    exception with more information.  Brett Cannon [2] brought up
    chained exceptions again in June 2003, prompting a long discussion.

    Greg Ewing [3] identified the case of an exception occuring in a
    'finally' block during unwinding triggered by an original exception,
    as distinct from the case of an exception occuring in an 'except'
    block that is handling the original exception.

    Greg Ewing [4] and Guido van Rossum [5], and probably others, have
    previously mentioned adding a traceback attribute to Exception
    instances.  This is noted in PEP 3000.

    This PEP was motivated by yet another recent Python-Dev reposting
    of the same ideas [6] [7].


Rationale

    The Python-Dev discussions revealed interest in exception chaining
    for two quite different purposes.  To handle the unexpected raising
    of a secondary exception, the exception must be retained implicitly.
    To support intentional translation of an exception, there must be a
    way to chain exceptions explicitly.  This PEP addresses both.

    Several attribute names for chained exceptions have been suggested
    on Python-Dev [2], including 'cause', 'antecedent', 'reason',
    'original', 'chain', 'chainedexc', 'exc_chain', 'excprev',
    'previous', and 'precursor'.  For an explicitly chained exception,
    this PEP suggests '__cause__' because of its specific meaning.  For
    an implicitly chained exception, this PEP proposes the name
    '__context__' because the intended meaning is more specific than
    temporal precedence but less specific than causation: an exception
    occurs in the context of handling another exception.
    
    This PEP suggests names with leading and trailing double-underscores
    for these three attributes because they are set by the Python VM.
    Only in very special cases should they be set by normal assignment.

    This PEP handles exceptions that occur during 'except' blocks and
    'finally' blocks in the same way.  Reading the traceback makes it
    clear where the exceptions occurred, so additional mechanisms for
    distinguishing the two cases would only add unnecessary complexity.

    This PEP proposes that the outermost exception object (the one
    exposed for matching by 'except' clauses) be the most recently
    raised exception for compatibility with current behaviour.

    This PEP proposes that tracebacks display the outermost exception
    last, because this would be consistent with the chronological order
    of tracebacks (from oldest to most recent frame) and because the
    actual thrown exception is easier to find on the last line.

    To keep things simpler, the C API calls for setting an exception
    will not automatically set the exception's '__context__'.  Guido
    van Rossum has has expressed concerns with making such changes [8].

    As for other languages, Java and Ruby both discard the original
    exception when another exception occurs in a 'catch'/'rescue' or
    'finally'/'ensure' clause.  Perl 5 lacks built-in structured
    exception handling.  For Perl 6, RFC number 88 [9] proposes an exception
    mechanism that implicitly retains chained exceptions in an array
    named @@.  In that RFC, the most recently raised exception is
    exposed for matching, as in this PEP; also, arbitrary expressions
    (possibly involving @@) can be evaluated for exception matching.

    Exceptions in C# contain a read-only 'InnerException' property that
    may point to another exception.  Its documentation [10] says that
    "When an exception X is thrown as a direct result of a previous
    exception Y, the InnerException property of X should contain a
    reference to Y."  This property is not set by the VM automatically;
    rather, all exception constructors take an optional 'innerException'
    argument to set it explicitly.  The '__cause__' attribute fulfills
    the same purpose as InnerException, but this PEP proposes a new form
    of 'raise' rather than extending the constructors of all exceptions.
    C# also provides a GetBaseException method that jumps directly to
    the end of the InnerException chain; this PEP proposes no analog.

    The reason all three of these attributes are presented together in
    one proposal is that the '__traceback__' attribute provides
    convenient access to the traceback on chained exceptions.


Implicit Exception Chaining

    Here is an example to illustrate the '__context__' attribute.

        def compute(a, b):
            try:
                a/b
            except Exception, exc:
                log(exc)

        def log(exc):
            file = open('logfile.txt')  # oops, forgot the 'w'
            print >>file, exc
            file.close()

    Calling compute(0, 0) causes a ZeroDivisionError.  The compute()
    function catches this exception and calls log(exc), but the log()
    function also raises an exception when it tries to write to a
    file that wasn't opened for writing.

    In today's Python, the caller of compute() gets thrown an IOError.
    The ZeroDivisionError is lost.  With the proposed change, the
    instance of IOError has an additional '__context__' attribute that
    retains the ZeroDivisionError.

    The following more elaborate example demonstrates the handling of a
    mixture of 'finally' and 'except' clauses:

        def main(filename):
            file = open(filename)       # oops, forgot the 'w'
            try:
                try:
                    compute()
                except Exception, exc:
                    log(file, exc)
            finally:
                file.clos()             # oops, misspelled 'close'
        
        def compute():
            1/0
        
        def log(file, exc):
            try:
                print >>file, exc       # oops, file is not writable
            except:
                display(exc)
        
        def display(exc):
            print ex                    # oops, misspelled 'exc'

    Calling main() with the name of an existing file will trigger four
    exceptions.  The ultimate result will be an AttributeError due to
    the misspelling of 'clos', whose __context__ points to a NameError
    due to the misspelling of 'ex', whose __context__ points to an
    IOError due to the file being read-only, whose __context__ points to
    a ZeroDivisionError, whose __context__ attribute is None.

    The proposed semantics are as follows:

    1.  Each thread has an exception context initially set to None.
    
    2.  Whenever an exception is raised, if the exception instance does
        not already have a '__context__' attribute, the interpreter sets
        it equal to the thread's exception context.

    3.  Immediately after an exception is raised, the thread's exception
        context is set to the exception.

    4.  Whenever the interpreter exits an 'except' block by reaching the
        end or executing a 'return', 'yield', 'continue', or 'break'
        statement, the thread's exception context is set to None.


Explicit Exception Chaining

    The '__cause__' attribute on exception objects is always initialized
    to None.  It is set by a new form of the 'raise' statement:

        raise EXCEPTION from CAUSE

    which is equivalent to:

        exc = EXCEPTION
        exc.__cause__ = CAUSE
        raise exc
    
    In the following example, a database provides implementations for a
    few different kinds of storage, with file storage as one kind.  The
    database designer wants errors to propagate as DatabaseError objects
    so that the client doesn't have to be aware of the storage-specific
    details, but doesn't want to lose the underlying error information.

        class DatabaseError(Exception):
            pass

        class FileDatabase(Database):
            def __init__(self, filename):
                try:
                    self.file = open(filename)
                except IOError, exc:
                    raise DatabaseError('failed to open') from exc

    If the call to open() raises an exception, the problem will be
    reported as a DatabaseError, with a __cause__ attribute that reveals
    the IOError as the original cause.


Traceback Attribute

    The following example illustrates the '__traceback__' attribute.

        def do_logged(file, work):
            try:
                work()
            except Exception, exc:
                write_exception(file, exc)
                raise exc

        from traceback import format_tb

        def write_exception(file, exc):
            ...
            type = exc.__class__
            message = str(exc)
            lines = format_tb(exc.__traceback__)
            file.write(... type ... message ... lines ...)
            ...

    In today's Python, the do_logged() function would have to extract
    the traceback from sys.exc_traceback or sys.exc_info()[2] and pass
    both the value and the traceback to write_exception().  With the
    proposed change, write_exception() simply gets one argument and
    obtains the exception using the '__traceback__' attribute.

    The proposed semantics are as follows:

    1.  Whenever an exception is caught, if the exception instance does
        not already have a '__traceback__' attribute, the interpreter
        sets it to the newly caught traceback.


Enhanced Reporting

    The default exception handler will be modified to report chained
    exceptions.  The chain of exceptions is traversed by following the
    '__cause__' and '__context__' attributes, with '__cause__' taking
    priority.  In keeping with the chronological order of tracebacks,
    the most recently raised exception is displayed last; that is, the
    display begins with the description of the innermost exception and
    backs up the chain to the outermost exception.  The tracebacks are
    formatted as usual, with one of the lines:

        The above exception was the direct cause of the following exception:

    or

        During handling of the above exception, another exception occurred:

    between tracebacks, depending whether they are linked by __cause__
    or __context__ respectively.  Here is a sketch of the procedure:
    
        def print_chain(exc):
            if exc.__cause__:
                print_chain(exc.__cause__)
                print '\nThe above exception was the direct cause...'
            elif exc.__context__:
                print_chain(exc.__context__)
                print '\nDuring handling of the above exception, ...'
            print_exc(exc)

    In the 'traceback' module, the format_exception, print_exception,
    print_exc, and print_last functions will be updated to accept an
    optional 'chain' argument, True by default.  When this argument is
    True, these functions will format or display the entire chain of
    exceptions as just described.  When it is False, these functions
    will format or display only the outermost exception.

    The 'cgitb' module should also be updated to display the entire
    chain of exceptions.


C API

    The PyErr_Set* calls for setting exceptions will not set the
    '__context__' attribute on exceptions.  PyErr_NormalizeException
    will always set the 'traceback' attribute to its 'tb' argument and
    the '__context__' and '__cause__' attributes to None.

    A new API function, PyErr_SetContext(context), will help C
    programmers provide chained exception information.  This function
    will first normalize the current exception so it is an instance,
    then set its '__context__' attribute.  A similar API function,
    PyErr_SetCause(cause), will set the '__cause__' attribute.


Compatibility

    Chained exceptions expose the type of the most recent exception, so
    they will still match the same 'except' clauses as they do now.

    The proposed changes should not break any code unless it sets or
    uses attributes named '__context__', '__cause__', or '__traceback__'
    on exception instances.  As of 2005-05-12, the Python standard
    library contains no mention of such attributes.


Open Issue:  Extra Information

    Walter Dörwald [11] expressed a desire to attach extra information
    to an exception during its upward propagation without changing its
    type.  This could be a useful feature, but it is not addressed by
    this PEP.  It could conceivably be addressed by a separate PEP
    establishing conventions for other informational attributes on
    exceptions.


Open Issue:  Suppressing Context

    As written, this PEP makes it impossible to suppress '__context__',
    since setting exc.__context__ to None in an 'except' or 'finally'
    clause will only result in it being set again when exc is raised.


Open Issue:  Limiting Exception Types

    To improve encapsulation, library implementors may want to wrap all
    implementation-level exceptions with an application-level exception.
    One could try to wrap exceptions by writing this:

        try:
            ... implementation may raise an exception ...
        except:
            import sys
            raise ApplicationError from sys.exc_value

    or this:

        try:
            ... implementation may raise an exception ...
        except Exception, exc:
            raise ApplicationError from exc

    but both are somewhat flawed.  It would be nice to be able to name
    the current exception in a catch-all 'except' clause, but that isn't
    addressed here.  Such a feature would allow something like this:

        try:
            ... implementation may raise an exception ...
        except *, exc:
            raise ApplicationError from exc


Open Issue:  yield

    The exception context is lost when a 'yield' statement is executed;
    resuming the frame after the 'yield' does not restore the context.
    Addressing this problem is out of the scope of this PEP; it is not a
    new problem, as demonstrated by the following example:

        >>> def gen():
        ...     try:
        ...         1/0
        ...     except:
        ...         yield 3
        ...         raise
        ...
        >>> g = gen()
        >>> g.next()
        3
        >>> g.next()
        TypeError: exceptions must be classes, instances, or strings
        (deprecated), not NoneType


Open Issue:  Garbage Collection

    The strongest objection to this proposal has been that it creates
    cycles between exceptions and stack frames [12].  Collection of
    cyclic garbage (and therefore resource release) can be greatly
    delayed.

        >>> try:
        >>>   1/0
        >>> except Exception, err:
        >>>   pass

    will introduce a cycle from err -> traceback -> stack frame -> err,
    keeping all locals in the same scope alive until the next GC happens.

    Today, these locals would go out of scope.  There is lots of code
    which assumes that "local" resources -- particularly open files -- will
    be closed quickly.  If closure has to wait for the next GC, a program
    (which runs fine today) may run out of file handles.

    Making the __traceback__ attribute a weak reference would avoid the
    problems with cyclic garbage.  Unfortunately, it would make saving
    the Exception for later (as unittest does) more awkward, and it would
    not allow as much cleanup of the sys module.

    A possible alternate solution, suggested by Adam Olsen, would be to
    instead turn the reference from the stack frame to the 'err' variable
    into a weak reference when the variable goes out of scope [13].

  
Possible Future Compatible Changes

    These changes are consistent with the appearance of exceptions as
    a single object rather than a triple at the interpreter level.

    - If PEP 340 or PEP 343 is accepted, replace the three (type, value,
      traceback) arguments to __exit__ with a single exception argument.

    - Deprecate sys.exc_type, sys.exc_value, sys.exc_traceback, and
      sys.exc_info() in favour of a single member, sys.exception.

    - Deprecate sys.last_type, sys.last_value, and sys.last_traceback
      in favour of a single member, sys.last_exception.

    - Deprecate the three-argument form of the 'raise' statement in
      favour of the one-argument form.

    - Upgrade cgitb.html() to accept a single value as its first
      argument as an alternative to a (type, value, traceback) tuple.


Possible Future Incompatible Changes

    These changes might be worth considering for Python 3000.

    - Remove sys.exc_type, sys.exc_value, sys.exc_traceback, and
      sys.exc_info().

    - Remove sys.last_type, sys.last_value, and sys.last_traceback.

    - Replace the three-argument sys.excepthook with a one-argument
      API, and changing the 'cgitb' module to match.

    - Remove the three-argument form of the 'raise' statement.

    - Upgrade traceback.print_exception to accept an 'exception'
      argument instead of the type, value, and traceback arguments.


Implementation

    The __traceback__ and __cause__ attributes and the new raise syntax were
    implemented in revision 57783 [14].


Acknowledgements

    Brett Cannon, Greg Ewing, Guido van Rossum, Jeremy Hylton, Phillip
    J. Eby, Raymond Hettinger, Walter Dörwald, and others.


References

    [1] Raymond Hettinger, "Idea for avoiding exception masking"
        http://mail.python.org/pipermail/python-dev/2003-January/032492.html

    [2] Brett Cannon explains chained exceptions
        http://mail.python.org/pipermail/python-dev/2003-June/036063.html

    [3] Greg Ewing points out masking caused by exceptions during finally
        http://mail.python.org/pipermail/python-dev/2003-June/036290.html

    [4] Greg Ewing suggests storing the traceback in the exception object
        http://mail.python.org/pipermail/python-dev/2003-June/036092.html

    [5] Guido van Rossum mentions exceptions having a traceback attribute
        http://mail.python.org/pipermail/python-dev/2005-April/053060.html

    [6] Ka-Ping Yee, "Tidier Exceptions"
        http://mail.python.org/pipermail/python-dev/2005-May/053671.html

    [7] Ka-Ping Yee, "Chained Exceptions"
        http://mail.python.org/pipermail/python-dev/2005-May/053672.html

    [8] Guido van Rossum discusses automatic chaining in PyErr_Set*
        http://mail.python.org/pipermail/python-dev/2003-June/036180.html

    [9] Tony Olensky, "Omnibus Structured Exception/Error Handling Mechanism"
        http://dev.perl.org/perl6/rfc/88.html
     
   [10] MSDN .NET Framework Library, "Exception.InnerException Property"
        http://msdn.microsoft.com/library/en-us/cpref/html/frlrfsystemexceptionclassinnerexceptiontopic.asp

   [11] Walter Dörwald suggests wrapping exceptions to add details
        http://mail.python.org/pipermail/python-dev/2003-June/036148.html

   [12] Guido van Rossum restates the objection to cyclic trash
        http://mail.python.org/pipermail/python-3000/2007-January/005322.html

   [13] Adam Olsen suggests using a weakref from stack frame to exception
        http://mail.python.org/pipermail/python-3000/2007-January/005363.html

   [14] Patch to implement the bulk of the PEP
        http://svn.python.org/view/python/branches/py3k/Include/?rev=57783&view=rev


Copyright

    This document has been placed in the public domain.


Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End:
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.