Numbering Note

This PEP started its life asPEP 344.Since it is now targeted for Python 3000, it has been moved into the 3xxx space.

Abstract

This PEP proposes three standard attributes on exception instances: the __context__attribute for implicitly chained exceptions, the__cause__ attribute for explicitly chained exceptions, and the__traceback__ attribute for the traceback. A newraise...fromstatement sets the __cause__attribute.

Motivation

During the handling of one exception (exception A), it is possible that another exception (exception B) may occur. In today’s Python (version 2.4), if this happens, exception B is propagated outward and exception A is lost. In order to debug the problem, it is useful to know about both exceptions. The __context__attribute retains this information automatically.

Sometimes it can be useful for an exception handler to intentionally re-raise an exception, either to provide extra information or to translate an exception to another type. The__cause__attribute provides an explicit way to record the direct cause of an exception.

In today’s Python implementation, exceptions are composed of three parts: the type, the value, and the traceback. Thesysmodule, exposes the current exception in three parallel variables,exc_type,exc_value,and exc_traceback,thesys.exc_info()function returns a tuple of these three parts, and theraisestatement has a three-argument form accepting these three parts. Manipulating exceptions often requires passing these three things in parallel, which can be tedious and error-prone. Additionally, the exceptstatement can only provide access to the value, not the traceback. Adding the__traceback__attribute to exception values makes all the exception information accessible from a single place.

History

Raymond Hettinger[1]raised the issue of masked exceptions on Python-Dev in January 2003 and proposed aPyErr_FormatAppend()function that C modules could use to augment the currently active exception with more information. Brett Cannon[2]brought up chained exceptions again in June 2003, prompting a long discussion.

Greg Ewing[3]identified the case of an exception occurring in afinally block during unwinding triggered by an original exception, as distinct from the case of an exception occurring in anexceptblock that is handling the original exception.

Greg Ewing[4]and Guido van Rossum[5],and probably others, have previously mentioned adding a traceback attribute to Exception instances. This is noted inPEP 3000.

This PEP was motivated by yet another recent Python-Dev reposting of the same ideas[6][7].

Rationale

The Python-Dev discussions revealed interest in exception chaining for two quite different purposes. To handle the unexpected raising of a secondary exception, the exception must be retained implicitly. To support intentional translation of an exception, there must be a way to chain exceptions explicitly. This PEP addresses both.

Several attribute names for chained exceptions have been suggested on Python-Dev[2],includingcause,antecedent,reason,original, chain,chainedexc,exc_chain,excprev,previous,and precursor.For an explicitly chained exception, this PEP suggests __cause__because of its specific meaning. For an implicitly chained exception, this PEP proposes the name__context__because the intended meaning is more specific than temporal precedence but less specific than causation: an exception occurs in the context of handling another exception.

This PEP suggests names with leading and trailing double-underscores for these three attributes because they are set by the Python VM. Only in very special cases should they be set by normal assignment.

This PEP handles exceptions that occur duringexceptblocks andfinally blocks in the same way. Reading the traceback makes it clear where the exceptions occurred, so additional mechanisms for distinguishing the two cases would only add unnecessary complexity.

This PEP proposes that the outermost exception object (the one exposed for matching byexceptclauses) be the most recently raised exception for compatibility with current behaviour.

This PEP proposes that tracebacks display the outermost exception last, because this would be consistent with the chronological order of tracebacks (from oldest to most recent frame) and because the actual thrown exception is easier to find on the last line.

To keep things simpler, the C API calls for setting an exception will not automatically set the exception’s__context__.Guido van Rossum has expressed concerns with making such changes[8].

As for other languages, Java and Ruby both discard the original exception when another exception occurs in acatch/rescueorfinally/ensure clause. Perl 5 lacks built-in structured exception handling. For Perl 6, RFC number 88[9]proposes an exception mechanism that implicitly retains chained exceptions in an array named@@.In that RFC, the most recently raised exception is exposed for matching, as in this PEP; also, arbitrary expressions (possibly involving@@) can be evaluated for exception matching.

Exceptions in C# contain a read-onlyInnerExceptionproperty that may point to another exception. Its documentation[10]says that “When an exception X is thrown as a direct result of a previous exception Y, theInnerException property of X should contain a reference to Y.” This property is not set by the VM automatically; rather, all exception constructors take an optional innerExceptionargument to set it explicitly. The__cause__attribute fulfills the same purpose asInnerException,but this PEP proposes a new form ofraiserather than extending the constructors of all exceptions. C# also provides aGetBaseExceptionmethod that jumps directly to the end of theInnerExceptionchain; this PEP proposes no analog.

The reason all three of these attributes are presented together in one proposal is that the__traceback__attribute provides convenient access to the traceback on chained exceptions.

Implicit Exception Chaining

Here is an example to illustrate the__context__attribute:

defcompute(a,b):
try:
a/b
exceptException,exc:
log(exc)

deflog(exc):
file=open('logfile.txt')# oops, forgot the 'w'
print>>file,exc
file.close()

Callingcompute(0,0)causes aZeroDivisionError.Thecompute() function catches this exception and callslog(exc),but thelog() function also raises an exception when it tries to write to a file that wasn’t opened for writing.

In today’s Python, the caller ofcompute()gets thrown anIOError.The ZeroDivisionErroris lost. With the proposed change, the instance of IOErrorhas an additional__context__attribute that retains the ZeroDivisionError.

The following more elaborate example demonstrates the handling of a mixture of finallyandexceptclauses:

defmain(filename):
file=open(filename)# oops, forgot the 'w'
try:
try:
compute()
exceptException,exc:
log(file,exc)
finally:
file.clos()# oops, misspelled 'close'

defcompute():
1/0

deflog(file,exc):
try:
print>>file,exc# oops, file is not writable
except:
display(exc)

defdisplay(exc):
printex# oops, misspelled 'exc'

Callingmain()with the name of an existing file will trigger four exceptions. The ultimate result will be anAttributeErrordue to the misspelling ofclos,whose__context__points to aNameErrordue to the misspelling ofex,whose__context__points to anIOError due to the file being read-only, whose__context__points to a ZeroDivisionError,whose__context__attribute isNone.

The proposed semantics are as follows:

1. Each thread has an exception context initially set toNone.
2. Whenever an exception is raised, if the exception instance does not already have a__context__attribute, the interpreter sets it equal to the thread’s exception context.
3. Immediately after an exception is raised, the thread’s exception context is set to the exception.
4. Whenever the interpreter exits anexceptblock by reaching the end or executing areturn,yield,continue,orbreakstatement, the thread’s exception context is set toNone.

Explicit Exception Chaining

The__cause__attribute on exception objects is always initialized to None.It is set by a new form of theraisestatement:

raiseEXCEPTIONfromCAUSE

which is equivalent to:

exc=EXCEPTION
exc.__cause__=CAUSE
raiseexc

In the following example, a database provides implementations for a few different kinds of storage, with file storage as one kind. The database designer wants errors to propagate asDatabaseErrorobjects so that the client doesn’t have to be aware of the storage-specific details, but doesn’t want to lose the underlying error information.

classDatabaseError(Exception):
pass

classFileDatabase(Database):
def__init__(self,filename):
try:
self.file=open(filename)
exceptIOError,exc:
raiseDatabaseError('failed to open')fromexc

If the call toopen()raises an exception, the problem will be reported as aDatabaseError,with a__cause__attribute that reveals the IOErroras the original cause.

Traceback Attribute

The following example illustrates the__traceback__attribute.

defdo_logged(file,work):
try:
work()
exceptException,exc:
write_exception(file,exc)
raiseexc

fromtracebackimportformat_tb

defwrite_exception(file,exc):
...
type=exc.__class__
message=str(exc)
lines=format_tb(exc.__traceback__)
file.write(...type...message...lines...)
...

In today’s Python, thedo_logged()function would have to extract the traceback fromsys.exc_tracebackorsys.exc_info()[2]and pass both the value and the traceback towrite_exception().With the proposed change,write_exception()simply gets one argument and obtains the exception using the__traceback__attribute.

The proposed semantics are as follows:

1. Whenever an exception is caught, if the exception instance does not already have a__traceback__attribute, the interpreter sets it to the newly caught traceback.

Enhanced Reporting

The default exception handler will be modified to report chained exceptions. The chain of exceptions is traversed by following the__cause__and __context__attributes, with__cause__taking priority. In keeping with the chronological order of tracebacks, the most recently raised exception is displayed last; that is, the display begins with the description of the innermost exception and backs up the chain to the outermost exception. The tracebacks are formatted as usual, with one of the lines:

Theaboveexceptionwasthedirectcauseofthefollowingexception:

or

Duringhandlingoftheaboveexception,anotherexceptionoccurred:

between tracebacks, depending whether they are linked by__cause__or __context__respectively. Here is a sketch of the procedure:

defprint_chain(exc):
ifexc.__cause__:
print_chain(exc.__cause__)
print'\nThe above exception was the direct cause...'
elifexc.__context__:
print_chain(exc.__context__)
print'\nDuring handling of the above exception,...'
print_exc(exc)

In thetracebackmodule, theformat_exception,print_exception, print_exc,andprint_lastfunctions will be updated to accept an optionalchainargument,Trueby default. When this argument is True,these functions will format or display the entire chain of exceptions as just described. When it isFalse,these functions will format or display only the outermost exception.

Thecgitbmodule should also be updated to display the entire chain of exceptions.

C API

ThePyErr_Set*calls for setting exceptions will not set the __context__attribute on exceptions.PyErr_NormalizeExceptionwill always set thetracebackattribute to itstbargument and the __context__and__cause__attributes toNone.

A new API function,PyErr_SetContext(context),will help C programmers provide chained exception information. This function will first normalize the current exception so it is an instance, then set its__context__attribute. A similar API function,PyErr_SetCause(cause),will set the__cause__ attribute.

Compatibility

Chained exceptions expose the type of the most recent exception, so they will still match the sameexceptclauses as they do now.

The proposed changes should not break any code unless it sets or uses attributes named__context__,__cause__,or__traceback__on exception instances. As of 2005-05-12, the Python standard library contains no mention of such attributes.

Open Issue: Extra Information

Walter Dörwald[11]expressed a desire to attach extra information to an exception during its upward propagation without changing its type. This could be a useful feature, but it is not addressed by this PEP. It could conceivably be addressed by a separate PEP establishing conventions for other informational attributes on exceptions.

Open Issue: Suppressing Context

As written, this PEP makes it impossible to suppress__context__,since settingexc.__context__toNonein anexceptorfinallyclause will only result in it being set again whenexcis raised.

Open Issue: Limiting Exception Types

To improve encapsulation, library implementors may want to wrap all implementation-level exceptions with an application-level exception. One could try to wrap exceptions by writing this:

try:
...implementationmayraiseanexception...
except:
importsys
raiseApplicationErrorfromsys.exc_value

or this:

try:
...implementationmayraiseanexception...
exceptException,exc:
raiseApplicationErrorfromexc

but both are somewhat flawed. It would be nice to be able to name the current exception in a catch-allexceptclause, but that isn’t addressed here. Such a feature would allow something like this:

try:
...implementationmayraiseanexception...
except*,exc:
raiseApplicationErrorfromexc

Open Issue: yield

The exception context is lost when ayieldstatement is executed; resuming the frame after theyielddoes not restore the context. Addressing this problem is out of the scope of this PEP; it is not a new problem, as demonstrated by the following example:

>>>defgen():
...try:
...1/0
...except:
...yield3
...raise
...
>>>g=gen()
>>>g.next()
3
>>>g.next()
TypeError:exceptionsmustbeclasses,instances,orstrings
(deprecated),notNoneType

Open Issue: Garbage Collection

The strongest objection to this proposal has been that it creates cycles between exceptions and stack frames[12].Collection of cyclic garbage (and therefore resource release) can be greatly delayed.

>>>try:
>>>1/0
>>>exceptException,err:
>>>pass

will introduce a cycle from err -> traceback -> stack frame -> err, keeping all locals in the same scope alive until the next GC happens.

Today, these locals would go out of scope. There is lots of code which assumes that “local” resources – particularly open files – will be closed quickly. If closure has to wait for the next GC, a program (which runs fine today) may run out of file handles.

Making the__traceback__attribute a weak reference would avoid the problems with cyclic garbage. Unfortunately, it would make saving the Exceptionfor later (asunittestdoes) more awkward, and it would not allow as much cleanup of thesysmodule.

A possible alternate solution, suggested by Adam Olsen, would be to instead turn the reference from the stack frame to theerrvariable into a weak reference when the variable goes out of scope[13].

Possible Future Compatible Changes

These changes are consistent with the appearance of exceptions as a single object rather than a triple at the interpreter level.

• IfPEP 340orPEP 343is accepted, replace the three (type,value, traceback) arguments to__exit__with a single exception argument.
• Deprecatesys.exc_type,sys.exc_value,sys.exc_traceback,and sys.exc_info()in favour of a single member,sys.exception.
• Deprecatesys.last_type,sys.last_value,andsys.last_traceback in favour of a single member,sys.last_exception.
• Deprecate the three-argument form of theraisestatement in favour of the one-argument form.
• Upgradecgitb.html()to accept a single value as its first argument as an alternative to a(type,value,traceback)tuple.

Possible Future Incompatible Changes

These changes might be worth considering for Python 3000.

• Removesys.exc_type,sys.exc_value,sys.exc_traceback,and sys.exc_info().
• Removesys.last_type,sys.last_value,andsys.last_traceback.
• Replace the three-argumentsys.excepthookwith a one-argument API, and changing thecgitbmodule to match.
• Remove the three-argument form of theraisestatement.
• Upgradetraceback.print_exceptionto accept anexceptionargument instead of thetype,value,andtracebackarguments.

Implementation

The__traceback__and__cause__attributes and the new raise syntax were implemented in revision 57783[14].

Acknowledgements

Brett Cannon, Greg Ewing, Guido van Rossum, Jeremy Hylton, Phillip J. Eby, Raymond Hettinger, Walter Dörwald, and others.

References

Copyright

This document has been placed in the public domain.