Tracing

Tracing generates events for calls, returns, exceptions, lines of source code executed, and, under some circumstances, instructions executed.

Only line events are covered by this PEP.

When tracing is turned on, line events will be generated when:

• A new line of source code is reached.
• A backwards jump occurs, even if it jumps to the same line, as may happen in list comprehensions.

Additionally, line events willneverbe generated for source code lines that are not executed.

What is considered to be code for the purposes of tracing

All expressions and parts of expressions are considered to be executable code.

In general, all statements are also considered to be executable code. However, when a statement is spread over several lines, we must consider which parts of a statement are considered to be executable code.

Statements are made up of keywords and expressions. Not all keywords have a direct runtime effect, so not all keywords are considered to be executable code. For example,else,is a necessary part of anifstatement, but there is no runtime effect associated with anelse.

For the purposes of tracing, the following keywords willnotbe considered to be executable code:

• del– The expression to be deleted is treated as the executable code.
• else– No runtime effect
• finally– No runtime effect
• global– Purely declarative
• nonlocal– Purely declarative

All other keywords are considered to be executable code.

Example event sequences

In the following examples, events are listed as “name”,f_linenopairs.

The code

1.globalx
2.x=a

generates the following event:

"line"2

The code

1.try:
2.pass
3.finally:
4.pass

generates the following events:

"line"1
"line"2
"line"4

The code

1.for(
2.x)in[1]:
3.pass
4.return

generates the following events:

"line"2# evaluate [1]
"line"1# for
"line"2# store to x
"line"3# pass
"line"1# for
"line"4# return
"return"1

The f_lineno attribute

• When a frame object is created, thef_linenoattribute will be set to the line at which the function or class is defined; that is the line on which thedeforclasskeyword appears. For modules it will be set to zero.
• Thef_linenoattribute will be updated to match the line number about to be executed, even if tracing is turned off and no event is generated.

The new co_lines() method of code objects

Theco_lines()method will return an iterator which yields tuples of values, each representing the line number of a range of bytecodes. Each tuple will consist of three values:

• start– The offset (inclusive) of the start of the bytecode range
• end– The offset (exclusive) of the end of the bytecode range
• line– The line number, orNoneif the bytecodes in the given range do not have a line number.

The sequence generated will have the following properties:

• The first range in the sequence with have astartof0
• The(start,end)ranges will be non-decreasing and consecutive. That is, for any pair of tuples thestartof the second will equal to theendof the first.
• No range will be backwards, that isend>=startfor all triples.
• The final range in the sequence with haveendequal to the size of the bytecode.
• linewill either be a positive integer, orNone

The co_linetable attribute

The co_linetable attribute will hold the line number information. The format is opaque, unspecified and may be changed without notice. The attribute is public only to support creation of new code objects.

The co_lnotab attribute

Historically theco_lnotabattribute held a mapping from bytecode offset to line number, but does not support bytecodes without a line number. For backward compatibility, theco_lnotabbytes object will be lazily created when needed. For ranges of bytecodes without a line number, the line number of the previous bytecode range will be used.

Tools that parse theco_lnotabtable should move to using the newco_lines()method as soon as is practical.

Examples of code for which the sequence of trace events will change

In the following examples, events are listed as “name”,f_linenopairs.

0.defspam(a):
1.ifa:
2.eggs()
3.else:
4.pass

"line"1
"line"2
"line"4
"return"4

"line"1
"line"2
"return"2

0.defbar():
1.pass
2.pass
3.pass

"line"3
"return"3

"line"1
"line"2
"line"3
"return"3

C API

Access to thef_linenoattribute of frame objects through C API functions is unchanged. f_linenocan be read byPyFrame_GetLineNumber.f_linenocan only be set viaPyObject_SetAttrand similar functions.

Accessingf_linenodirectly through the underlying data structure is forbidden.

Out of process debuggers and profilers

Out of process tools, such as py-spy[1],cannot use the C-API, and must parse the line number table themselves. Although the line number table format may change without warning, it will not change during a release unless absolutely necessary for a bug fix.

To reduce the work required to implement these tools, the following C struct and utility functions are provided. Note that these functions are not part of the C-API, so will be need to be linked into any code that needs to use them.

typedefstructaddressrange{
intar_start;
intar_end;
intar_line;
struct_opaqueopaque;
}PyCodeAddressRange;

voidPyLineTable_InitAddressRange(char*linetable,Py_ssize_tlength,intfirstlineno,PyCodeAddressRange*range);
intPyLineTable_NextAddressRange(PyCodeAddressRange*range);
intPyLineTable_PreviousAddressRange(PyCodeAddressRange*range);

PyLineTable_InitAddressRangeinitializes thePyCodeAddressRangestruct from the line number table and first line number.

PyLineTable_NextAddressRangeadvances the range to the next entry, returning non-zero if valid.

PyLineTable_PreviousAddressRangeretreats the range to the previous entry, returning non-zero if valid.

Although these functions are not part of C-API, they will provided by all future versions of CPython. ThePyLineTable_functions do not call into the C-API, so can be safely copied into any tool that needs to use them. ThePyCodeAddressRangestruct will not be changed, but the_opaquestruct is not part of the specification and may change.

For example, the following code prints out all the address ranges:

voidprint_address_ranges(char*linetable,Py_ssize_tlength,intfirstlineno)
{
PyCodeAddressRangerange;
PyLineTable_InitAddressRange(linetable,length,firstlineno,&range);
while(PyLineTable_NextAddressRange(&range)){
printf("Bytecodes from%d(inclusive) to%d(exclusive) ",
range.start,range.end);
if(range.line<0){
/*line<0meansnolinenumber*/
printf("have no line number\n");
}
else{
printf("have line number%d\n",range.line);
}
}
}