Supporting Cyclic Garbage Collection

Python’s support for detecting and collecting garbage which involves circular references requires support from object types which are “containers” for other objects which may also be containers. Types which do not store references to other objects, or which only store references to atomic types (such as numbers or strings), do not need to provide any explicit support for garbage collection.

To create a container type, thetp_flagsfield of the type object must include thePy_TPFLAGS_HAVE_GCand provide an implementation of the tp_traversehandler. If instances of the type are mutable, a tp_clearimplementation must also be provided.

Py_TPFLAGS_HAVE_GC

Objects with a type with this flag set must conform with the rules documented here. For convenience these objects will be referred to as container objects.

Constructors for container types must conform to two rules:

1. The memory for the object must be allocated usingPyObject_GC_New orPyObject_GC_NewVar.

2. Once all the fields which may contain references to other containers are initialized, it must callPyObject_GC_Track().

Similarly, the deallocator for the object must conform to a similar pair of rules:

1. Before fields which refer to other containers are invalidated, PyObject_GC_UnTrack()must be called.

2. The object’s memory must be deallocated usingPyObject_GC_Del().

   Warning

   If a type adds the Py_TPFLAGS_HAVE_GC, then itmustimplement at least atp_traversehandler or explicitly use one from its subclass or subclasses.

   When callingPyType_Ready()or some of the APIs that indirectly call it likePyType_FromSpecWithBases()or PyType_FromSpec()the interpreter will automatically populate the tp_flags,tp_traverse andtp_clearfields if the type inherits from a class that implements the garbage collector protocol and the child class doesnotinclude thePy_TPFLAGS_HAVE_GCflag.

PyObject_GC_New(TYPE,typeobj)

Analogous toPyObject_Newbut for container objects with the Py_TPFLAGS_HAVE_GCflag set.

PyObject_GC_NewVar(TYPE,typeobj,size)

Analogous toPyObject_NewVarbut for container objects with the Py_TPFLAGS_HAVE_GCflag set.

PyObject*PyUnstable_Object_GC_NewWithExtraData(PyTypeObject*type,size_textra_size)

This isUnstable API.It may change without warning in minor releases.

Analogous toPyObject_GC_Newbut allocatesextra_size bytes at the end of the object (at offset tp_basicsize). The allocated memory is initialized to zeros, except for thePythonobjectheader.

The extra data will be deallocated with the object, but otherwise it is not managed by Python.

Warning

The function is marked as unstable because the final mechanism for reserving extra data after an instance is not yet decided. For allocating a variable number of fields, prefer using PyVarObjectandtp_itemsize instead.

Added in version 3.12.

PyObject_GC_Resize(TYPE,op,newsize)

Resize an object allocated byPyObject_NewVar. Returns the resized object of typeTYPE*(refers to any C type) orNULLon failure.

opmust be of typePyVarObject* and must not be tracked by the collector yet. newsizemust be of typePy_ssize_t.

voidPyObject_GC_Track(PyObject*op)

Part of theStable ABI.

Adds the objectopto the set of container objects tracked by the collector. The collector can run at unexpected times so objects must be valid while being tracked. This should be called once all the fields followed by thetp_traversehandler become valid, usually near the end of the constructor.

intPyObject_IS_GC(PyObject*obj)

Returns non-zero if the object implements the garbage collector protocol, otherwise returns 0.

The object cannot be tracked by the garbage collector if this function returns 0.

intPyObject_GC_IsTracked(PyObject*op)

Part of theStable ABIsince version 3.9.

Returns 1 if the object type ofopimplements the GC protocol andopis being currently tracked by the garbage collector and 0 otherwise.

This is analogous to the Python functiongc.is_tracked().

Added in version 3.9.

intPyObject_GC_IsFinalized(PyObject*op)

Part of theStable ABIsince version 3.9.

Returns 1 if the object type ofopimplements the GC protocol andophas been already finalized by the garbage collector and 0 otherwise.

This is analogous to the Python functiongc.is_finalized().

Added in version 3.9.

voidPyObject_GC_Del(void*op)

Part of theStable ABI.

Releases memory allocated to an object usingPyObject_GC_Newor PyObject_GC_NewVar.

voidPyObject_GC_UnTrack(void*op)

Part of theStable ABI.

Remove the objectopfrom the set of container objects tracked by the collector. Note thatPyObject_GC_Track()can be called again on this object to add it back to the set of tracked objects. The deallocator (tp_deallochandler) should call this for the object before any of the fields used by thetp_traversehandler become invalid.

Changed in version 3.8:The_PyObject_GC_TRACK()and_PyObject_GC_UNTRACK()macros have been removed from the public C API.

Thetp_traversehandler accepts a function parameter of this type:

typedefint(*visitproc)(PyObject*object,void*arg)

Part of theStable ABI.

Type of the visitor function passed to thetp_traversehandler. The function should be called with an object to traverse asobjectand the third parameter to thetp_traversehandler asarg.The Python core uses several visitor functions to implement cyclic garbage detection; it’s not expected that users will need to write their own visitor functions.

Thetp_traversehandler must have the following type:

typedefint(*traverseproc)(PyObject*self,visitprocvisit,void*arg)

Part of theStable ABI.

Traversal function for a container object. Implementations must call the visitfunction for each object directly contained byself,with the parameters tovisitbeing the contained object and theargvalue passed to the handler. Thevisitfunction must not be called with aNULL object argument. Ifvisitreturns a non-zero value that value should be returned immediately.

To simplify writingtp_traversehandlers, aPy_VISIT()macro is provided. In order to use this macro, thetp_traverseimplementation must name its arguments exactlyvisitandarg:

voidPy_VISIT(PyObject*o)

Ifois notNULL,call thevisitcallback, with argumentso andarg.Ifvisitreturns a non-zero value, then return it. Using this macro,tp_traversehandlers look like:

staticint
my_traverse(Noddy*self,visitprocvisit,void*arg)
{
Py_VISIT(self->foo);
Py_VISIT(self->bar);
return0;
}

Thetp_clearhandler must be of theinquirytype, orNULL if the object is immutable.

typedefint(*inquiry)(PyObject*self)

Part of theStable ABI.

Drop references that may have created reference cycles. Immutable objects do not have to define this method since they can never directly create reference cycles. Note that the object must still be valid after calling this method (don’t just callPy_DECREF()on a reference). The collector will call this method if it detects that this object is involved in a reference cycle.

Controlling the Garbage Collector State

The C-API provides the following functions for controlling garbage collection runs.

Py_ssize_tPyGC_Collect(void)

Part of theStable ABI.

Perform a full garbage collection, if the garbage collector is enabled. (Note thatgc.collect()runs it unconditionally.)

Returns the number of collected + unreachable objects which cannot be collected. If the garbage collector is disabled or already collecting, returns0immediately. Errors during garbage collection are passed tosys.unraisablehook. This function does not raise exceptions.

intPyGC_Enable(void)

Part of theStable ABIsince version 3.10.

Enable the garbage collector: similar togc.enable(). Returns the previous state, 0 for disabled and 1 for enabled.

Added in version 3.10.

intPyGC_Disable(void)

Part of theStable ABIsince version 3.10.

Disable the garbage collector: similar togc.disable(). Returns the previous state, 0 for disabled and 1 for enabled.

Added in version 3.10.

intPyGC_IsEnabled(void)

Part of theStable ABIsince version 3.10.

Query the state of the garbage collector: similar togc.isenabled(). Returns the current state, 0 for disabled and 1 for enabled.

Added in version 3.10.

Querying Garbage Collector State

The C-API provides the following interface for querying information about the garbage collector.

voidPyUnstable_GC_VisitObjects(gcvisitobjects_tcallback,void*arg)

This isUnstable API.It may change without warning in minor releases.

Run suppliedcallbackon all live GC-capable objects.argis passed through to all invocations ofcallback.

Warning

If new objects are (de)allocated by the callback it is undefined if they will be visited.

Garbage collection is disabled during operation. Explicitly running a collection in the callback may lead to undefined behaviour e.g. visiting the same objects multiple times or not at all.

Added in version 3.12.

typedefint(*gcvisitobjects_t)(PyObject*object,void*arg)

Type of the visitor function to be passed toPyUnstable_GC_VisitObjects(). argis the same as theargpassed toPyUnstable_GC_VisitObjects. Return0to continue iteration, return1to stop iteration. Other return values are reserved for now so behavior on returning anything else is undefined.

Added in version 3.12.