Process-Global State

C-level static variables. Since this is very low-level memory storage, it must be managed carefully.

Per-module State

State local to a module object, allocated dynamically as part of a module object’s initialization. This isolates the state from other instances of the module (including those in other subinterpreters).

Accessed byPyModule_GetState().

Static Type

A type object defined as a C-level static variable, i.e. a compiled-in type object.

A static type needs to be shared between module instances and has no information of what module it belongs to. Static types do not have__dict__(although their instances might).

Heap Type

A type object created at run time.

Defining Class

The defining class of a method (either bound or unbound) is the class on which the method was defined. A class that merely inherits the method from its base is not the defining class.

For example,intis the defining class ofTrue.to_bytes, True.__floor__andint.__repr__.

In C, the defining class is the one defined with the corresponding tp_methodsor “tp slots”[1]entry. For methods defined in Python, the defining class is saved in the __class__closure cell.

C-API

The “Python/C API” as described in Python documentation. CPython implements the C-API, but other implementations exist.

Slot methods

The above changes don’t cover slot methods, such astp_iterornb_add.

The problem with slot methods is that their C API is fixed, so we can’t simply add a new argument to pass in the defining class. Two possible solutions have been proposed to this problem:

• Look up the class through walking the MRO. This is potentially expensive, but will be usable if performance is not a problem (such as when raising a module-level exception).
• Storing a pointer to the defining class of each slot in a separate table, __typeslots__[2].This is technically feasible and fast, but quite invasive.

Modules affected by this concern also have the option of using thread-local stateorPEP 567 context variablesas a caching mechanism, or else defining their own reload-friendly lookup caching scheme.

Solving the issue generally is deferred to a future PEP.

Adding module references to heap types

A new factory method will be added to the C-API for creating modules:

PyObject*PyType_FromModuleAndSpec(PyObject*module,
PyType_Spec*spec,
PyObject*bases)

This acts the same asPyType_FromSpecWithBases,and additionally associates the provided module object with the new type. (In CPython, this will set ht_moduledescribed below.)

Additionally, an accessor,PyObject*PyType_GetModule(PyTypeObject*) will be provided. It will return the type’s associated module if one is set, otherwise it will setTypeErrorand return NULL. When given a static type, it will always setTypeErrorand return NULL.

To implement this in CPython, thePyHeapTypeObjectstruct will get a new member,PyObject*ht_module,that will store a pointer to the associated module. It will beNULLby default and should not be modified after the type object is created.

Theht_modulemember will not be inherited by subclasses; it needs to be set usingPyType_FromSpecWithBasesfor each individual type that needs it.

Usually, creating a class withht_moduleset will create a reference cycle involving the class and the module. This is not a problem, as tearing down modules is not a performance-sensitive operation, and module-level functions typically also create reference cycles. The existing “set all module globals to None” code that breaks function cycles throughf_globalswill also break the new cycles throughht_module.

Passing the defining class to extension methods

A new signature flag,METH_METHOD,will be added for use in PyMethodDef.ml_flags.Conceptually, it addsdefining_class to the function signature. To make the initial implementation easier, the flag can only be used as (METH_FASTCALL|METH_KEYWORDS|METH_METHOD). (It can’t be used with other flags likeMETH_Oor bareMETH_FASTCALL, though it may be combined withMETH_CLASSorMETH_STATIC).

C functions for methods defined using this flag combination will be called using a new C signature calledPyCMethod:

PyObject*PyCMethod(PyObject*self,
PyTypeObject*defining_class,
PyObject*const*args,
size_tnargsf,
PyObject*kwnames)

Additional combinations like(METH_VARARGS|METH_METHOD)may be added in the future (or even in the initial implementation of this PEP). However,METH_METHODshould always be anadditionalflag, i.e., the defining class should only be passed in if needed.

In CPython, a new structure extendingPyCFunctionObjectwill be added to hold the extra information:

typedefstruct{
PyCFunctionObjectfunc;
PyTypeObject*mm_class;/*Passedas'defining_class'argtotheCfunc*/
}PyCMethodObject;

ThePyCFunctionimplementation will passmm_classinto a PyCMethodC function when it finds theMETH_METHODflag being set. A new macroPyCFunction_GET_CLASS(cls)will be added for easier access tomm_class.

C methods may continue to use the otherMETH_*signatures if they do not require access to their defining class/module. IfMETH_METHODis not set, casting toPyCMethodObjectis invalid.

Argument Clinic

To support passing the defining class to methods using Argument Clinic, a new converter calleddefining_classwill be added to CPython’s Argument Clinic tool.

Each method may only have one argument using this converter, and it must appear afterself,or, ifselfis not used, as the first argument. The argument will be of typePyTypeObject*.

When used, Argument Clinic will select METH_FASTCALL|METH_KEYWORDS|METH_METHODas the calling convention. The argument will not appear in__text_signature__.

The new converter will initially not be compatible with__init__and __new__methods, which cannot use theMETH_METHODconvention.

Helpers

Getting toper-module statefrom a heap type is a very common task. To make this easier, a helper will be added:

void*PyType_GetModuleState(PyObject*type)

This function takes a heap type and on success, it returns pointer to the state of the module that the heap type belongs to.

On failure, two scenarios may occur. When a non-type object, or a type without a module is passed in,TypeErroris set andNULLreturned. If the module is found, the pointer to the state, which may beNULL,is returned without setting any exception.

Modules Converted in the Initial Implementation

To validate the approach, the_elementtreemodule will be modified during the initial implementation.

Slot methods

A way of passing defining class (or module state) to slot methods may be added in the future.

A previous version of this PEP proposed a helper function that would determine a defining class by searching the MRO for a class that defines a slot to a particular function. However, this approach would fail if a class is mutated (which is, for heap types, possible from Python code). Solving this problem is left to future discussions.

Easy creation of types with module references

It would be possible to add aPEP 489execution slot type to make creating heap types significantly easier than calling PyType_FromModuleAndSpec. This is left to a future PEP.

It may be good to add a good way to create static exception types from the limited API. Such exception types could be shared between subinterpreters, but instantiated without needing specific module state. This is also left to possible future discussions.

Optimization

As proposed here, methods defined with theMETH_METHODflag only support one specific signature.

If it turns out that other signatures are needed for performance reasons, they may be added.