Following system colour scheme Selected dark colour scheme Selected light colour scheme

Python Enhancement Proposals

PEP 673 – Self Type

Author:
Pradeep Kumar Srinivasan <gohanpra at gmail.com>, James Hilton-Balfe <gobot1234yt at gmail.com>
Sponsor:
Jelle Zijlstra <jelle.zijlstra at gmail.com>
Discussions-To:
Typing-SIG list
Status:
Final
Type:
Standards Track
Topic:
Typing
Created:
10-Nov-2021
Python-Version:
3.11
Post-History:
17-Nov-2021
Resolution:
Python-Dev thread
Table of Contents

Attention

This PEP is a historical document: seeSelfand typing.Selffor up-to-date specs and documentation. Canonical typing specs are maintained at thetyping specs site;runtime typing behaviour is described in the CPython documentation.

×

See thetyping specification update processfor how to propose changes to the typing spec.

Abstract

This PEP introduces a simple and intuitive way to annotate methods that return an instance of their class. This behaves the same as theTypeVar-based approach specified inPEP 484 but is more concise and easier to follow.

Motivation

A common use case is to write a method that returns an instance of the same class, usually by returningself.

classShape:
defset_scale(self,scale:float):
self.scale=scale
returnself

Shape().set_scale(0.5)# => should be Shape

One way to denote the return type is to specify it as the current class, say, Shape.Using the method makes the type checker infer the typeShape, as expected.

classShape:
defset_scale(self,scale:float)->Shape:
self.scale=scale
returnself

Shape().set_scale(0.5)# => Shape

However, when we callset_scaleon a subclass ofShape,the type checker still infers the return type to beShape.This is problematic in situations such as the one shown below, where the type checker will return an error because we are trying to use attributes or methods not present on the base class.

classCircle(Shape):
defset_radius(self,r:float)->Circle:
self.radius=r
returnself

Circle().set_scale(0.5)# *Shape*, not Circle
Circle().set_scale(0.5).set_radius(2.7)
# => Error: Shape has no attribute set_radius

The present workaround for such instances is to define aTypeVarwith the base class as the bound and use it as the annotation for theself parameter and the return type:

fromtypingimportTypeVar

TShape=TypeVar("TShape",bound="Shape")

classShape:
defset_scale(self:TShape,scale:float)->TShape:
self.scale=scale
returnself


classCircle(Shape):
defset_radius(self,radius:float)->Circle:
self.radius=radius
returnself

Circle().set_scale(0.5).set_radius(2.7)# => Circle

Unfortunately, this is verbose and unintuitive. Becauseselfis usually not explicitly annotated, the above solution doesn’t immediately come to mind, and even if it does, it is very easy to go wrong by forgetting either the bound on theTypeVar(bound= "Shape" )or the annotation forself.

This difficulty means that users often give up and either use fallback types likeAnyor just omit the type annotation completely, both of which make the code less safe.

We propose a more intuitive and succinct way of expressing the above intention. We introduce a special formSelfthat stands for a type variable bound to the encapsulating class. For situations such as the one above, the user simply has to annotate the return type asSelf:

fromtypingimportSelf

classShape:
defset_scale(self,scale:float)->Self:
self.scale=scale
returnself


classCircle(Shape):
defset_radius(self,radius:float)->Self:
self.radius=radius
returnself

By annotating the return type asSelf,we no longer have to declare a TypeVarwith an explicit bound on the base class. The return typeSelf mirrors the fact that the function returnsselfand is easier to understand.

As in the above example, the type checker will correctly infer the type of Circle().set_scale(0.5)to beCircle,as expected.

Usage statistics

Weanalyzedpopular open-source projects and found that patterns like the above were used about 40%as often as popular types likedictorCallable.For example, in typeshed alone, such “Self” types are used 523 times, compared to 1286 uses ofdictand 1314 uses ofCallableas of October 2021. This suggests that aSelftype will be used quite often and users will benefit a lot from the simpler approach above.

Users of Python types have also frequently requested this feature, both on theproposal doc and onGitHub.

Specification

Use in Method Signatures

Selfused in the signature of a method is treated as if it were a TypeVarbound to the class.

fromtypingimportSelf

classShape:
defset_scale(self,scale:float)->Self:
self.scale=scale
returnself

is treated equivalently to:

fromtypingimportTypeVar

SelfShape=TypeVar("SelfShape",bound="Shape")

classShape:
defset_scale(self:SelfShape,scale:float)->SelfShape:
self.scale=scale
returnself

This works the same for a subclass too:

classCircle(Shape):
defset_radius(self,radius:float)->Self:
self.radius=radius
returnself

which is treated equivalently to:

SelfCircle=TypeVar("SelfCircle",bound="Circle")

classCircle(Shape):
defset_radius(self:SelfCircle,radius:float)->SelfCircle:
self.radius=radius
returnself

One implementation strategy is to simply desugar the former to the latter in a preprocessing step. If a method usesSelfin its signature, the type of selfwithin a method will beSelf.In other cases, the type of selfwill remain the enclosing class.

Use in Classmethod Signatures

TheSelftype annotation is also useful for classmethods that return an instance of the class that they operate on. For example,from_configin the following snippet builds aShapeobject from a givenconfig.

classShape:
def__init__(self,scale:float)->None:...

@classmethod
deffrom_config(cls,config:dict[str,float])->Shape:
returncls(config["scale"])

However, this means thatCircle.from_config(...)is inferred to return a value of typeShape,when in fact it should beCircle:

classCircle(Shape):
defcircumference(self)->float:...

shape=Shape.from_config({"scale":7.0})
# => Shape

circle=Circle.from_config({"scale":7.0})
# => *Shape*, not Circle

circle.circumference()
# Error: `Shape` has no attribute `circumference`

The current workaround for this is unintuitive and error-prone:

Self=TypeVar("Self",bound="Shape")

classShape:
@classmethod
deffrom_config(
cls:type[Self],config:dict[str,float]
)->Self:
returncls(config["scale"])

We propose usingSelfdirectly:

fromtypingimportSelf

classShape:
@classmethod
deffrom_config(cls,config:dict[str,float])->Self:
returncls(config["scale"])

This avoids the complicatedcls:type[Self]annotation and theTypeVar declaration with abound.Once again, the latter code behaves equivalently to the former code.

Use in Parameter Types

Another use forSelfis to annotate parameters that expect instances of the current class:

Self=TypeVar("Self",bound="Shape")

classShape:
defdifference(self:Self,other:Self)->float:...

defapply(self:Self,f:Callable[[Self],None])->None:...

We propose usingSelfdirectly to achieve the same behavior:

fromtypingimportSelf

classShape:
defdifference(self,other:Self)->float:...

defapply(self,f:Callable[[Self],None])->None:...

Note that specifyingself:Selfis harmless, so some users may find it more readable to write the above as:

classShape:
defdifference(self:Self,other:Self)->float:...

Use in Attribute Annotations

Another use forSelfis to annotate attributes. One example is where we have aLinkedListwhose elements must be subclasses of the current class.

fromdataclassesimportdataclass
fromtypingimportGeneric,TypeVar

T=TypeVar("T")

@dataclass
classLinkedList(Generic[T]):
value:T
next:LinkedList[T]|None=None

# OK
LinkedList[int](value=1,next=LinkedList[int](value=2))
# Not OK
LinkedList[int](value=1,next=LinkedList[str](value="hello"))

However, annotating thenextattribute asLinkedList[T]allows invalid constructions with subclasses:

@dataclass
classOrdinalLinkedList(LinkedList[int]):
defordinal_value(self)->str:
returnas_ordinal(self.value)

# Should not be OK because LinkedList[int] is not a subclass of
# OrdinalLinkedList, # but the type checker allows it.
xs=OrdinalLinkedList(value=1,next=LinkedList[int](value=2))

ifxs.next:
print(xs.next.ordinal_value())# Runtime Error.

We propose expressing this constraint usingnext:Self|None:

fromtypingimportSelf

@dataclass
classLinkedList(Generic[T]):
value:T
next:Self|None=None

@dataclass
classOrdinalLinkedList(LinkedList[int]):
defordinal_value(self)->str:
returnas_ordinal(self.value)

xs=OrdinalLinkedList(value=1,next=LinkedList[int](value=2))
# Type error: Expected OrdinalLinkedList, got LinkedList[int].

ifxs.nextisnotNone:
xs.next=OrdinalLinkedList(value=3,next=None)# OK
xs.next=LinkedList[int](value=3,next=None)# Not OK

The code above is semantically equivalent to treating each attribute containing aSelftype as apropertythat returns that type:

fromdataclassesimportdataclass
fromtypingimportAny,Generic,TypeVar

T=TypeVar("T")
Self=TypeVar("Self",bound="LinkedList")


classLinkedList(Generic[T]):
value:T

@property
defnext(self:Self)->Self|None:
returnself._next

@next.setter
defnext(self:Self,next:Self|None)->None:
self._next=next

classOrdinalLinkedList(LinkedList[int]):
defordinal_value(self)->str:
returnstr(self.value)

Use in Generic Classes

Selfcan also be used in generic class methods:

classContainer(Generic[T]):
value:T
defset_value(self,value:T)->Self:...

This is equivalent to writing:

Self=TypeVar("Self",bound="Container[Any]")

classContainer(Generic[T]):
value:T
defset_value(self:Self,value:T)->Self:...

The behavior is to preserve the type argument of the object on which the method was called. When called on an object with concrete type Container[int],Selfis bound toContainer[int].When called with an object of generic typeContainer[T],Selfis bound to Container[T]:

defobject_with_concrete_type()->None:
int_container:Container[int]
str_container:Container[str]
reveal_type(int_container.set_value(42))# => Container[int]
reveal_type(str_container.set_value("hello"))# => Container[str]

defobject_with_generic_type(
container:Container[T],value:T,
)->Container[T]:
returncontainer.set_value(value)# => Container[T]

The PEP doesn’t specify the exact type ofself.valuewithin the method set_value.Some type checkers may choose to implementSelftypes using class-local type variables withSelf=TypeVar( “Self”, bound=Container[T]),which will infer a precise typeT.However, given that class-local type variables are not a standardized type system feature, it is also acceptable to inferAnyforself.value.We leave this up to the type checker.

Note that we reject usingSelfwith type arguments, such asSelf[int]. This is because it creates ambiguity about the type of theselfparameter and introduces unnecessary complexity:

classContainer(Generic[T]):
deffoo(
self,other:Self[int],other2:Self,
)->Self[str]:# Rejected
...

In such cases, we recommend using an explicit type forself:

classContainer(Generic[T]):
deffoo(
self:Container[T],
other:Container[int],
other2:Container[T]
)->Container[str]:...

Use in Protocols

Selfis valid within Protocols, similar to its use in classes:

fromtypingimportProtocol,Self

classShapeProtocol(Protocol):
scale:float

defset_scale(self,scale:float)->Self:
self.scale=scale
returnself

is treated equivalently to:

fromtypingimportTypeVar

SelfShape=TypeVar("SelfShape",bound="ShapeProtocol")

classShapeProtocol(Protocol):
scale:float

defset_scale(self:SelfShape,scale:float)->SelfShape:
self.scale=scale
returnself

SeePEP 544for details on the behavior of TypeVars bound to protocols.

Checking a class for compatibility with a protocol: If a protocol uses Selfin methods or attribute annotations, then a classFoois considered compatible with the protocol if its corresponding methods and attribute annotations use eitherSelforFooor any ofFoo’s subclasses. See the examples below:

fromtypingimportProtocol

classShapeProtocol(Protocol):
defset_scale(self,scale:float)->Self:...

classReturnSelf:
scale:float=1.0

defset_scale(self,scale:float)->Self:
self.scale=scale
returnself

classReturnConcreteShape:
scale:float=1.0

defset_scale(self,scale:float)->ReturnConcreteShape:
self.scale=scale
returnself

classBadReturnType:
scale:float=1.0

defset_scale(self,scale:float)->int:
self.scale=scale
return42

classReturnDifferentClass:
scale:float=1.0

defset_scale(self,scale:float)->ReturnConcreteShape:
returnReturnConcreteShape(...)

defaccepts_shape(shape:ShapeProtocol)->None:
y=shape.set_scale(0.5)
reveal_type(y)

defmain()->None:
return_self_shape:ReturnSelf
return_concrete_shape:ReturnConcreteShape
bad_return_type:BadReturnType
return_different_class:ReturnDifferentClass

accepts_shape(return_self_shape)# OK
accepts_shape(return_concrete_shape)# OK
accepts_shape(bad_return_type)# Not OK
# Not OK because it returns a non-subclass.
accepts_shape(return_different_class)

Valid Locations forSelf

ASelfannotation is only valid in class contexts, and will always refer to the encapsulating class. In contexts involving nested classes,Self will always refer to the innermost class.

The following uses ofSelfare accepted:

classReturnsSelf:
deffoo(self)->Self:...# Accepted

@classmethod
defbar(cls)->Self:# Accepted
returncls()

def__new__(cls,value:int)->Self:...# Accepted

defexplicitly_use_self(self:Self)->Self:...# Accepted

# Accepted (Self can be nested within other types)
defreturns_list(self)->list[Self]:...

# Accepted (Self can be nested within other types)
@classmethod
defreturn_cls(cls)->type[Self]:
returncls

classChild(ReturnsSelf):
# Accepted (we can override a method that uses Self annotations)
deffoo(self)->Self:...

classTakesSelf:
deffoo(self,other:Self)->bool:...# Accepted

classRecursive:
# Accepted (treated as an @property returning ``Self | None``)
next:Self|None

classCallableAttribute:
deffoo(self)->int:...

# Accepted (treated as an @property returning the Callable type)
bar:Callable[[Self],int]=foo

classHasNestedFunction:
x:int=42

deffoo(self)->None:

# Accepted (Self is bound to HasNestedFunction).
defnested(z:int,inner_self:Self)->Self:
print(z)
print(inner_self.x)
returninner_self

nested(42,self)# OK


classOuter:
classInner:
deffoo(self)->Self:...# Accepted (Self is bound to Inner)

The following uses ofSelfare rejected.

deffoo(bar:Self)->Self:...# Rejected (not within a class)

bar:Self# Rejected (not within a class)

classFoo:
# Rejected (Self is treated as unknown).
defhas_existing_self_annotation(self:T)->Self:...

classFoo:
defreturn_concrete_type(self)->Self:
returnFoo()# Rejected (see FooChild below for rationale)

classFooChild(Foo):
child_value:int=42

defchild_method(self)->None:
# At runtime, this would be Foo, not FooChild.
y=self.return_concrete_type()

y.child_value
# Runtime error: Foo has no attribute child_value

classBar(Generic[T]):
defbar(self)->T:...

classBaz(Bar[Self]):...# Rejected

We reject type aliases containingSelf.SupportingSelf outside class definitions can require a lot of special-handling in type checkers. Given that it also goes against the rest of the PEP to useSelfoutside a class definition, we believe the added convenience of aliases is not worth it:

TupleSelf=Tuple[Self,Self]# Rejected

classAlias:
defreturn_tuple(self)->TupleSelf:# Rejected
return(self,self)

Note that we rejectSelfin staticmethods.Selfdoes not add much value since there is noselforclsto return. The only possible use cases would be to return a parameter itself or some element from a container passed in as a parameter. These don’t seem worth the additional complexity.

classBase:
@staticmethod
defmake()->Self:# Rejected
...

@staticmethod
defreturn_parameter(foo:Self)->Self:# Rejected
...

Likewise, we rejectSelfin metaclasses.Selfin this PEP consistently refers to the same type (that ofself). But in metaclasses, it would have to refer to different types in different method signatures. For example, in __mul__,Selfin the return type would refer to the implementing class Foo,not the enclosing classMyMetaclass.But, in__new__,Self in the return type would refer to the enclosing classMyMetaclass.To avoid confusion, we reject this edge case.

classMyMetaclass(type):
def__new__(cls,*args:Any)->Self:# Rejected
returnsuper().__new__(cls,*args)

def__mul__(cls,count:int)->list[Self]:# Rejected
return[cls()]*count

classFoo(metaclass=MyMetaclass):...

Runtime behavior

BecauseSelfis not subscriptable, we propose an implementation similar to typing.NoReturn.

@_SpecialForm
defSelf(self,params):
"""Used to spell the type of" self "in classes.

Example::

from typing import Self

class ReturnsSelf:
def parse(self, data: bytes) -> Self:
...
return self

"""
raiseTypeError(f"{self}is not subscriptable ")

Rejected Alternatives

Allow the Type Checker to Infer the Return Type

One proposal is to leave theSelftype implicit and let the type checker infer from the body of the method that the return type must be the same as the type of theselfparameter:

classShape:
defset_scale(self,scale:float):
self.scale=scale
returnself# Type checker infers that we are returning self

We reject this because Explicit Is Better Than Implicit. Beyond that, the above approach will fail for type stubs, which don’t have method bodies to analyze.

Reference Implementations

Mypy: Proof of concept implementation inMypy.

Pyright: v1.1.184

Runtime implementation ofSelf:PR.

Resources

Similar discussions on aSelftype in Python started in Mypy around 2016: Mypy issue #1212- SelfType or another way to spell “type of self”. However, the approach ultimately taken there was the boundedTypeVarapproach shown in our “before” examples. Other issues that discuss this includeMypy issue #2354- Self types in generic classes.

Pradeep made a concrete proposal at the PyCon Typing Summit 2021:
recorded talk,slides.

James brought up the proposal independently on typing-sig: Typing-sig thread.

Other languages have similar ways to express the type of the enclosing class:

Thanks to the following people for their feedback on the PEP:

Jia Chen, Rebecca Chen, Sergei Lebedev, Kaylynn Morgan, Tuomas Suutari, Eric Traut, Alex Waygood, Shannon Zhu, and Никита Соболев

Source:https://github.com/python/peps/blob/main/peps/pep-0673.rst

Last modified:2024-06-11 22:12:09 GMT