PEP 696 – Type Defaults for Type Parameters

Author:: James Hilton-Balfe <gobot1234yt at gmail >
Sponsor:: Jelle Zijlstra <jelle.zijlstra at gmail >
Discussions-To:: Discourse thread
Status:: Final
Type:: Standards Track
Topic:: Typing
Created:: 14-Jul-2022
Python-Version:: 3.13
Post-History:: 22-Mar-2022, 08-Jan-2023
Resolution:: Discourse message

Table of Contents

Attention

This PEP is a historical document: seeDefaults for Type Parametersand Type parameter listsfor up-to-date specs and documentation. Canonical typing specs are maintained at thetyping specs site;runtime typing behaviour is described in the CPython documentation.

See thetyping specification update processfor how to propose changes to the typing spec.

Abstract

This PEP introduces the concept of type defaults for type parameters, includingTypeVar,ParamSpec,andTypeVarTuple, which act as defaults for type parameters for which no type is specified.

Default type argument support is available in some popular languages such as C++, TypeScript, and Rust. A survey of type parameter syntax in some common languages has been conducted by the author ofPEP 695 and can be found in its Appendix A.

Motivation

T=TypeVar("T",default=int)# This means that if no type is specified T = int

@dataclass
classBox(Generic[T]):
value:T|None=None

reveal_type(Box())# type is Box[int]
reveal_type(Box(value="Hello World!"))# type is Box[str]

One place thisregularly comes upisGenerator.I propose changing thestub definitionto something like:

YieldT=TypeVar("YieldT")
SendT=TypeVar("SendT",default=None)
ReturnT=TypeVar("ReturnT",default=None)

classGenerator(Generic[YieldT,SendT,ReturnT]):...

Generator[int]==Generator[int,None]==Generator[int,None,None]

This is also useful for aGenericthat is commonly over one type.

classBot:...

BotT=TypeVar("BotT",bound=Bot,default=Bot)

classContext(Generic[BotT]):
bot:BotT

classMyBot(Bot):...

reveal_type(Context().bot)# type is Bot # notice this is not Any which is what it would be currently
reveal_type(Context[MyBot]().bot)# type is MyBot

Not only does this improve typing for those who explicitly use it, it also helps non-typing users who rely on auto-complete to speed up their development.

This design pattern is common in projects like:

discord.py— where the example above was taken from.
NumPy— the default for types likendarray’sdtypewould befloat64.Currently it’s UnknownorAny.
TensorFlow— this could be used for Tensor similarly tonumpy.ndarrayand would be useful to simplify the definition ofLayer.

Specification

Default Ordering and Subscription Rules

The order for defaults should follow the standard function parameter rules, so a type parameter with nodefaultcannot follow one with adefaultvalue. Doing so should ideally raise aTypeErrorin typing._GenericAlias/types.GenericAlias,and a type checker should flag this as an error.

DefaultStrT=TypeVar("DefaultStrT",default=str)
DefaultIntT=TypeVar("DefaultIntT",default=int)
DefaultBoolT=TypeVar("DefaultBoolT",default=bool)
T=TypeVar("T")
T2=TypeVar("T2")

classNonDefaultFollowsDefault(Generic[DefaultStrT,T]):...# Invalid: non-default TypeVars cannot follow ones with defaults


classNoNonDefaults(Generic[DefaultStrT,DefaultIntT]):...

(
NoNoneDefaults==
NoNoneDefaults[str]==
NoNoneDefaults[str,int]
)# All valid


classOneDefault(Generic[T,DefaultBoolT]):...

OneDefault[float]==OneDefault[float,bool]# Valid
reveal_type(OneDefault)# type is type[OneDefault[T, DefaultBoolT = bool]]
reveal_type(OneDefault[float]())# type is OneDefault[float, bool]


classAllTheDefaults(Generic[T1,T2,DefaultStrT,DefaultIntT,DefaultBoolT]):...

reveal_type(AllTheDefaults)# type is type[AllTheDefaults[T1, T2, DefaultStrT = str, DefaultIntT = int, DefaultBoolT = bool]]
reveal_type(AllTheDefaults[int,complex]())# type is AllTheDefaults[int, complex, str, int, bool]
AllTheDefaults[int]# Invalid: expected 2 arguments to AllTheDefaults
(
AllTheDefaults[int,complex]==
AllTheDefaults[int,complex,str]==
AllTheDefaults[int,complex,str,int]==
AllTheDefaults[int,complex,str,int,bool]
)# All valid

With the new Python 3.12 syntax for generics (introduced byPEP 695), this can be enforced at compile time:

typeAlias[DefaultT=int,T]=tuple[DefaultT,T]# SyntaxError: non-default TypeVars cannot follow ones with defaults

defgeneric_func[DefaultT=int,T](x:DefaultT,y:T)->None:...# SyntaxError: non-default TypeVars cannot follow ones with defaults

classGenericClass[DefaultT=int,T]:...# SyntaxError: non-default TypeVars cannot follow ones with defaults

`ParamSpec`Defaults

ParamSpecdefaults are defined using the same syntax as TypeVars but use alistof types or an ellipsis literal “...”or another in-scopeParamSpec(seeScoping Rules).

DefaultP=ParamSpec("DefaultP",default=[str,int])

classFoo(Generic[DefaultP]):...

reveal_type(Foo)# type is type[Foo[DefaultP = [str, int]]]
reveal_type(Foo())# type is Foo[[str, int]]
reveal_type(Foo[[bool,bool]]())# type is Foo[[bool, bool]]

`TypeVarTuple`Defaults

TypeVarTupledefaults are defined using the same syntax as TypeVars but use an unpacked tuple of types instead of a single type or another in-scopeTypeVarTuple(seeScoping Rules).

DefaultTs=TypeVarTuple("DefaultTs",default=Unpack[tuple[str,int]])

classFoo(Generic[*DefaultTs]):...

reveal_type(Foo)# type is type[Foo[DefaultTs = *tuple[str, int]]]
reveal_type(Foo())# type is Foo[str, int]
reveal_type(Foo[int,bool]())# type is Foo[int, bool]

Using Another Type Parameter as`default`

This allows for a value to be used again when the type parameter to a generic is missing but another type parameter is specified.

To use another type parameter as a default thedefaultand the type parameter must be the same type (aTypeVar’s default must be aTypeVar,etc.).

This could be used on builtins.slice where thestartparameter should default toint,stop default to the type ofstartand step default toint|None.

StartT=TypeVar("StartT",default=int)
StopT=TypeVar("StopT",default=StartT)
StepT=TypeVar("StepT",default=int|None)

classslice(Generic[StartT,StopT,StepT]):...

reveal_type(slice)# type is type[slice[StartT = int, StopT = StartT, StepT = int | None]]
reveal_type(slice())# type is slice[int, int, int | None]
reveal_type(slice[str]())# type is slice[str, str, int | None]
reveal_type(slice[str,bool,timedelta]())# type is slice[str, bool, timedelta]

T2=TypeVar("T2",default=DefaultStrT)

classFoo(Generic[DefaultStrT,T2]):
def__init__(self,a:DefaultStrT,b:T2)->None:...

reveal_type(Foo(1,""))# type is Foo[int, str]
Foo[int](1,"")# Invalid: Foo[int, str] cannot be assigned to self: Foo[int, int] in Foo.__init__
Foo[int]("",1)# Invalid: Foo[str, int] cannot be assigned to self: Foo[int, int] in Foo.__init__

When using a type parameter as the default to another type parameter, the following rules apply, whereT1is the default forT2.

Scoping Rules

T1must be used beforeT2in the parameter list of the generic.

T2=TypeVar("T2",default=T1)

classFoo(Generic[T1,T2]):...# Valid
classFoo(Generic[T1]):
classBar(Generic[T2]):...# Valid

StartT=TypeVar("StartT",default="StopT")# Swapped defaults around from previous example
StopT=TypeVar("StopT",default=int)
classslice(Generic[StartT,StopT,StepT]):...
# ^^^^^^ Invalid: ordering does not allow StopT to be bound

Using a type parameter from an outer scope as a default is not supported.

Bound Rules

T1’s bound must be a subtype ofT2’s bound.

T1=TypeVar("T1",bound=int)
TypeVar("Ok",default=T1,bound=float)# Valid
TypeVar("AlsoOk",default=T1,bound=int)# Valid
TypeVar("Invalid",default=T1,bound=str)# Invalid: int is not a subtype of str

Constraint Rules

The constraints ofT2must be a superset of the constraints ofT1.

T1=TypeVar("T1",bound=int)
TypeVar("Invalid",float,str,default=T1)# Invalid: upper bound int is incompatible with constraints float or str

T1=TypeVar("T1",int,str)
TypeVar("AlsoOk",int,str,bool,default=T1)# Valid
TypeVar("AlsoInvalid",bool,complex,default=T1)# Invalid: {bool, complex} is not a superset of {int, str}

Type Parameters as Parameters to Generics

Type parameters are valid as parameters to generics inside of a defaultwhen the first parameter is in scope as determined by the previous section.

T=TypeVar("T")
ListDefaultT=TypeVar("ListDefaultT",default=list[T])

classBar(Generic[T,ListDefaultT]):
def__init__(self,x:T,y:ListDefaultT):...

reveal_type(Bar)# type is type[Bar[T, ListDefaultT = list[T]]]
reveal_type(Bar[int])# type is type[Bar[int, list[int]]]
reveal_type(Bar[int]())# type is Bar[int, list[int]]
reveal_type(Bar[int,list[str]]())# type is Bar[int, list[str]]
reveal_type(Bar[int,str]())# type is Bar[int, str]

Specialisation Rules

Type parameters currently cannot be further subscripted. This might change ifHigher Kinded TypeVars are implemented.

`GenericTypeAlias`es

GenericTypeAliases should be able to be further subscripted following normal subscription rules. If a type parameter has a default that hasn’t been overridden it should be treated like it was substituted into theTypeAlias.However, it can be specialised further down the line.

classSomethingWithNoDefaults(Generic[T,T2]):...

MyAlias:TypeAlias=SomethingWithNoDefaults[int,DefaultStrT]# Valid
reveal_type(MyAlias)# type is type[SomethingWithNoDefaults[int, DefaultStrT]]
reveal_type(MyAlias[bool]())# type is SomethingWithNoDefaults[int, bool]

MyAlias[bool,int]# Invalid: too many arguments passed to MyAlias

Subclassing

Subclasses ofGenerics with type parameters that have defaults behave similarly toGenericTypeAliases. That is, subclasses can be further subscripted following normal subscription rules, non-overridden defaults should be substituted in, and type parameters with such defaults can be further specialised down the line.

classSubclassMe(Generic[T,DefaultStrT]):
x:DefaultStrT

classBar(SubclassMe[int,DefaultStrT]):...
reveal_type(Bar)# type is type[Bar[DefaultStrT = str]]
reveal_type(Bar())# type is Bar[str]
reveal_type(Bar[bool]())# type is Bar[bool]

classFoo(SubclassMe[float]):...

reveal_type(Foo().x)# type is str

Foo[str]# Invalid: Foo cannot be further subscripted

classBaz(Generic[DefaultIntT,DefaultStrT]):...

classSpam(Baz):...
reveal_type(Spam())# type is <subclass of Baz[int, str]>

Using`bound`and`default`

If bothboundanddefaultare passeddefaultmust be a subtype ofbound.Otherwise the type checker should generate an error.

TypeVar("Ok",bound=float,default=int)# Valid
TypeVar("Invalid",bound=str,default=int)# Invalid: the bound and default are incompatible

Constraints

For constrainedTypeVars, the default needs to be one of the constraints. A type checker should generate an error even if it is a subtype of one of the constraints.

TypeVar("Ok",float,str,default=float)# Valid
TypeVar("Invalid",float,str,default=int)# Invalid: expected one of float or str got int

Function Defaults

In generic functions, type checkers may use a type parameter’s default when the type parameter cannot be solved to anything. We leave the semantics of this usage unspecified, as ensuring thedefaultis returned in every code path where the type parameter can go unsolved may be too hard to implement. Type checkers are free to either disallow this case or experiment with implementing support.

T=TypeVar('T',default=int)
deffunc(x:int|set[T])->T:...
reveal_type(func(0))# a type checker may reveal T's default of int here

Defaults following`TypeVarTuple`

ATypeVarthat immediately follows aTypeVarTupleis not allowed to have a default, because it would be ambiguous whether a type argument should be bound to theTypeVarTupleor the defaultedTypeVar.

Ts=TypeVarTuple("Ts")
T=TypeVar("T",default=bool)

classFoo(Generic[Ts,T]):...# Type checker error

# Could be reasonably interpreted as either Ts = (int, str, float), T = bool
# or Ts = (int, str), T = float
Foo[int,str,float]

With the Python 3.12 built-in generic syntax, this case should raise a SyntaxError.

However, it is allowed to have aParamSpecwith a default following a TypeVarTuplewith a default, as there can be no ambiguity between a type argument for theParamSpecand one for theTypeVarTuple.

Ts=TypeVarTuple("Ts")
P=ParamSpec("P",default=[float,bool])

classFoo(Generic[Ts,P]):...# Valid

Foo[int,str]# Ts = (int, str), P = [float, bool]
Foo[int,str,[bytes]]# Ts = (int, str), P = [bytes]

Subtyping

Type parameter defaults do not affect the subtyping rules for generic classes. In particular, defaults can be ignored when considering whether a class is compatible with a generic protocol.

`TypeVarTuple`s as Defaults

Using aTypeVarTupleas a default is not supported because:

Scoping Rulesdoes not allow usage of type parameters from outer scopes.
MultipleTypeVarTuples cannot appear in the type parameter list for a single object, as specified in PEP 646.

These reasons leave no current valid location where a TypeVarTuplecould be used as the default of anotherTypeVarTuple.

Binding rules

Type parameter defaults should be bound by attribute access (including call and subscript).

classFoo[T=int]:
defmeth(self)->Self:
returnself

reveal_type(Foo.meth)# type is (self: Foo[int]) -> Foo[int]

Implementation

At runtime, this would involve the following changes to thetyping module.

The classesTypeVar,ParamSpec,andTypeVarTupleshould expose the type passed todefault.This would be available as a__default__attribute, which would beNoneif no argument is passed andNoneTypeifdefault=None.

The following changes would be required to bothGenericAliases:

logic to determine the defaults required for a subscription.
ideally, logic to determine if subscription (like Generic[T,DefaultT]) would be valid.

The grammar for type parameter lists would need to be updated to allow defaults; see below.

A reference implementation of the runtime changes can be found at https://github /Gobot1234/c Python /tree/pep-696

A reference implementation of the type checker can be found at https://github /Gobot1234/mypy/tree/TypeVar-defaults

Pyright currently supports this functionality.

Grammar changes

The syntax added inPEP 695will be extended to introduce a way to specify defaults for type parameters using the “=” operator inside of the square brackets like so:

# TypeVars
classFoo[T=str]:...

# ParamSpecs
classBaz[**P=[int,str]]:...

# TypeVarTuples
classQux[*Ts=*tuple[int,bool]]:...

# TypeAliases
typeFoo[T,U=str]=Bar[T,U]
typeBaz[**P=[int,str]]=Spam[**P]
typeQux[*Ts=*tuple[str]]=Ham[*Ts]
typeRab[U,T=str]=Bar[T,U]

Similarly to the bound for a type parameter, defaults should be lazily evaluated, with the same scoping rules to avoid the unnecessary usage of quotes around them.

This functionality was included in the initial draft ofPEP 695but was removed due to scope creep.

The following changes would be made to the grammar:

type_param:
|a=NAMEb=[type_param_bound]d=[type_param_default]
|a=NAMEc=[type_param_constraint]d=[type_param_default]
|'*'a=NAMEd=[type_param_default]
|'**'a=NAMEd=[type_param_default]

type_param_default:
|'='e=expression
|'='e=starred_expression

The compiler would enforce that type parameters without defaults cannot follow type parameters with defaults and thatTypeVars with defaults cannot immediately followTypeVarTuples.

Rejected Alternatives

Allowing the Type Parameters Defaults to Be Passed to`type.new`’s`**kwargs`

T=TypeVar("T")

@dataclass
classBox(Generic[T],T=int):
value:T|None=None

While this is much easier to read and follows a similar rationale to the TypeVarunary syntax,it would not be backwards compatible asTmight already be passed to a metaclass/superclass or support classes that don’t subclassGeneric at runtime.

Ideally, ifPEP 637wasn’t rejected, the following would be acceptable:

T=TypeVar("T")

@dataclass
classBox(Generic[T=int]):
value:T|None=None

Allowing Non-defaults to Follow Defaults

YieldT=TypeVar("YieldT",default=Any)
SendT=TypeVar("SendT",default=Any)
ReturnT=TypeVar("ReturnT")

classCoroutine(Generic[YieldT,SendT,ReturnT]):...

Coroutine[int]==Coroutine[Any,Any,int]

Allowing non-defaults to follow defaults would alleviate the issues with returning types likeCoroutinefrom functions where the most used type argument is the last (the return). Allowing non-defaults to follow defaults is too confusing and potentially ambiguous, even if only the above two forms were valid. Changing the argument order now would also break a lot of codebases. This is also solvable in most cases using a TypeAlias.

Coro:TypeAlias=Coroutine[Any,Any,T]
Coro[int]==Coroutine[Any,Any,int]

Having`default`Implicitly Be`bound`

In an earlier version of this PEP, thedefaultwas implicitly set toboundif no value was passed fordefault.This while convenient, could have a type parameter with no default follow a type parameter with a default. Consider:

T=TypeVar("T",bound=int)# default is implicitly int
U=TypeVar("U")

classFoo(Generic[T,U]):
...

# would expand to

T=TypeVar("T",bound=int,default=int)
U=TypeVar("U")

classFoo(Generic[T,U]):
...

This would have also been a breaking change for a small number of cases where the code relied onAnybeing the implicit default.

Allowing Type Parameters With Defaults To Be Used in Function Signatures

A previous version of this PEP allowedTypeVarLikes with defaults to be used in function signatures. This was removed for the reasons described in Function Defaults.Hopefully, this can be added in the future if a way to get the runtime value of a type parameter is added.

Last modified:2024-09-03 17:24:02 GMT

Python Enhancement Proposals