PEP 696 – Type Defaults for Type Parameters
- Author:
- James Hilton-Balfe <gobot1234yt at gmail >
- Sponsor:
- Jelle Zijlstra <jelle.zijlstra at gmail >
- Discussions-To:
- Discourse thread
- Status:
- Final
- Type:
- Standards Track
- Topic:
- Typing
- Created:
- 14-Jul-2022
- Python-Version:
- 3.13
- Post-History:
- 22-Mar-2022, 08-Jan-2023
- Resolution:
- Discourse message
Table of Contents
- Abstract
- Motivation
- Specification
- Binding rules
- Implementation
- Rejected Alternatives
- Acknowledgements
- Copyright
Abstract
This PEP introduces the concept of type defaults for type parameters,
includingTypeVar
,ParamSpec
,andTypeVarTuple
,
which act as defaults for type parameters for which no type is specified.
Default type argument support is available in some popular languages such as C++, TypeScript, and Rust. A survey of type parameter syntax in some common languages has been conducted by the author ofPEP 695 and can be found in its Appendix A.
Motivation
T=TypeVar("T",default=int)# This means that if no type is specified T = int
@dataclass
classBox(Generic[T]):
value:T|None=None
reveal_type(Box())# type is Box[int]
reveal_type(Box(value="Hello World!"))# type is Box[str]
One place thisregularly comes
upisGenerator
.I
propose changing thestub definitionto something like:
YieldT=TypeVar("YieldT")
SendT=TypeVar("SendT",default=None)
ReturnT=TypeVar("ReturnT",default=None)
classGenerator(Generic[YieldT,SendT,ReturnT]):...
Generator[int]==Generator[int,None]==Generator[int,None,None]
This is also useful for aGeneric
that is commonly over one type.
classBot:...
BotT=TypeVar("BotT",bound=Bot,default=Bot)
classContext(Generic[BotT]):
bot:BotT
classMyBot(Bot):...
reveal_type(Context().bot)# type is Bot # notice this is not Any which is what it would be currently
reveal_type(Context[MyBot]().bot)# type is MyBot
Not only does this improve typing for those who explicitly use it, it also helps non-typing users who rely on auto-complete to speed up their development.
This design pattern is common in projects like:
- discord.py— where the example above was taken from.
- NumPy— the default for types
like
ndarray
’sdtype
would befloat64
.Currently it’sUnknown
orAny
. - TensorFlow— this
could be used for Tensor similarly to
numpy.ndarray
and would be useful to simplify the definition ofLayer
.
Specification
Default Ordering and Subscription Rules
The order for defaults should follow the standard function parameter
rules, so a type parameter with nodefault
cannot follow one with
adefault
value. Doing so should ideally raise aTypeError
in
typing._GenericAlias
/types.GenericAlias
,and a type checker
should flag this as an error.
DefaultStrT=TypeVar("DefaultStrT",default=str)
DefaultIntT=TypeVar("DefaultIntT",default=int)
DefaultBoolT=TypeVar("DefaultBoolT",default=bool)
T=TypeVar("T")
T2=TypeVar("T2")
classNonDefaultFollowsDefault(Generic[DefaultStrT,T]):...# Invalid: non-default TypeVars cannot follow ones with defaults
classNoNonDefaults(Generic[DefaultStrT,DefaultIntT]):...
(
NoNoneDefaults==
NoNoneDefaults[str]==
NoNoneDefaults[str,int]
)# All valid
classOneDefault(Generic[T,DefaultBoolT]):...
OneDefault[float]==OneDefault[float,bool]# Valid
reveal_type(OneDefault)# type is type[OneDefault[T, DefaultBoolT = bool]]
reveal_type(OneDefault[float]())# type is OneDefault[float, bool]
classAllTheDefaults(Generic[T1,T2,DefaultStrT,DefaultIntT,DefaultBoolT]):...
reveal_type(AllTheDefaults)# type is type[AllTheDefaults[T1, T2, DefaultStrT = str, DefaultIntT = int, DefaultBoolT = bool]]
reveal_type(AllTheDefaults[int,complex]())# type is AllTheDefaults[int, complex, str, int, bool]
AllTheDefaults[int]# Invalid: expected 2 arguments to AllTheDefaults
(
AllTheDefaults[int,complex]==
AllTheDefaults[int,complex,str]==
AllTheDefaults[int,complex,str,int]==
AllTheDefaults[int,complex,str,int,bool]
)# All valid
With the new Python 3.12 syntax for generics (introduced byPEP 695), this can be enforced at compile time:
typeAlias[DefaultT=int,T]=tuple[DefaultT,T]# SyntaxError: non-default TypeVars cannot follow ones with defaults
defgeneric_func[DefaultT=int,T](x:DefaultT,y:T)->None:...# SyntaxError: non-default TypeVars cannot follow ones with defaults
classGenericClass[DefaultT=int,T]:...# SyntaxError: non-default TypeVars cannot follow ones with defaults
ParamSpec
Defaults
ParamSpec
defaults are defined using the same syntax as
TypeVar
s but use alist
of types or an ellipsis
literal “...
”or another in-scopeParamSpec
(seeScoping Rules).
DefaultP=ParamSpec("DefaultP",default=[str,int])
classFoo(Generic[DefaultP]):...
reveal_type(Foo)# type is type[Foo[DefaultP = [str, int]]]
reveal_type(Foo())# type is Foo[[str, int]]
reveal_type(Foo[[bool,bool]]())# type is Foo[[bool, bool]]
TypeVarTuple
Defaults
TypeVarTuple
defaults are defined using the same syntax as
TypeVar
s but use an unpacked tuple of types instead of a single type
or another in-scopeTypeVarTuple
(seeScoping Rules).
DefaultTs=TypeVarTuple("DefaultTs",default=Unpack[tuple[str,int]])
classFoo(Generic[*DefaultTs]):...
reveal_type(Foo)# type is type[Foo[DefaultTs = *tuple[str, int]]]
reveal_type(Foo())# type is Foo[str, int]
reveal_type(Foo[int,bool]())# type is Foo[int, bool]
Using Another Type Parameter asdefault
This allows for a value to be used again when the type parameter to a generic is missing but another type parameter is specified.
To use another type parameter as a default thedefault
and the
type parameter must be the same type (aTypeVar
’s default must be
aTypeVar
,etc.).
This could be used on builtins.slice
where thestart
parameter should default toint
,stop
default to the type ofstart
and step default toint|None
.
StartT=TypeVar("StartT",default=int)
StopT=TypeVar("StopT",default=StartT)
StepT=TypeVar("StepT",default=int|None)
classslice(Generic[StartT,StopT,StepT]):...
reveal_type(slice)# type is type[slice[StartT = int, StopT = StartT, StepT = int | None]]
reveal_type(slice())# type is slice[int, int, int | None]
reveal_type(slice[str]())# type is slice[str, str, int | None]
reveal_type(slice[str,bool,timedelta]())# type is slice[str, bool, timedelta]
T2=TypeVar("T2",default=DefaultStrT)
classFoo(Generic[DefaultStrT,T2]):
def__init__(self,a:DefaultStrT,b:T2)->None:...
reveal_type(Foo(1,""))# type is Foo[int, str]
Foo[int](1,"")# Invalid: Foo[int, str] cannot be assigned to self: Foo[int, int] in Foo.__init__
Foo[int]("",1)# Invalid: Foo[str, int] cannot be assigned to self: Foo[int, int] in Foo.__init__
When using a type parameter as the default to another type parameter, the
following rules apply, whereT1
is the default forT2
.
Scoping Rules
T1
must be used beforeT2
in the parameter list of the generic.
T2=TypeVar("T2",default=T1)
classFoo(Generic[T1,T2]):...# Valid
classFoo(Generic[T1]):
classBar(Generic[T2]):...# Valid
StartT=TypeVar("StartT",default="StopT")# Swapped defaults around from previous example
StopT=TypeVar("StopT",default=int)
classslice(Generic[StartT,StopT,StepT]):...
# ^^^^^^ Invalid: ordering does not allow StopT to be bound
Using a type parameter from an outer scope as a default is not supported.
Bound Rules
T1
’s bound must be a subtype ofT2
’s bound.
T1=TypeVar("T1",bound=int)
TypeVar("Ok",default=T1,bound=float)# Valid
TypeVar("AlsoOk",default=T1,bound=int)# Valid
TypeVar("Invalid",default=T1,bound=str)# Invalid: int is not a subtype of str
Constraint Rules
The constraints ofT2
must be a superset of the constraints ofT1
.
T1=TypeVar("T1",bound=int)
TypeVar("Invalid",float,str,default=T1)# Invalid: upper bound int is incompatible with constraints float or str
T1=TypeVar("T1",int,str)
TypeVar("AlsoOk",int,str,bool,default=T1)# Valid
TypeVar("AlsoInvalid",bool,complex,default=T1)# Invalid: {bool, complex} is not a superset of {int, str}
Type Parameters as Parameters to Generics
Type parameters are valid as parameters to generics inside of a
default
when the first parameter is in scope as determined by the
previous section.
T=TypeVar("T")
ListDefaultT=TypeVar("ListDefaultT",default=list[T])
classBar(Generic[T,ListDefaultT]):
def__init__(self,x:T,y:ListDefaultT):...
reveal_type(Bar)# type is type[Bar[T, ListDefaultT = list[T]]]
reveal_type(Bar[int])# type is type[Bar[int, list[int]]]
reveal_type(Bar[int]())# type is Bar[int, list[int]]
reveal_type(Bar[int,list[str]]())# type is Bar[int, list[str]]
reveal_type(Bar[int,str]())# type is Bar[int, str]
Specialisation Rules
Type parameters currently cannot be further subscripted. This might change ifHigher Kinded TypeVars are implemented.
Generic
TypeAlias
es
Generic
TypeAlias
es should be able to be further subscripted
following normal subscription rules. If a type parameter has a default
that hasn’t been overridden it should be treated like it was
substituted into theTypeAlias
.However, it can be specialised
further down the line.
classSomethingWithNoDefaults(Generic[T,T2]):...
MyAlias:TypeAlias=SomethingWithNoDefaults[int,DefaultStrT]# Valid
reveal_type(MyAlias)# type is type[SomethingWithNoDefaults[int, DefaultStrT]]
reveal_type(MyAlias[bool]())# type is SomethingWithNoDefaults[int, bool]
MyAlias[bool,int]# Invalid: too many arguments passed to MyAlias
Subclassing
Subclasses ofGeneric
s with type parameters that have defaults
behave similarly toGeneric
TypeAlias
es. That is, subclasses can be
further subscripted following normal subscription rules, non-overridden
defaults should be substituted in, and type parameters with such defaults can be
further specialised down the line.
classSubclassMe(Generic[T,DefaultStrT]):
x:DefaultStrT
classBar(SubclassMe[int,DefaultStrT]):...
reveal_type(Bar)# type is type[Bar[DefaultStrT = str]]
reveal_type(Bar())# type is Bar[str]
reveal_type(Bar[bool]())# type is Bar[bool]
classFoo(SubclassMe[float]):...
reveal_type(Foo().x)# type is str
Foo[str]# Invalid: Foo cannot be further subscripted
classBaz(Generic[DefaultIntT,DefaultStrT]):...
classSpam(Baz):...
reveal_type(Spam())# type is <subclass of Baz[int, str]>
Usingbound
anddefault
If bothbound
anddefault
are passeddefault
must be a
subtype ofbound
.Otherwise the type checker should generate an
error.
TypeVar("Ok",bound=float,default=int)# Valid
TypeVar("Invalid",bound=str,default=int)# Invalid: the bound and default are incompatible
Constraints
For constrainedTypeVar
s, the default needs to be one of the
constraints. A type checker should generate an error even if it is a
subtype of one of the constraints.
TypeVar("Ok",float,str,default=float)# Valid
TypeVar("Invalid",float,str,default=int)# Invalid: expected one of float or str got int
Function Defaults
In generic functions, type checkers may use a type parameter’s default when the
type parameter cannot be solved to anything. We leave the semantics of this
usage unspecified, as ensuring thedefault
is returned in every code path
where the type parameter can go unsolved may be too hard to implement. Type
checkers are free to either disallow this case or experiment with implementing
support.
T=TypeVar('T',default=int)
deffunc(x:int|set[T])->T:...
reveal_type(func(0))# a type checker may reveal T's default of int here
Defaults followingTypeVarTuple
ATypeVar
that immediately follows aTypeVarTuple
is not allowed
to have a default, because it would be ambiguous whether a type argument
should be bound to theTypeVarTuple
or the defaultedTypeVar
.
Ts=TypeVarTuple("Ts")
T=TypeVar("T",default=bool)
classFoo(Generic[Ts,T]):...# Type checker error
# Could be reasonably interpreted as either Ts = (int, str, float), T = bool
# or Ts = (int, str), T = float
Foo[int,str,float]
With the Python 3.12 built-in generic syntax, this case should raise a SyntaxError.
However, it is allowed to have aParamSpec
with a default following a
TypeVarTuple
with a default, as there can be no ambiguity between a type argument
for theParamSpec
and one for theTypeVarTuple
.
Ts=TypeVarTuple("Ts")
P=ParamSpec("P",default=[float,bool])
classFoo(Generic[Ts,P]):...# Valid
Foo[int,str]# Ts = (int, str), P = [float, bool]
Foo[int,str,[bytes]]# Ts = (int, str), P = [bytes]
Subtyping
Type parameter defaults do not affect the subtyping rules for generic classes. In particular, defaults can be ignored when considering whether a class is compatible with a generic protocol.
TypeVarTuple
s as Defaults
Using aTypeVarTuple
as a default is not supported because:
- Scoping Rulesdoes not allow usage of type parameters from outer scopes.
- Multiple
TypeVarTuple
s cannot appear in the type parameter list for a single object, as specified in PEP 646.
These reasons leave no current valid location where a
TypeVarTuple
could be used as the default of anotherTypeVarTuple
.
Binding rules
Type parameter defaults should be bound by attribute access (including call and subscript).
classFoo[T=int]:
defmeth(self)->Self:
returnself
reveal_type(Foo.meth)# type is (self: Foo[int]) -> Foo[int]
Implementation
At runtime, this would involve the following changes to thetyping
module.
- The classes
TypeVar
,ParamSpec
,andTypeVarTuple
should expose the type passed todefault
.This would be available as a__default__
attribute, which would beNone
if no argument is passed andNoneType
ifdefault=None
.
The following changes would be required to bothGenericAlias
es:
- logic to determine the defaults required for a subscription.
- ideally, logic to determine if subscription (like
Generic[T,DefaultT]
) would be valid.
The grammar for type parameter lists would need to be updated to allow defaults; see below.
A reference implementation of the runtime changes can be found at https://github /Gobot1234/c Python /tree/pep-696
A reference implementation of the type checker can be found at https://github /Gobot1234/mypy/tree/TypeVar-defaults
Pyright currently supports this functionality.
Grammar changes
The syntax added inPEP 695will be extended to introduce a way to specify defaults for type parameters using the “=” operator inside of the square brackets like so:
# TypeVars
classFoo[T=str]:...
# ParamSpecs
classBaz[**P=[int,str]]:...
# TypeVarTuples
classQux[*Ts=*tuple[int,bool]]:...
# TypeAliases
typeFoo[T,U=str]=Bar[T,U]
typeBaz[**P=[int,str]]=Spam[**P]
typeQux[*Ts=*tuple[str]]=Ham[*Ts]
typeRab[U,T=str]=Bar[T,U]
Similarly to the bound for a type parameter, defaults should be lazily evaluated, with the same scoping rules to avoid the unnecessary usage of quotes around them.
This functionality was included in the initial draft ofPEP 695but was removed due to scope creep.
The following changes would be made to the grammar:
type_param:
|a=NAMEb=[type_param_bound]d=[type_param_default]
|a=NAMEc=[type_param_constraint]d=[type_param_default]
|'*'a=NAMEd=[type_param_default]
|'**'a=NAMEd=[type_param_default]
type_param_default:
|'='e=expression
|'='e=starred_expression
The compiler would enforce that type parameters without defaults cannot
follow type parameters with defaults and thatTypeVar
s with defaults
cannot immediately followTypeVarTuple
s.
Rejected Alternatives
Allowing the Type Parameters Defaults to Be Passed totype.__new__
’s**kwargs
T=TypeVar("T")
@dataclass
classBox(Generic[T],T=int):
value:T|None=None
While this is much easier to read and follows a similar rationale to the
TypeVar
unary
syntax,it would not be
backwards compatible asT
might already be passed to a
metaclass/superclass or support classes that don’t subclassGeneric
at runtime.
Ideally, ifPEP 637wasn’t rejected, the following would be acceptable:
T=TypeVar("T")
@dataclass
classBox(Generic[T=int]):
value:T|None=None
Allowing Non-defaults to Follow Defaults
YieldT=TypeVar("YieldT",default=Any)
SendT=TypeVar("SendT",default=Any)
ReturnT=TypeVar("ReturnT")
classCoroutine(Generic[YieldT,SendT,ReturnT]):...
Coroutine[int]==Coroutine[Any,Any,int]
Allowing non-defaults to follow defaults would alleviate the issues with
returning types likeCoroutine
from functions where the most used
type argument is the last (the return). Allowing non-defaults to follow
defaults is too confusing and potentially ambiguous, even if only the
above two forms were valid. Changing the argument order now would also
break a lot of codebases. This is also solvable in most cases using a
TypeAlias
.
Coro:TypeAlias=Coroutine[Any,Any,T]
Coro[int]==Coroutine[Any,Any,int]
Havingdefault
Implicitly Bebound
In an earlier version of this PEP, thedefault
was implicitly set
tobound
if no value was passed fordefault
.This while
convenient, could have a type parameter with no default follow a
type parameter with a default. Consider:
T=TypeVar("T",bound=int)# default is implicitly int
U=TypeVar("U")
classFoo(Generic[T,U]):
...
# would expand to
T=TypeVar("T",bound=int,default=int)
U=TypeVar("U")
classFoo(Generic[T,U]):
...
This would have also been a breaking change for a small number of cases
where the code relied onAny
being the implicit default.
Allowing Type Parameters With Defaults To Be Used in Function Signatures
A previous version of this PEP allowedTypeVarLike
s with defaults to be used in
function signatures. This was removed for the reasons described in
Function Defaults.Hopefully, this can be added in the future if
a way to get the runtime value of a type parameter is added.
Allowing Type Parameters from Outer Scopes indefault
This was deemed too niche a feature to be worth the added complexity. If any cases arise where this is needed, it can be added in a future PEP.
Acknowledgements
Thanks to the following people for their feedback on the PEP:
Eric Traut, Jelle Zijlstra, Joshua Butt, Danny Yamamoto, Kaylynn Morgan and Jakub Kuczys
Copyright
This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.
Source:https://github / Python /peps/blob/main/peps/pep-0696.rst
Last modified:2024-09-03 17:24:02 GMT