Why can’ttype[]be used?

One might think you could define the example functions above to take a type[C]- which is syntax that already exists - rather than a TypeExpr[T].However if you were to do that then certain type expressions likestr|None- which are not class objects and therefore nottypes at runtime - would be rejected:

# NOTE: Uses a type[C] parameter rather than a TypeExpr[T]
deftrycast_type[C](typ:type[C],value:object)->T|None:...

trycast_type(str,'hi')# ok; str is a type
trycast_type(Optional[str],'hi')# ERROR; Optional[str] is not a type
trycast_type(str|int,'hi')# ERROR; (str | int) is not a type
trycast_type(MyTypedDict,dict(value='hi'))# questionable; accepted by mypy 1.9.0

To solve that problem,type[]could be widened to include the additional values allowed byTypeExpr.However doing so would lose type[]’s current ability to spell a class object which always supports instantiation andisinstancechecks, unlike arbitrary type expression objects. ThereforeTypeExpris proposed as new notation instead.

For a longer explanation of why we don’t just widentype[T]to accept all type expressions, see Widen type[C] to support all type expressions.

Common kinds of functions that would benefit from TypeExpr

A survey of various Python librariesrevealed a few kinds of commonly defined functions which would benefit fromTypeExpr[]:

• Assignability checkers:
  • Returns whether a value is assignable to a type expression. If so then also narrows the type of the value to match the type expression.
  • Pattern 1: defisassignable[T](value:object,typx:TypeExpr[T])->TypeIs[T]
  • Pattern 2: defismatch[T](value:object,typx:TypeExpr[T])->TypeGuard[T]
  • Examples: beartype.is_bearable,trycast.isassignable, typeguard.check_type,xdsl.isa

• Converters:
  • If a value is assignable to (or coercible to) a type expression, aconverterreturns the value narrowed to (or coerced to) that type expression. Otherwise, it raises an exception.
  • Pattern 1: defconvert[T](value:object,typx:TypeExpr[T])->T
    • Examples: cattrs.BaseConverter.structure,trycast.checkcast, typedload.load
  • Pattern 2:

    classConverter[T]:
    def__init__(self,typx:TypeExpr[T])->None:...
    defconvert(self,value:object)->T:...
    

    • Examples: pydantic.TypeAdapter(T).validate_ Python, mashumaro.JSONDecoder(T).decode

• Typed field definitions:
  • Pattern:

    classField[T]:
    value_type:TypeExpr[T]
    

  • Examples: attrs.make_class, dataclasses.make_dataclass[3],openapify

The survey also identified some introspection functions that take annotation expressions as input using plainobjects which would notgain functionality by marking those inputs asTypeExpr[]:

• General introspection operations:
  • Pattern:defget_annotation_info(maybe_annx:object)->object
  • Examples: typing.{get_origin,get_args}, typing_inspect.{is_*_type, get_origin, get_parameters}

Using TypeExprs

ATypeExpris a new kind of type expression, usable in any context where a type expression is valid, as a function parameter type, a return type, or a variable type:

defis_union_type(typx:TypeExpr)->bool:...# parameter type

defunion_of[S,T](s:TypeExpr[S],t:TypeExpr[T])\
->TypeExpr[S|T]:...# return type

STR_TYPE:TypeExpr=str# variable type

Note however that anunannotatedvariable assigned a type expression literal will not be inferred to be ofTypeExprtype by type checkers because PEP 484reserves that syntax for defining type aliases:

• No:

  STR_TYPE=str# OOPS; treated as a type alias!
  

If you want a type checker to recognize a type expression literal in a bare assignment you’ll need to explicitly declare the assignment-target as havingTypeExprtype:

• Yes:

  STR_TYPE:TypeExpr=str
  

• Yes:

  STR_TYPE:TypeExpr
  STR_TYPE=str
  

• Okay, but discouraged:

  STR_TYPE=str# type: TypeExpr # the type comment is significant
  

TypeExprvalues can be passed around and assigned just like normal values:

defswap1[S,T](t1:TypeExpr[S],t2:TypeExpr[T])->tuple[TypeExpr[T],TypeExpr[S]]:
t1_new:TypeExpr[T]=t2# assigns a TypeExpr value to a new annotated variable
t2_new:TypeExpr[S]=t1
return(t1_new,t2_new)

defswap2[S,T](t1:TypeExpr[S],t2:TypeExpr[T])->tuple[TypeExpr[T],TypeExpr[S]]:
t1_new=t2# assigns a TypeExpr value to a new unannotated variable
t2_new=t1
assert_type(t1_new,TypeExpr[T])
assert_type(t2_new,TypeExpr[S])
return(t1_new,t2_new)

# NOTE: A more straightforward implementation would use isinstance()
defensure_int(value:object)->None:
value_type:TypeExpr=type(value)# assigns a type (a subtype of TypeExpr)
assertvalue_type==int

TypeExpr Values

A variable of typeTypeExpr[T]whereTis a type, can hold any type expression object- the result of evaluating a type expression at runtime - which is a subtype ofT.

Incomplete expressions like a bareOptionalorUnionwhich do not spell a type are notTypeExprvalues.

TypeExpr[...]is itself aTypeExprvalue:

OPTIONAL_INT_TYPE:TypeExpr=TypeExpr[int|None]# OK
assertisassignable(int|None,OPTIONAL_INT_TYPE)

TypeExpr[]values includealltype expressions including some non-universal type expressionswhich are not valid in all annotation contexts. In particular:

• Self(valid only in some contexts)
• TypeGuard[...](valid only in some contexts)
• TypeIs[...](valid only in some contexts)

NONE=TypeExpr(None)
INT1=TypeExpr('int')# stringified type expression
INT2=TypeExpr(int)

STRS_TYPE_NAME:Literal['str','list[str]']='str'
STRS_TYPE:TypeExpr=STRS_TYPE_NAME# ERROR: Literal[] value is not a TypeExpr

DIRECTION_TYPE:TypeExpr[Literal['left','right']]=Literal['left','right']# OK

>>>IntTree=list[typing.Union[int,'IntTree']]
>>>IntTree
list[typing.Union[int, ForwardRef('IntTree')]]

INT_TREE:TypeExpr=ForwardRef('IntTree')# ERROR: Runtime-only form

INT_TREE=cast(TypeExpr,ForwardRef('IntTree'))# OK

Subtyping

Whether aTypeExprvalue can be assigned from one variable to another is determined by the following rules:

Interactions with isinstance() and issubclass()

TheTypeExprspecial form cannot be used as thetypeargument to isinstance:

>>>isinstance(str,TypeExpr)
TypeError: typing.TypeExpr cannot be used with isinstance()

>>>isinstance(str,TypeExpr[str])
TypeError: isinstance() argument 2 cannot be a parameterized generic

TheTypeExprspecial form cannot be used as any argument to issubclass:

>>>issubclass(TypeExpr,object)
TypeError: issubclass() arg 1 must be a class

>>>issubclass(object,TypeExpr)
TypeError: typing.TypeExpr cannot be used with issubclass()

Affected signatures in the standard library

Introspecting TypeExpr Values

ATypeExpris very similar to anobjectat runtime, with no additional attributes or methods defined.

You can use existing introspection functions liketyping.get_originand typing.get_argsto extract the components of a type expression that looks likeOrigin[Arg1,Arg2,...,ArgN]:

importtyping

defstrip_annotated_metadata(typx:TypeExpr[T])->TypeExpr[T]:
iftyping.get_origin(typx)istyping.Annotated:
typx=cast(TypeExpr[T],typing.get_args(typx)[0])
returntypx

You can also useisinstanceandisto distinguish one kind of type expression from another:

importtypes
importtyping

defsplit_union(typx:TypeExpr)->tuple[TypeExpr,...]:
ifisinstance(typx,types.UnionType):# X | Y
returncast(tuple[TypeExpr,...],typing.get_args(typx))
iftyping.get_origin(typx)istyping.Union:# Union[X, Y]
returncast(tuple[TypeExpr,...],typing.get_args(typx))
iftypxin(typing.Never,typing.NoReturn,):
return()
return(typx,)

Combining with a type variable

TypeExpr[]can be parameterized by a type variable that is used elsewhere within the same function definition:

defas_instance[T](typx:TypeExpr[T])->T|None:
returntypx()ifisinstance(typx,type)elseNone

Combining with type[]

BothTypeExpr[]andtype[]can be parameterized by the same type variable within the same function definition:

defas_type[T](typx:TypeExpr[T])->type[T]|None:
returntypxifisinstance(typx,type)elseNone

Combining with TypeIs[] and TypeGuard[]

A type variable parameterizing aTypeExpr[]can also be used by aTypeIs[] within the same function definition:

defisassignable[T](value:object,typx:TypeExpr[T])->TypeIs[T]:...

count:int|str=...
ifisassignable(count,int):
assert_type(count,int)
else:
assert_type(count,str)

or by aTypeGuard[]within the same function definition:

defisdefault[T](value:object,typx:TypeExpr[T])->TypeGuard[T]:
return(value==typx())ifisinstance(typx,type)elseFalse

value:int|str=''
ifisdefault(value,int):
assert_type(value,int)
assert0==value
elifisdefault(value,str):
assert_type(value,str)
assert''==value
else:
assert_type(value,int|str)

Challenges When Accepting All TypeExprs

A function that takes anarbitraryTypeExpras input must support a large variety of possible type expressions and is not easy to write. Some challenges faced by such a function include:

• An ever-increasing number of typing special forms are introduced with each new Python version which must be recognized, with special handling required for each one.
• Stringified type annotations[5](like'list[str]') must beparsed(to something liketyping.List[str]) to be introspected.
  • In practice it is extremely difficult for stringified type annotations to be handled reliably at runtime, so runtime type checkers may opt to not support them at all.
• Resolving string-based forward references inside type expressions to actual values must typically be done usingeval(), which is difficult/impossible to use in a safe way.
• Recursive types likeIntTree=list[typing.Union[int,'IntTree']] are not possible to fully resolve.
• Supporting user-defined generic types (like Django’s QuerySet[User]) requires user-defined functions to recognize/parse, which a runtime type checker should provide a registration API for.

Widen type[C] to support all type expressions

typewasdesignedto only be used to describe class objects. A class object can always be used as the second argument ofisinstance() and can usually be instantiated by calling it.

TypeExpron the other hand is typically introspected by the user in some way, is not necessarily directly instantiable, and is not necessarily directly usable in a regularisinstance()check.

It would be possible to widentypeto include the additional values allowed byTypeExprbut it would reduce clarity about the user’s intentions when working with atype.Different concepts and usage patterns; different spellings.

Accept arbitrary annotation expressions

Certain typing special forms can be used insomebut notall annotation contexts:

For exampleFinal[]can be used as a variable type but not as a parameter type or a return type:

some_const:Final[str]=...# OK

deffoo(not_reassignable:Final[object]):...# ERROR: Final[] not allowed here

defnonsense()->Final[object]:...# ERROR: Final[] not meaningful here

TypeExpr[T]does not allow matching such annotation expressions because it is not clear what it would mean for such an expression to parameterized by a type variable in positionT:

defismatch[T](value:object,typx:TypeExpr[T])->TypeGuard[T]:...

deffoo(some_arg):
ifismatch(some_arg,Final[int]):# ERROR: Final[int] is not a TypeExpr
reveal_type(some_arg)#? NOT Final[int], because invalid for a parameter

Functions that wish to operate onallkinds of annotation expressions, including those that are notTypeExprs, can continue to accept such inputs asobjectparameters, as they must do so today.

Accept only universal type expressions

Earlier drafts of this PEP only allowedTypeExpr[]to match the subset of type expressions which are valid inallcontexts, excluding non-universal type expressions. However doing that would effectively create a new subset of annotation expressions that Python typing users would have to understand, on top of all the existing distinctions between “class objects”, “type expressions”, and “annotation expressions”.

To avoid introducing yet another concept that everyone has to learn, this proposal just roundsTypeExpr[]to exactly match the existing definition of a “type expression”.

Support pattern matching on type expressions

It was asserted that some functions may wish to pattern match on the interior of type expressions in their signatures.

One use case is to allow a function to explicitly enumerate all the specifickinds of type expressions it supports as input. Consider the following possible pattern matching syntax:

@overload
defcheckcast(typx:TypeExpr[AT=Annotated[T,*Anns]],value:str)->T:...
@overload
defcheckcast(typx:TypeExpr[UT=Union[*Ts]],value:str)->Union[*Ts]:...
@overload
defcheckcast(typx:type[C],value:str)->C:...
#... (more)

All functions observed in the wild that conceptually take aTypeExpr[] generally try to supportallkinds of type expressions, so it doesn’t seem valuable to enumerate a particular subset.

Additionally the above syntax isn’t precise enough to fully describe the actual input constraints for a typical function in the wild. For example many functions recognize un-stringified type expressions like list[Movie]but may not recognize type expressions with stringified subcomponents likelist['Movie'].

A second use case for pattern matching on the interior of type expressions is to explicitly match anAnnotated[]form to pull out the interior type argument and strip away the metadata:

defcheckcast(
typx:TypeExpr[T]|TypeExpr[AT=Annotated[T,*Anns]],
value:object
)->T:

HoweverAnnotated[T,metadata]is already treated equivalent toTanyway. There’s no additional value in being explicit about this behavior. The example above could be more-straightforwardly written as the equivalent:

defcheckcast(typx:TypeExpr[T],value:object)->T:

Recognize (T1 | T2) as an implicit TypeExpr value

It would be nice if a value expression likeint|strcould be recognized as an implicitTypeExprvalue and be used directly in a context where a TypeExprwas expected. However making that possible would require making changes to the rules that type checkers use for the|operator. These rules are currently underspecified and would need to be make explicit first, before making changes to them. The PEP author is not sufficiently motivated to take on that specification work at the time of writing.