Abstract

Currently**kwargscan be type hinted as long as all of the keyword arguments specified by them are of the same type. However, that behaviour can be very limiting. Therefore, in this PEP we propose a new way to enable more precise**kwargstyping. The new approach revolves around using TypedDictto type**kwargsthat comprise keyword arguments of different types.

Motivation

Currently annotating**kwargswith a typeTmeans that thekwargs type is in factdict[str,T].For example:

deffoo(**kwargs:str)->None:...

means that all keyword arguments infooare strings (i.e.,kwargsis of typedict[str,str]). This behaviour limits the ability to type annotate**kwargsonly to the cases where all of them are of the same type. However, it is often the case that keyword arguments conveyed by**kwargs have different types that are dependent on the keyword’s name. In those cases type annotating**kwargsis not possible. This is especially a problem for already existing codebases where the need of refactoring the code in order to introduce proper type annotations may be considered not worth the effort. This in turn prevents the project from getting all of the benefits that type hinting can provide.

Moreover,**kwargscan be used to reduce the amount of code needed in cases when there is a top-level function that is a part of a public API and it calls a bunch of helper functions, all of which expect the same keyword arguments. Unfortunately, if those helper functions were to use**kwargs, there is no way to properly type hint them if the keyword arguments they expect are of different types. In addition, even if the keyword arguments are of the same type, there is no way to check whether the function is being called with keyword names that it actually expects.

As described in theIntended Usagesection, using**kwargsis not always the best tool for the job. Despite that, it is still a widely used pattern. As a consequence, there has been a lot of discussion around supporting more precise**kwargstyping and it became a feature that would be valuable for a large part of the Python community. This is best illustrated by themypy GitHub issue 4441which contains a lot of real world cases that could benefit from this propsal.

One more use case worth mentioning for which**kwargsare also convenient, is when a function should accommodate optional keyword-only arguments that don’t have default values. A need for a pattern like that can arise when values that are usually used as defaults to indicate no user input, such asNone, can be passed in by a user and should result in a valid, non-default behavior. For example, this issuecame upin the popularhttpxlibrary.

Rationale

PEP 589introduced theTypedDicttype constructor that supports dictionary types consisting of string keys and values of potentially different types. A function’s keyword arguments represented by a formal parameter that begins with double asterisk, such as**kwargs,are received as a dictionary. Additionally, such functions are often called using unpacked dictionaries to provide keyword arguments. This makesTypedDicta perfect candidate to be used for more precise**kwargstyping. In addition, withTypedDict keyword names can be taken into account during static type analysis. However, specifying**kwargstype with aTypedDictmeans, as mentioned earlier, that each keyword argument specified by**kwargsis aTypedDictitself. For instance:

classMovie(TypedDict):
name:str
year:int

deffoo(**kwargs:Movie)->None:...

means that each keyword argument infoois itself aMoviedictionary that has anamekey with a string type value and ayearkey with an integer type value. Therefore, in order to support specifyingkwargstype as aTypedDictwithout breaking current behaviour, a new construct has to be introduced.

To support this use case, we propose reusingUnpackwhich was initially introduced inPEP 646.There are several reasons for doing so:

• Its name is quite suitable and intuitive for the**kwargstyping use case as our intention is to “unpack” the keywords arguments from the supplied TypedDict.
• The current way of typing*argswould be extended to**kwargs and those are supposed to behave similarly.
• There would be no need to introduce any new special forms.
• The use ofUnpackfor the purposes described in this PEP does not interfere with the use cases described inPEP 646.

Specification

WithUnpackwe introduce a new way of annotating**kwargs. Continuing the previous example:

deffoo(**kwargs:Unpack[Movie])->None:...

would mean that the**kwargscomprise two keyword arguments specified by Movie(i.e. anamekeyword of typestrand ayearkeyword of typeint). This indicates that the function should be called as follows:

kwargs:Movie={"name":"Life of Brian","year":1979}

foo(**kwargs)# OK!
foo(name="The Meaning of Life",year=1983)# OK!

WhenUnpackis used, type checkers treatkwargsinside the function body as aTypedDict:

deffoo(**kwargs:Unpack[Movie])->None:
assert_type(kwargs,Movie)# OK!

Using the new annotation will not have any runtime effect - it should only be taken into account by type checkers. Any mention of errors in the following sections relates to type checker errors.

deffoo(**kwargs:Unpack[Movie])->None:...

movie:dict[str,object]={"name":"Life of Brian","year":1979}
foo(**movie)# WRONG! Movie is of type dict[str, object]

typed_movie:Movie={"name":"The Meaning of Life","year":1983}
foo(**typed_movie)# OK!

another_movie={"name":"Life of Brian","year":1979}
foo(**another_movie)# Depends on the type checker.

deffoo(name,**kwargs:Unpack[Movie])->None:...# WRONG! "name" will
# always bind to the
# first parameter.

deffoo(name,/,**kwargs:Unpack[Movie])->None:...# OK! "name" is a
# positional-only parameter,
# so **kwargs can contain
# a "name" keyword.

classMovie(TypedDict):
title:str
year:NotRequired[int]

classAnimal(TypedDict):
name:str

classDog(Animal):
breed:str

defaccept_animal(**kwargs:Unpack[Animal]):...
defaccept_dog(**kwargs:Unpack[Dog]):...

accept_dog=accept_animal# OK! Expression of type Dog can be
# assigned to a variable of type Animal.

accept_animal=accept_dog# WRONG! Expression of type Animal
# cannot be assigned to a variable of type Dog.

classExample(TypedDict):
animal:Animal
string:str
number:NotRequired[int]

defsrc(**kwargs:Unpack[Example]):...
defdest(*,animal:Dog,string:str,number:int=...):...

dest=src# OK!

defdest(dog:Dog,string:str,number:int=...):...

dog:Dog={"name":"Daisy","breed":"labrador"}

dest(dog,"some string")# OK!

dest=src# Type checker error!
dest(dog,"some string")# The same call fails at
# runtime now because 'src' expects
# keyword arguments.

defdest(**kwargs:Unpack[Animal]):...
defsrc(name:str):...

dog:Dog={"name":"Daisy","breed":"Labrador"}
animal:Animal=dog

dest=src# WRONG!
dest(**animal)# Fails at runtime.

defsrc(**kwargs):...
defdest(**kwargs:Unpack[Movie]):...

dest=src# OK!

classVehicle:
...

classCar(Vehicle):
...

classMotorcycle(Vehicle):
...

classVehicles(TypedDict):
car:Car
moto:Motorcycle

defdest(**kwargs:Unpack[Vehicles]):...
defsrc(**kwargs:Vehicle):...

dest=src# OK!

classAnimal(TypedDict):
name:str

classDog(Animal):
breed:str

deftakes_name(name:str):...

dog:Dog={"name":"Daisy","breed":"Labrador"}
animal:Animal=dog

deffoo(**kwargs:Unpack[Animal]):
print(kwargs["name"].capitalize())

defbar(**kwargs:Unpack[Animal]):
takes_name(**kwargs)

defbaz(animal:Animal):
takes_name(**animal)

defspam(**kwargs:Unpack[Animal]):
baz(kwargs)

foo(**animal)# OK! foo only expects and uses keywords of 'Animal'.

bar(**animal)# WRONG! This will fail at runtime because 'breed' keyword
# will be passed to 'takes_name' as well.

spam(**animal)# WRONG! Again, 'breed' keyword will be eventually passed
# to 'takes_name'.

defbar(**kwargs:Unpack[Animal]):
name=kwargs["name"]
takes_name(name)

>>>Unpack[Movie]
*<class '__main__.Movie'>

Intended Usage

The intended use cases for this proposal are described in the Motivationsection. In summary, more precise**kwargstyping can bring benefits to already existing codebases that decided to use **kwargsinitially, but now are mature enough to use a stricter contract via type hints. Using**kwargscan also help in reducing code duplication and the amount of copy-pasting needed when there is a bunch of functions that require the same set of keyword arguments. Finally,**kwargsare useful for cases when a function needs to facilitate optional keyword arguments that don’t have obvious default values.

However, it has to be pointed out that in some cases there are better tools for the job than usingTypedDictto type**kwargsas proposed in this PEP. For example, when writing new code if all the keyword arguments are required or have default values then writing everything explicitly is better than using**kwargsand aTypedDict:

deffoo(name:str,year:int):...# Preferred way.
deffoo(**kwargs:Unpack[Movie]):...

Similarly, when type hinting third party libraries via stubs it is again better to state the function signature explicitly - this is the only way to type such a function if it has default arguments. Another issue that may arise in this case when trying to type hint the function with aTypedDictis that some standard function parameters may be treated as keyword only:

deffoo(name,year):...# Function in a third party library.

deffoo(Unpack[Movie]):...# Function signature in a stub file.

foo("Life of Brian",1979)# This would be now failing type
# checking but is fine.

foo(name="Life of Brian",year=1979)# This would be the only way to call
# the function now that passes type
# checking.

Therefore, in this case it is again preferred to type hint such function explicitly as:

deffoo(name:str,year:int):...

Also, for the benefit of IDEs and documentation pages, functions that are part of the public API should prefer explicit keyword parameters whenever possible.

How to Teach This

This PEP could be linked in thetypingmodule’s documentation. Moreover, a new section on usingUnpackcould be added to the aforementioned docs. Similar sections could be also added to the mypy documentationand the typing RTD documentation.

Reference Implementation

Themypy type checkeralready supportsmore precise **kwargstyping usingUnpack.

Pyright type checkeralso provides provisional support forthis feature.

Rejected Ideas

classBook(TypedDict):
genre:str
pages:int

TypedDictUnion=Movie|Book

deffoo(**kwargs:Unpack[TypedDictUnion])->None:...# WRONG! Unsupported use
# of a union of
# TypedDicts to type
# **kwargs

@overload
deffoo(**kwargs:Unpack[Movie]):...

@overload
deffoo(**kwargs:Unpack[Book]):...

deffoo(**kwargs:**Movie):...

deffoo(**kwargs:Unpack[Movie]):...

Copyright

This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.