Acknowledgements

Talin wrote the Rationale below[1]as well as most of the section on ABCs vs. Interfaces. For that alone he deserves co-authorship. The rest of the PEP uses “I” referring to the first author.

Overloadingisinstance()andissubclass()

During the development of this PEP and of its companion,PEP 3141,we repeatedly faced the choice between standardizing more, fine-grained ABCs or fewer, coarse-grained ones. For example, at one stage, PEP 3141 introduced the following stack of base classes used for complex numbers: MonoidUnderPlus, AdditiveGroup, Ring, Field, Complex (each derived from the previous). And the discussion mentioned several other algebraic categorizations that were left out: Algebraic, Transcendental, and IntegralDomain, and PrincipalIdealDomain. In earlier versions of the current PEP, we considered the use cases for separate classes like Set, ComposableSet, MutableSet, HashableSet, MutableComposableSet, HashableComposableSet.

The dilemma here is that we’d rather have fewer ABCs, but then what should a user do who needs a less refined ABC? Consider e.g. the plight of a mathematician who wants to define their own kind of Transcendental numbers, but also wants float and int to be considered Transcendental.PEP 3141originally proposed to patch float.__bases__ for that purpose, but there are some good reasons to keep the built-in types immutable (for one, they are shared between all Python interpreters running in the same address space, as is used by mod_ Python[16]).

Another example would be someone who wants to define a generic function (PEP 3124) for any sequence that has anappend()method. TheSequenceABC (see below) doesn’t promise theappend() method, whileMutableSequencerequires not onlyappend()but also various other mutating methods.

To solve these and similar dilemmas, the next section will propose a metaclass for use with ABCs that will allow us to add an ABC as a “virtual base class” (not the same concept as in C++) to any class, including to another ABC. This allows the standard library to define ABCsSequenceandMutableSequenceand register these as virtual base classes for built-in types likebasestring,tuple andlist,so that for example the following conditions are all true:

isinstance([],Sequence)
issubclass(list,Sequence)
issubclass(list,MutableSequence)
isinstance((),Sequence)
notissubclass(tuple,MutableSequence)
isinstance("",Sequence)
issubclass(bytearray,MutableSequence)

The primary mechanism proposed here is to allow overloading the built-in functionsisinstance()andissubclass().The overloading works as follows: The callisinstance(x,C)first checks whetherC.__instancecheck__exists, and if so, calls C.__instancecheck__(x)instead of its normal implementation. Similarly, the callissubclass(D,C)first checks whether C.__subclasscheck__exists, and if so, calls C.__subclasscheck__(D)instead of its normal implementation.

Note that the magic names are not__isinstance__and __issubclass__;this is because the reversal of the arguments could cause confusion, especially for theissubclass()overloader.

A prototype implementation of this is given in[12].

Here is an example with (naively simple) implementations of __instancecheck__and__subclasscheck__:

classABCMeta(type):

def__instancecheck__(cls,inst):
"""Implement isinstance(inst, cls)." ""
returnany(cls.__subclasscheck__(c)
forcin{type(inst),inst.__class__})

def__subclasscheck__(cls,sub):
"""Implement issubclass(sub, cls)." ""
candidates=cls.__dict__.get("__subclass__",set())|{cls}
returnany(cincandidatesforcinsub.mro())

classSequence(metaclass=ABCMeta):
__subclass__={list,tuple}

assertissubclass(list,Sequence)
assertissubclass(tuple,Sequence)

classAppendableSequence(Sequence):
__subclass__={list}

assertissubclass(list,AppendableSequence)
assertisinstance([],AppendableSequence)

assertnotissubclass(tuple,AppendableSequence)
assertnotisinstance((),AppendableSequence)

The next section proposes a full-fledged implementation.

TheabcModule: an ABC Support Framework

The new standard library moduleabc,written in pure Python, serves as an ABC support framework. It defines a metaclass ABCMetaand decorators@abstractmethodand @abstractproperty.A sample implementation is given by[13].

TheABCMetaclass overrides__instancecheck__and __subclasscheck__and defines aregistermethod. The registermethod takes one argument, which must be a class; after the callB.register(C),the callissubclass(C,B)will return True, by virtue ofB.__subclasscheck__(C)returning True. Also,isinstance(x,B)is equivalent toissubclass(x.__class__, B)orissubclass(type(x),B).(It is possibletype(x)and x.__class__are not the same object, e.g. when x is a proxy object.)

These methods are intended to be called on classes whose metaclass is (derived from)ABCMeta;for example:

fromabcimportABCMeta

classMyABC(metaclass=ABCMeta):
pass

MyABC.register(tuple)

assertissubclass(tuple,MyABC)
assertisinstance((),MyABC)

The last two asserts are equivalent to the following two:

assertMyABC.__subclasscheck__(tuple)
assertMyABC.__instancecheck__(())

Of course, you can also directly subclass MyABC:

classMyClass(MyABC):
pass

assertissubclass(MyClass,MyABC)
assertisinstance(MyClass(),MyABC)

Also, of course, a tuple is not aMyClass:

assertnotissubclass(tuple,MyClass)
assertnotisinstance((),MyClass)

You can register another class as a subclass ofMyClass:

MyClass.register(list)

assertissubclass(list,MyClass)
assertissubclass(list,MyABC)

You can also register another ABC:

classAnotherClass(metaclass=ABCMeta):
pass

AnotherClass.register(basestring)

MyClass.register(AnotherClass)

assertisinstance(str,MyABC)

That last assert requires tracing the following superclass-subclass relationships:

MyABC->MyClass(usingregularsubclassing)
MyClass->AnotherClass(usingregistration)
AnotherClass->basestring(usingregistration)
basestring->str(usingregularsubclassing)

Theabcmodule also defines a new decorator,@abstractmethod, to be used to declare abstract methods. A class containing at least one method declared with this decorator that hasn’t been overridden yet cannot be instantiated. Such methods may be called from the overriding method in the subclass (usingsuperor direct invocation). For example:

fromabcimportABCMeta,abstractmethod

classA(metaclass=ABCMeta):
@abstractmethod
deffoo(self):pass

A()# raises TypeError

classB(A):
pass

B()# raises TypeError

classC(A):
deffoo(self):print(42)

C()# works

Note:The@abstractmethoddecorator should only be used inside a class body, and only for classes whose metaclass is (derived from)ABCMeta.Dynamically adding abstract methods to a class, or attempting to modify the abstraction status of a method or class once it is created, are not supported. The@abstractmethodonly affects subclasses derived using regular inheritance; “virtual subclasses” registered with theregister()method are not affected.

Implementation:The@abstractmethoddecorator sets the function attribute__isabstractmethod__to the valueTrue. TheABCMeta.__new__method computes the type attribute __abstractmethods__as the set of all method names that have an __isabstractmethod__attribute whose value is true. It does this by combining the__abstractmethods__attributes of the base classes, adding the names of all methods in the new class dict that have a true__isabstractmethod__attribute, and removing the names of all methods in the new class dict that don’t have a true __isabstractmethod__attribute. If the resulting __abstractmethods__set is non-empty, the class is considered abstract, and attempts to instantiate it will raiseTypeError. (If this were implemented in CPython, an internal flag Py_TPFLAGS_ABSTRACTcould be used to speed up this check[6].)

Discussion:Unlike Java’s abstract methods or C++’s pure abstract methods, abstract methods as defined here may have an implementation. This implementation can be called via thesupermechanism from the class that overrides it. This could be useful as an end-point for a super-call in framework using cooperative multiple-inheritance[7], [8].

A second decorator,@abstractproperty,is defined in order to define abstract data attributes. Its implementation is a subclass of the built-inpropertyclass that adds an__isabstractmethod__ attribute:

classabstractproperty(property):
__isabstractmethod__=True

It can be used in two ways:

classC(metaclass=ABCMeta):

# A read-only property:

@abstractproperty
defreadonly(self):
returnself.__x

# A read-write property (cannot use decorator syntax):

defgetx(self):
returnself.__x
defsetx(self,value):
self.__x=value
x=abstractproperty(getx,setx)

Similar to abstract methods, a subclass inheriting an abstract property (declared using either the decorator syntax or the longer form) cannot be instantiated unless it overrides that abstract property with a concrete property.

ABCs for Containers and Iterators

Thecollectionsmodule will define ABCs necessary and sufficient to work with sets, mappings, sequences, and some helper types such as iterators and dictionary views. All ABCs have the above-mentioned ABCMetaas their metaclass.

The ABCs provide implementations of their abstract methods that are technically valid but fairly useless; e.g.__hash__returns 0, and __iter__returns an empty iterator. In general, the abstract methods represent the behavior of an empty container of the indicated type.

Some ABCs also provide concrete (i.e. non-abstract) methods; for example, theIteratorclass has an__iter__method returning itself, fulfilling an important invariant of iterators (which in Python 2 has to be implemented anew by each iterator class). These ABCs can be considered “mix-in” classes.

No ABCs defined in the PEP override__init__,__new__, __str__or__repr__.Defining a standard constructor signature would unnecessarily constrain custom container types, for example Patricia trees or gdbm files. Defining a specific string representation for a collection is similarly left up to individual implementations.

Note:There are no ABCs for ordering operations (__lt__, __le__,__ge__,__gt__). Defining these in a base class (abstract or not) runs into problems with the accepted type for the second operand. For example, if classOrderingdefined __lt__,one would assume that for anyOrderinginstancesx andy,x<ywould be defined (even if it just defines a partial ordering). But this cannot be the case: If bothlistand strderived fromOrdering,this would imply that[1,2]< (1,2)should be defined (and presumably return False), while in fact (in Python 3000!) such “mixed-mode comparisons” operations are explicitly forbidden and raiseTypeError.SeePEP 3100and[14] for more information. (This is a special case of a more general issue with operations that take another argument of the same type).

classSized(metaclass=ABCMeta):
@abstractmethod
def__hash__(self):
return0
@classmethod
def__instancecheck__(cls,x):
returnhasattr(x,"__len__")
@classmethod
def__subclasscheck__(cls,C):
returnhasattr(C,"__bases__")andhasattr(C,"__len__")

Strings

Python 3000 will likely have at least two built-in string types: byte strings (bytes), deriving fromMutableSequence,and (Unicode) character strings (str), deriving fromSequenceand Hashable.

Open issues:define the base interfaces for these so alternative implementations and subclasses know what they are in for. This may be the subject of a new PEP or PEPs (PEP 358should be co-opted for the bytestype).

ABCs vs. Duck Typing

Does the introduction of ABCs mean the end of Duck Typing? I don’t think so. Python will not require that a class derives from BasicMappingorSequencewhen it defines a__getitem__ method, nor will thex[y]syntax require thatxis an instance of either ABC. You will still be able to assign any “file-like” object tosys.stdout,as long as it has awritemethod.

Of course, there will be some carrots to encourage users to derive from the appropriate base classes; these vary from default implementations for certain functionality to an improved ability to distinguish between mappings and sequences. But there are no sticks. Ifhasattr(x,"__len__" )works for you, great! ABCs are intended to solve problems that don’t have a good solution at all in Python 2, such as distinguishing between mappings and sequences.

ABCs vs. Generic Functions

ABCs are compatible with Generic Functions (GFs). For example, my own Generic Functions implementation[4]uses the classes (types) of the arguments as the dispatch key, allowing derived classes to override base classes. Since (from Python’s perspective) ABCs are quite ordinary classes, using an ABC in the default implementation for a GF can be quite appropriate. For example, if I have an overloaded prettyprintfunction, it would make total sense to define pretty-printing of sets like this:

@prettyprint.register(Set)
defpp_set(s):
return"{"+...+"}"# Details left as an exercise

and implementations for specific subclasses of Set could be added easily.

I believe ABCs also won’t present any problems for RuleDispatch, Phillip Eby’s GF implementation in PEAK[5].

Of course, GF proponents might claim that GFs (and concrete, or implementation, classes) are all you need. But even they will not deny the usefulness of inheritance; and one can easily consider the ABCs proposed in this PEP as optional implementation base classes; there is no requirement that all user-defined mappings derive from BasicMapping.

ABCs vs. Interfaces

ABCs are not intrinsically incompatible with Interfaces, but there is considerable overlap. For now, I’ll leave it to proponents of Interfaces to explain why Interfaces are better. I expect that much of the work that went into e.g. defining the various shades of “mapping-ness” and the nomenclature could easily be adapted for a proposal to use Interfaces instead of ABCs.

“Interfaces” in this context refers to a set of proposals for additional metadata elements attached to a class which are not part of the regular class hierarchy, but do allow for certain types of inheritance testing.

Such metadata would be designed, at least in some proposals, so as to be easily mutable by an application, allowing application writers to override the normal classification of an object.

The drawback to this idea of attaching mutable metadata to a class is that classes are shared state, and mutating them may lead to conflicts of intent. Additionally, the need to override the classification of an object can be done more cleanly using generic functions: In the simplest case, one can define a “category membership” generic function that simply returns False in the base implementation, and then provide overrides that return True for any classes of interest.