PEP 367 – New Super

Author:: Calvin Spealman <ironfroggy at gmail >, Tim Delaney <timothy.c.delaney at gmail >
Status:: Superseded
Type:: Standards Track
Created:: 28-Apr-2007
Python-Version:: 2.6
Post-History:: 28-Apr-2007, 29-Apr-2007, 29-Apr-2007, 14-May-2007

Table of Contents

Numbering Note

This PEP has been renumbered toPEP 3135.The text below is the last version submitted under the old number.

This PEP proposes syntactic sugar for use of thesupertype to automatically construct instances of the super type binding to the class that a method was defined in, and the instance (or class object for classmethods) that the method is currently acting upon.

The premise of the new super usage suggested is as follows:

super.foo(1,2)

to replace the old:

super(Foo,self).foo(1,2)

and the current__builtin__.superbe aliased to__builtin__.__super__ (with__builtin__.superto be removed in Python 3.0).

It is further proposed that assignment tosuperbecome aSyntaxError, similar to the behaviour ofNone.

Rationale

The current usage of super requires an explicit passing of both the class and instance it must operate from, requiring a breaking of the DRY (Don’t Repeat Yourself) rule. This hinders any change in class name, and is often considered a wart by many.

Specification

Within the specification section, some special terminology will be used to distinguish similar and closely related concepts. “super type” will refer to the actual builtin type named “super”. A “super instance” is simply an instance of the super type, which is associated with a class and possibly with an instance of that class.

Because the newsupersemantics are not backwards compatible with Python 2.5, the new semantics will require a__future__import:

from__future__importnew_super

The current__builtin__.superwill be aliased to__builtin__.__super__. This will occur regardless of whether the newsupersemantics are active. It is not possible to simply rename__builtin__.super,as that would affect modules that do not use the newsupersemantics. In Python 3.0 it is proposed that the name__builtin__.superwill be removed.

Replacing the old usage of super, calls to the next class in the MRO (method resolution order) can be made without explicitly creating asuper instance (although doing so will still be supported via__super__). Every function will have an implicit local namedsuper.This name behaves identically to a normal local, including use by inner functions via a cell, with the following exceptions:

Assigning to the namesuperwill raise aSyntaxErrorat compile time;
Calling a static method or normal function that accesses the namesuper will raise aTypeErrorat runtime.

Every function that uses the namesuper,or has an inner function that uses the namesuper,will include a preamble that performs the equivalent of:

super=__builtin__.__super__(<class>,<instance>)

where<class>is the class that the method was defined in, and <instance>is the first parameter of the method (normallyselffor instance methods, andclsfor class methods). For static methods and normal functions,<class>will beNone,resulting in aTypeErrorbeing raised during the preamble.

Note: The relationship betweensuperand__super__is similar to that betweenimportand__import__.

Much of this was discussed in the thread of the Python -dev list, “Fi xing super anyone?”[1].

Open Issues

Determining the class object to use

The exact mechanism for associating the method with the defining class is not specified in this PEP, and should be chosen for maximum performance. For CPython, it is suggested that the class instance be held in a C-level variable on the function object which is bound to one ofNULL(not part of a class), Py_None(static method) or a class object (instance or class method).

Should`super`actually become a keyword?

With this proposal,superwould become a keyword to the same extent that Noneis a keyword. It is possible that further restricting thesuper name may simplify implementation, however some are against the actual keyword-ization of super. The simplest solution is often the correct solution and the simplest solution may well not be adding additional keywords to the language when they are not needed. Still, it may solve other open issues.

Closed Issues

super used with call attributes

It was considered that it might be a problem that instantiating super instances the classic way, because calling it would lookup the __call__ attribute and thus try to perform an automatic super lookup to the next class in the MRO. However, this was found to be false, because calling an object only looks up the __call__ method directly on the object’s type. The following example shows this in action.

classA(object):
def__call__(self):
return'__call__'
def__getattribute__(self,attr):
ifattr=='__call__':
returnlambda:'__getattribute__'
a=A()
asserta()=='__call__'
asserta.__call__()=='__getattribute__'

In any case, with the renaming of__builtin__.superto __builtin__.__super__this issue goes away entirely.

Reference Implementation

It is impossible to implement the above specification entirely in Python. This reference implementation has the following differences to the specification:

Newsupersemantics are implemented using bytecode hacking.
Assignment tosuperis not aSyntaxError.Also see point #4.
Classes must either use the metaclassautosuper_metaor inherit from the base classautosuperto acquire the newsupersemantics.
superis not an implicit local variable. In particular, for inner functions to be able to use the super instance, there must be an assignment of the formsuper=superin the method.

The reference implementation assumes that it is being run on Python 2.5+.

#!/usr/bin/env Python
#
# autosuper.py

fromarrayimportarray
importdis
importnew
importtypes
import__builtin__
__builtin__.__super__=__builtin__.super
del__builtin__.super

# We need these for modifying bytecode
fromopcodeimportopmap,HAVE_ARGUMENT,EXTENDED_ARG

LOAD_GLOBAL=opmap['LOAD_GLOBAL']
LOAD_NAME=opmap['LOAD_NAME']
LOAD_CONST=opmap['LOAD_CONST']
LOAD_FAST=opmap['LOAD_FAST']
LOAD_ATTR=opmap['LOAD_ATTR']
STORE_FAST=opmap['STORE_FAST']
LOAD_DEREF=opmap['LOAD_DEREF']
STORE_DEREF=opmap['STORE_DEREF']
CALL_FUNCTION=opmap['CALL_FUNCTION']
STORE_GLOBAL=opmap['STORE_GLOBAL']
DUP_TOP=opmap['DUP_TOP']
POP_TOP=opmap['POP_TOP']
NOP=opmap['NOP']
JUMP_FORWARD=opmap['JUMP_FORWARD']
ABSOLUTE_TARGET=dis.hasjabs

def_oparg(code,opcode_pos):
returncode[opcode_pos+1]+(code[opcode_pos+2]<<8)

def_bind_autosuper(func,cls):
co=func.func_code
name=func.func_name
newcode=array('B',co.co_code)
codelen=len(newcode)
newconsts=list(co.co_consts)
newvarnames=list(co.co_varnames)

# Check if the global 'super' keyword is already present
try:
sn_pos=list(co.co_names).index('super')
exceptValueError:
sn_pos=None

# Check if the varname 'super' keyword is already present
try:
sv_pos=newvarnames.index('super')
exceptValueError:
sv_pos=None

# Check if the cellvar 'super' keyword is already present
try:
sc_pos=list(co.co_cellvars).index('super')
exceptValueError:
sc_pos=None

# If 'super' isn't used anywhere in the function, we don't have anything to do
ifsn_posisNoneandsv_posisNoneandsc_posisNone:
returnfunc

c_pos=None
s_pos=None
n_pos=None

# Check if the 'cls_name' and 'super' objects are already in the constants
forpos,oinenumerate(newconsts):
ifoiscls:
c_pos=pos

ifois__super__:
s_pos=pos

ifo==name:
n_pos=pos

# Add in any missing objects to constants and varnames
ifc_posisNone:
c_pos=len(newconsts)
newconsts.append(cls)

ifn_posisNone:
n_pos=len(newconsts)
newconsts.append(name)

ifs_posisNone:
s_pos=len(newconsts)
newconsts.append(__super__)

ifsv_posisNone:
sv_pos=len(newvarnames)
newvarnames.append('super')

# This goes at the start of the function. It is:
#
# super = __super__(cls, self)
#
# If 'super' is a cell variable, we store to both the
# local and cell variables (i.e. STORE_FAST and STORE_DEREF).
#
preamble=[
LOAD_CONST,s_pos&0xFF,s_pos>>8,
LOAD_CONST,c_pos&0xFF,c_pos>>8,
LOAD_FAST,0,0,
CALL_FUNCTION,2,0,
]

ifsc_posisNone:
# 'super' is not a cell variable - we can just use the local variable
preamble+=[
STORE_FAST,sv_pos&0xFF,sv_pos>>8,
]
else:
# If 'super' is a cell variable, we need to handle LOAD_DEREF.
preamble+=[
DUP_TOP,
STORE_FAST,sv_pos&0xFF,sv_pos>>8,
STORE_DEREF,sc_pos&0xFF,sc_pos>>8,
]

preamble=array('B',preamble)

# Bytecode for loading the local 'super' variable.
load_super=array('B',[
LOAD_FAST,sv_pos&0xFF,sv_pos>>8,
])

preamble_len=len(preamble)
need_preamble=False
i=0

whilei<codelen:
opcode=newcode[i]
need_load=False
remove_store=False

ifopcode==EXTENDED_ARG:
raiseTypeError("Cannot use 'super' in function with EXTENDED_ARG opcode")

# If the opcode is an absolute target it needs to be adjusted
# to take into account the preamble.
elifopcodeinABSOLUTE_TARGET:
oparg=_oparg(newcode,i)+preamble_len
newcode[i+1]=oparg&0xFF
newcode[i+2]=oparg>>8

# If LOAD_GLOBAL(super) or LOAD_NAME(super) then we want to change it into
# LOAD_FAST(super)
elif(opcode==LOAD_GLOBALoropcode==LOAD_NAME)and_oparg(newcode,i)==sn_pos:
need_preamble=need_load=True

# If LOAD_FAST(super) then we just need to add the preamble
elifopcode==LOAD_FASTand_oparg(newcode,i)==sv_pos:
need_preamble=need_load=True

# If LOAD_DEREF(super) then we change it into LOAD_FAST(super) because
# it's slightly faster.
elifopcode==LOAD_DEREFand_oparg(newcode,i)==sc_pos:
need_preamble=need_load=True

ifneed_load:
newcode[i:i+3]=load_super

i+=1

ifopcode>=HAVE_ARGUMENT:
i+=2

# No changes needed - get out.
ifnotneed_preamble:
returnfunc

# Our preamble will have 3 things on the stack
co_stacksize=max(3,co.co_stacksize)

# Conceptually, our preamble is on the `def` line.
co_lnotab=array('B',co.co_lnotab)

ifco_lnotab:
co_lnotab[0]+=preamble_len

co_lnotab=co_lnotab.tostring()

# Our code consists of the preamble and the modified code.
codestr=(preamble+newcode).tostring()

codeobj=new.code(co.co_argcount,len(newvarnames),co_stacksize,
co.co_flags,codestr,tuple(newconsts),co.co_names,
tuple(newvarnames),co.co_filename,co.co_name,
co.co_firstlineno,co_lnotab,co.co_freevars,
co.co_cellvars)

func.func_code=codeobj
func.func_class=cls
returnfunc

classautosuper_meta(type):
def__init__(cls,name,bases,clsdict):
UnboundMethodType=types.UnboundMethodType

forvinvars(cls):
o=getattr(cls,v)
ifisinstance(o,UnboundMethodType):
_bind_autosuper(o.im_func,cls)

classautosuper(object):
__metaclass__=autosuper_meta

if__name__=='__main__':
classA(autosuper):
deff(self):
return'A'

classB(A):
deff(self):
return'B'+super.f()

classC(A):
deff(self):
definner():
return'C'+super.f()

# Needed to put 'super' into a cell
super=super
returninner()

classD(B,C):
deff(self,arg=None):
var=None
return'D'+super.f()

assertD().f()=='DBCA'

Disassembly of B.f and C.f reveals the different preambles used whensuper is simply a local variable compared to when it is used by an inner function.

>>>dis.dis(B.f)

0 LOAD_CONST 4 (<type 'super'>)
LOAD_CONST 2 (<class '__main__.B'>)
LOAD_FAST 0 (self)
CALL_FUNCTION 2
STORE_FAST 1 (super)

15 LOAD_CONST 1 ('B')
LOAD_FAST 1 (super)
LOAD_ATTR 1 (f)
CALL_FUNCTION 0
BINARY_ADD
RETURN_VALUE

>>>dis.dis(C.f)

0 LOAD_CONST 4 (<type 'super'>)
LOAD_CONST 2 (<class '__main__.C'>)
LOAD_FAST 0 (self)
CALL_FUNCTION 2
DUP_TOP
STORE_FAST 1 (super)
STORE_DEREF 0 (super)

19 LOAD_CLOSURE 0 (super)
LOAD_CONST 1 (<code object inner at 00C160A0, file "autosuper.py", line 219>)
MAKE_CLOSURE 0
STORE_FAST 2 (inner)

31 LOAD_FAST 1 (super)
STORE_DEREF 0 (super)

37 LOAD_FAST 2 (inner)
CALL_FUNCTION 0
RETURN_VALUE

Note that in the final implementation, the preamble would not be part of the bytecode of the method, but would occur immediately following unpacking of parameters.

Alternative Proposals

No Changes

Although its always attractive to just keep things how they are, people have sought a change in the usage of super calling for some time, and for good reason, all mentioned previously.

Decoupling from the class name (which might not even be bound to the right class anymore!)
Simpler looking, cleaner super calls would be better

Dynamic attribute on super type

The proposal adds a dynamic attribute lookup to the super type, which will automatically determine the proper class and instance parameters. Each super attribute lookup identifies these parameters and performs the super lookup on the instance, as the current super implementation does with the explicit invocation of a super instance upon a class and instance.

This proposal relies on sys._getframe(), which is not appropriate for anything except a prototype implementation.

super(__this_class__, self)

This is nearly an anti-proposal, as it basically relies on the acceptance of the __this_class__ PEP, which proposes a special name that would always be bound to the class within which it is used. If that is accepted, __this_class__ could simply be used instead of the class’ name explicitly, solving the name binding issues[2].

self.super.foo(*args)

The __super__ attribute is mentioned in this PEP in several places, and could be a candidate for the complete solution, actually using it explicitly instead of any super usage directly. However, double-underscore names are usually an internal detail, and attempted to be kept out of everyday code.

super(self, args) or super(self, args)

This solution only solves the problem of the type indication, does not handle differently named super methods, and is explicit about the name of the instance. It is less flexible without being able to enacted on other method names, in cases where that is needed. One use case this fails is where a base-class has a factory classmethod and a subclass has two factory classmethods, both of which needing to properly make super calls to the one in the base-class.

super.foo(self, *args)

This variation actually eliminates the problems with locating the proper instance, and if any of the alternatives were pushed into the spotlight, I would want it to be this one.

super or super()

This proposal leaves no room for different names, signatures, or application to other classes, or instances. A way to allow some similar use alongside the normal proposal would be favorable, encouraging good design of multiple inheritance trees and compatible methods.

super(*p, **kw)

There has been the proposal that directly callingsuper(*p,**kw)would be equivalent to calling the method on thesuperobject with the same name as the method currently being executed i.e. the following two methods would be equivalent:

deff(self,*p,**kw):
super.f(*p,**kw)

deff(self,*p,**kw):
super(*p,**kw)

There is strong sentiment for and against this, but implementation and style concerns are obvious. Guido has suggested that this should be excluded from this PEP on the principle of KISS (Keep It Simple Stupid).

History

29-Apr-2007 - Changed title from “Super As A Keyword” to “New Super”

Updated much of the language and added a terminology section for clarification in confusing places.
Added reference implementation and history sections.

06-May-2007 - Updated by Tim Delaney to reflect discussions on the Python -3000

and Python -dev mailing lists.

References

Copyright

This document has been placed in the public domain.

Source:https://github / Python /peps/blob/main/peps/pep-0367.rst

Last modified:2023-09-09 17:39:29 GMT

Python Enhancement Proposals