Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python 3.11, and 3.12 versions bytecode support #3130

Open
1 of 3 tasks
XVilka opened this issue Oct 26, 2022 · 1 comment
Open
1 of 3 tasks

Python 3.11, and 3.12 versions bytecode support #3130

XVilka opened this issue Oct 26, 2022 · 1 comment
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed test-required

Comments

@XVilka
Copy link
Member

XVilka commented Oct 26, 2022

See also dis module from the corresponding Python versions: https://docs.python.org/3/library/dis.html

The changes are possibly required in these files:

3.10

CPython bytecode changes
The MAKE_FUNCTION instruction now accepts either a dict or a tuple of strings as the function’s annotations. (Contributed by Yurii Karabas and Inada Naoki in bpo-42202.)

3.11

CPython bytecode changes

The bytecode now contains inline cache entries, which take the form of the newly-added CACHE instructions. Many opcodes expect to be followed by an exact number of caches, and instruct the interpreter to skip over them at runtime. Populated caches can look like arbitrary instructions, so great care should be taken when reading or modifying raw, adaptive bytecode containing quickened data.

New opcodes

Replaced opcodes

Replaced Opcode(s) New Opcode(s) Notes
BINARY_INPLACE_ BINARY_OP Replaced all numeric binary/in-place opcodes with a single opcode
CALL_FUNCTIONCALL_FUNCTION_KWCALL_METHOD CALLKW_NAMESPRECALLPUSH_NULL Decouples argument shifting for methods from handling of keyword arguments; allows better specialization of calls
DUP_TOPDUP_TOP_TWOROT_TWOROT_THREEROT_FOURROT_N COPYSWAP Stack manipulation instructions
JUMP_IF_NOT_EXC_MATCH CHECK_EXC_MATCH Now performs check but doesn’t jump
JUMP_ABSOLUTEPOP_JUMP_IF_FALSEPOP_JUMP_IF_TRUE JUMP_BACKWARDPOP_JUMP_BACKWARD_IF_POP_JUMP_FORWARD_IF_ See 3; TRUE, FALSE, NONE and NOT_NONE variants for each direction
SETUP_WITHSETUP_ASYNC_WITH BEFORE_WITH with block setup
3

All jump opcodes are now relative, including the existing JUMP_IF_TRUE_OR_POP and JUMP_IF_FALSE_OR_POP. The argument is now an offset from the current instruction rather than an absolute location.

Changed/removed opcodes

  • Changed MATCH_CLASS and MATCH_KEYS to no longer push an additional boolean value to indicate success/failure. Instead, None is pushed on failure in place of the tuple of extracted values.

  • Changed opcodes that work with exceptions to reflect them now being represented as one item on the stack instead of three (see gh-89874).

  • Removed COPY_DICT_WITHOUT_KEYS, GEN_START, POP_BLOCK, SETUP_FINALLY and YIELD_FROM.

CPython bytecode changes The bytecode now contains inline cache entries, which take the form of the newly-added [CACHE](https://docs.python.org/3.11/library/dis.html#opcode-CACHE) instructions. Many opcodes expect to be followed by an exact number of caches, and instruct the interpreter to skip over them at runtime. Populated caches can look like arbitrary instructions, so great care should be taken when reading or modifying raw, adaptive bytecode containing quickened data.

New opcodes
ASYNC_GEN_WRAP, RETURN_GENERATOR and SEND, used in generators and co-routines.

COPY_FREE_VARS, which avoids needing special caller-side code for closures.

JUMP_BACKWARD_NO_INTERRUPT, for use in certain loops where handling interrupts is undesirable.

MAKE_CELL, to create Cell Objects.

CHECK_EG_MATCH and PREP_RERAISE_STAR, to handle the new exception groups and except* added in PEP 654.

PUSH_EXC_INFO, for use in exception handlers.

RESUME, a no-op, for internal tracing, debugging and optimization checks.

Replaced opcodes
Replaced Opcode(s)

New Opcode(s)

Notes

BINARY_*
INPLACE_*
BINARY_OP

Replaced all numeric binary/in-place opcodes with a single opcode

CALL_FUNCTION
CALL_FUNCTION_KW
CALL_METHOD
CALL
KW_NAMES
PRECALL
PUSH_NULL
Decouples argument shifting for methods from handling of keyword arguments; allows better specialization of calls

DUP_TOP
DUP_TOP_TWO
ROT_TWO
ROT_THREE
ROT_FOUR
ROT_N
COPY
SWAP
Stack manipulation instructions

JUMP_IF_NOT_EXC_MATCH
CHECK_EXC_MATCH
Now performs check but doesn’t jump

JUMP_ABSOLUTE
POP_JUMP_IF_FALSE
POP_JUMP_IF_TRUE
JUMP_BACKWARD
POP_JUMP_BACKWARD_IF_*
POP_JUMP_FORWARD_IF_*
See 3; TRUE, FALSE, NONE and NOT_NONE variants for each direction

SETUP_WITH
SETUP_ASYNC_WITH
BEFORE_WITH

with block setup

3
All jump opcodes are now relative, including the existing JUMP_IF_TRUE_OR_POP and JUMP_IF_FALSE_OR_POP. The argument is now an offset from the current instruction rather than an absolute location.

Changed/removed opcodes
Changed MATCH_CLASS and MATCH_KEYS to no longer push an additional boolean value to indicate success/failure. Instead, None is pushed on failure in place of the tuple of extracted values.

Changed opcodes that work with exceptions to reflect them now being represented as one item on the stack instead of three (see gh-89874).

Removed COPY_DICT_WITHOUT_KEYS, GEN_START, POP_BLOCK, SETUP_FINALLY and YIELD_FROM.

3.12

  • Removed the LOAD_METHOD instruction. It has been merged into LOAD_ATTR. LOAD_ATTR will now behave like the old LOAD_METHOD instruction if the low bit of its oparg is set. (Contributed by Ken Jin in gh-93429.)

  • Removed the JUMP_IF_FALSE_OR_POP and JUMP_IF_TRUE_OR_POP instructions. (Contributed by Irit Katriel in gh-102859.)

@XVilka XVilka added good first issue Good for newcomers help wanted Extra attention is needed test-required labels Oct 26, 2022
@XVilka XVilka added this to the 0.5.0 milestone Jan 3, 2023
@XVilka XVilka changed the title Python 3.11 version bytecode support Python 3.10 and 3.11 versions bytecode support Jan 3, 2023
@ret2libc ret2libc added the enhancement New feature or request label Jan 17, 2023
@ret2libc ret2libc removed this from the 0.5.0 milestone Jan 23, 2023
@XVilka XVilka changed the title Python 3.10 and 3.11 versions bytecode support Python 3.10, 3.11, and 3.12 versions bytecode support Apr 20, 2023
@borzacchiello borzacchiello mentioned this issue Jun 13, 2023
5 tasks
@XVilka XVilka changed the title Python 3.10, 3.11, and 3.12 versions bytecode support Python 3.11, and 3.12 versions bytecode support Jul 20, 2023
@blackjackal010
Copy link

hi, i would like to work on this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed test-required
Projects
None yet
Development

No branches or pull requests

3 participants