Add API to allow extensions to set callback function on creation and destruction of PyCodeObject #91054

mpage · 2022-03-01T22:20:59Z

BPO	46898
Nosy	@carljm, @DinoV, @itamaro, @mpage

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2022-03-01.22:20:58.520>
labels = ['expert-C-API', 'type-feature', '3.11']
title = 'Add API to allow extensions to set callback function on creation and destruction of PyCodeObject'
updated_at = <Date 2022-03-01.22:20:58.520>
user = 'https://github.com/mpage'

bugs.python.org fields:

activity = <Date 2022-03-01.22:20:58.520>
actor = 'mpage'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['C API']
creation = <Date 2022-03-01.22:20:58.520>
creator = 'mpage'
dependencies = []
files = []
hgrepos = []
issue_num = 46898
keywords = []
message_count = 1.0
messages = ['414309']
nosy_count = 4.0
nosy_names = ['carljm', 'dino.viehland', 'itamaro', 'mpage']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue46898'
versions = ['Python 3.11']

Linked PRs

mpage · 2022-03-01T22:20:58Z

CPython extensions providing optimized execution of Python bytecode (e.g. the Cinder JIT) may need to hook into the lifecycle of code objects to determine what to optimize or to free resources allocated for code objects that no longer exist. We propose adding an API to allow extensions to set callbacks that will be invoked whenever code objects are created or destroyed.

Proposed API:

typedef enum {
  PYCODE_LCEVT_CREATED,
  PYCODE_LCEVT_DESTROYED
} PyCode_LifecycleEvent;

// A callback to be called when a code object is created or about to be destroyed.
typedef void(*PyCode_LifecycleCallback)(
  PyCode_LifecycleEvent event,
  PyCodeObject* code);

void PyCode_SetLifecycleCallback(PyCode_LifecycleCallback callback);
PyCode_LifecycleCallback PyCode_GetLifecycleCallback();

gvanrossum · 2022-09-15T03:41:33Z

In which header file would you put that? Maybe this is simple enough that you should just submit a PR?

itamaro · 2022-09-15T04:20:23Z

In which header file would you put that?

would Include/cpython/code.h make sense for this API?

Maybe this is simple enough that you should just submit a PR?

hopefully yes, we intend to do that for 3.12.
does the proposed API look reasonable?

gvanrossum · 2022-09-24T03:38:55Z

The API looks reasonable -- I'd spell LifeCycle with two capital letters (though Wikipedia seems divided on this), and I'd start the enum values with PY_CODE, similar to _PyCodeLocationInfoKind in the same file. The implementation would seem straightforward, and code objects aren't created or destroyed at a monstrous rate (it might be worth seeing if this slows down unmarshalling noticeably, but I doubt it). The callback should probably be stored in the per-interpreter state.

…on creation and destruction of PyCodeObject

…on creation and destruction of PyCodeObject Co-authored-by: Ye11ow-Flash <[email protected]>

* Add API to allow extensions to set callback function on creation and destruction of PyCodeObject Co-authored-by: Ye11ow-Flash <[email protected]>

This reverts commit 3c137dc.

pablogsal · 2022-12-03T20:57:46Z

Unfotunately, there are many buildbots failing to build this PR. Check buildbot.python.org/all/#/release_status for reference

As per our buildbot policy, this PR will be reverted if the buildbot fleet is not fixed in the next 24h. @markshannon @itamaro can you please take a look?

I have opened #99976. I will close it if the issue is fixed before the deadline.

pythonGH-99859 introduced new buildbot failures, as reported [here](python#91054 (comment)). I was able to reproduce the failures with: ``` ./python.exe -m test -v test_capi.test_watchers -m "*TestCodeObjectWatchers*" -R 3:3 ``` The root cause appears to be to static events counters used in the tests, when running the tests with repetitions (using the same interpreter state), the counts from the first test run affected the next runs. This fixes it by resetting the counts when adding and clearing test watchers.

itamaro · 2022-12-04T22:22:52Z

this issue can be closed now!

…ythonGH-107821) (cherry picked from commit 2ec16fe) Co-authored-by: Carl Meyer <[email protected]>

…H-107821) (#107835) gh-91054: make code watcher tests resilient to other watchers (GH-107821) (cherry picked from commit 2ec16fe) Co-authored-by: Carl Meyer <[email protected]>

mpage mannequin added 3.11 only security fixes topic-C-API type-feature A feature request or enhancement labels Mar 1, 2022

ezio-melotti transferred this issue from another repository Apr 10, 2022

itamaro mentioned this issue Sep 15, 2022

Allow extensions to set a callback to be invoked when a type is modified #91051

Closed

gvanrossum added 3.12 bugs and security fixes and removed 3.11 only security fixes labels Sep 24, 2022

itamaro added a commit to itamaro/cpython that referenced this issue Nov 23, 2022

pythongh-91054: Add API to allow extensions to set callback function …

f30e512

…on creation and destruction of PyCodeObject

itamaro added a commit to itamaro/cpython that referenced this issue Nov 26, 2022

pythongh-91054: Add API to allow extensions to set callback function …

ccf51d6

…on creation and destruction of PyCodeObject

itamaro added a commit to itamaro/cpython that referenced this issue Nov 27, 2022

pythongh-91054: Add API to allow extensions to set callback function …

3b871f8

…on creation and destruction of PyCodeObject

itamaro added a commit to itamaro/cpython that referenced this issue Nov 28, 2022

pythongh-91054: Add API to allow extensions to set callback function …

2475ce4

…on creation and destruction of PyCodeObject Co-authored-by: Ye11ow-Flash <[email protected]>

itamaro mentioned this issue Nov 28, 2022

gh-91054: Add code object watchers API #99859

Merged

markshannon pushed a commit that referenced this issue Dec 2, 2022

GH-91054: Add code object watchers API (GH-99859)

3c137dc

* Add API to allow extensions to set callback function on creation and destruction of PyCodeObject Co-authored-by: Ye11ow-Flash <[email protected]>

bedevere-bot mentioned this issue Dec 3, 2022

gh-91054: Revert "gh-91054: Add code object watchers API" #99976

Closed

pablogsal added a commit that referenced this issue Dec 3, 2022

Revert "GH-91054: Add code object watchers API (GH-99859)"

9cf2109

This reverts commit 3c137dc.

itamaro mentioned this issue Dec 3, 2022

GH-91054: Reset static events counts in code watchers tests #99978

Merged

pablogsal pushed a commit that referenced this issue Dec 4, 2022

GH-91054: Reset static events counts in code watchers tests (#99978)

1012dc1

pablogsal closed this as completed Dec 4, 2022

carljm added a commit to carljm/cpython that referenced this issue Aug 9, 2023

pythongh-91054: make code watcher tests resilient to other watchers

f32672c

bedevere-bot mentioned this issue Aug 9, 2023

gh-91054: make code watcher tests resilient to other watchers #107821

Merged

carljm added a commit to carljm/cpython that referenced this issue Aug 9, 2023

pythongh-91054: make code watcher tests resilient to other watchers

b40b696

carljm added a commit that referenced this issue Aug 9, 2023

gh-91054: make code watcher tests resilient to other watchers (#107821)

2ec16fe

bedevere-bot mentioned this issue Aug 10, 2023

[3.12] gh-91054: make code watcher tests resilient to other watchers (GH-107821) #107835

Merged

miss-islington pushed a commit to miss-islington/cpython that referenced this issue Aug 10, 2023

pythongh-91054: make code watcher tests resilient to other watchers (p…

f53bd4d

…ythonGH-107821) (cherry picked from commit 2ec16fe) Co-authored-by: Carl Meyer <[email protected]>

fried mentioned this issue May 21, 2024

Add c-api to set callback function on enter and exit of PyContext #119333

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add API to allow extensions to set callback function on creation and destruction of PyCodeObject #91054

Add API to allow extensions to set callback function on creation and destruction of PyCodeObject #91054

mpage mannequin commented Mar 1, 2022 •

edited by bedevere-bot

Loading

mpage mannequin commented Mar 1, 2022

gvanrossum commented Sep 15, 2022

itamaro commented Sep 15, 2022

gvanrossum commented Sep 24, 2022

pablogsal commented Dec 3, 2022

itamaro commented Dec 4, 2022

Add API to allow extensions to set callback function on creation and destruction of PyCodeObject #91054

Add API to allow extensions to set callback function on creation and destruction of PyCodeObject #91054

Comments

mpage mannequin commented Mar 1, 2022 • edited by bedevere-bot Loading

Linked PRs

mpage mannequin commented Mar 1, 2022

gvanrossum commented Sep 15, 2022

itamaro commented Sep 15, 2022

gvanrossum commented Sep 24, 2022

pablogsal commented Dec 3, 2022

itamaro commented Dec 4, 2022

mpage mannequin commented Mar 1, 2022 •

edited by bedevere-bot

Loading