Skip to content
New issue

Have a question about this project?Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of serviceand privacy statement.We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add API to allow extensions to set callback function on creation and destruction of PyCodeObject #91054

Closed
mpagemannequinopened this issue Mar 1, 2022 · 6 comments
Labels
3.12 bugs and security fixes topic-C-API type-feature A feature request or enhancement

Comments

@mpage
Copy link
Mannequin

mpage mannequin commented Mar 1, 2022

BPO 46898
Nosy @carljm,@DinoV,@itamaro,@mpage

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee=None
closed_at=None
created_at=<Date2022-03-01.22:20:58.520>
labels=['expert-C-API','type-feature','3.11']
title='Add API to allow extensions to set callback function on creation and destruction of PyCodeObject'
updated_at=<Date2022-03-01.22:20:58.520>
user='https://github.com/mpage'

bugs.python.org fields:

activity=<Date2022-03-01.22:20:58.520>
actor='mpage'
assignee='none'
closed=False
closed_date=None
closer=None
components=['C API']
creation=<Date2022-03-01.22:20:58.520>
creator='mpage'
dependencies=[]
files=[]
hgrepos=[]
issue_num=46898
keywords=[]
message_count=1.0
messages=['414309']
nosy_count=4.0
nosy_names=['carljm','dino.viehland','itamaro','mpage']
pr_nums=[]
priority='normal'
resolution=None
stage=None
status='open'
superseder=None
type='enhancement'
url='https://bugs.python.org/issue46898'
versions=['Python 3.11']

Linked PRs

@mpage
Copy link
Mannequin Author

mpage mannequin commented Mar 1, 2022

CPython extensions providing optimized execution of Python bytecode (e.g. the Cinder JIT) may need to hook into the lifecycle of code objects to determine what to optimize or to free resources allocated for code objects that no longer exist. We propose adding an API to allow extensions to set callbacks that will be invoked whenever code objects are created or destroyed.

Proposed API:

typedef enum {
PYCODE_LCEVT_CREATED,
PYCODE_LCEVT_DESTROYED
} PyCode_LifecycleEvent;

// A callback to be called when a code object is created or about to be destroyed.
typedef void(*PyCode_LifecycleCallback)(
PyCode_LifecycleEvent event,
PyCodeObject* code);

void PyCode_SetLifecycleCallback(PyCode_LifecycleCallback callback);
PyCode_LifecycleCallback PyCode_GetLifecycleCallback();

@mpage mpage mannequin added 3.11 only security fixes topic-C-API type-feature A feature request or enhancement labels Mar 1, 2022
@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
@gvanrossum
Copy link
Member

In which header file would you put that? Maybe this is simple enough that you should just submit a PR?

@itamaro
Copy link
Contributor

itamaro commented Sep 15, 2022

In which header file would you put that?

wouldInclude/cpython/code.hmake sense for this API?

Maybe this is simple enough that you should just submit a PR?

hopefully yes, we intend to do that for 3.12.
does the proposed API look reasonable?

@gvanrossum gvanrossum added 3.12 bugs and security fixes and removed 3.11 only security fixes labels Sep 24, 2022
@gvanrossum
Copy link
Member

The API looks reasonable -- I'd spellLifeCyclewith two capital letters (thoughWikipedia seems divided on this), and I'd start the enum values withPY_CODE,similar to_PyCodeLocationInfoKindin the same file. The implementation would seem straightforward, and code objects aren't created or destroyed at a monstrous rate (it might be worth seeing if this slows down unmarshalling noticeably, but I doubt it). The callback should probably be stored in the per-interpreter state.

itamaro added a commit to itamaro/cpython that referenced this issue Nov 23, 2022
…on creation and destruction of PyCodeObject
itamaro added a commit to itamaro/cpython that referenced this issue Nov 26, 2022
…on creation and destruction of PyCodeObject
itamaro added a commit to itamaro/cpython that referenced this issue Nov 27, 2022
…on creation and destruction of PyCodeObject
itamaro added a commit to itamaro/cpython that referenced this issue Nov 28, 2022
…on creation and destruction of PyCodeObject

Co-authored-by: Ye11ow-Flash <[email protected]>
markshannon pushed a commit that referenced this issue Dec 2, 2022
* Add API to allow extensions to set callback function on creation and destruction of PyCodeObject

Co-authored-by: Ye11ow-Flash <[email protected]>
pablogsal added a commit that referenced this issue Dec 3, 2022
@pablogsal
Copy link
Member

Unfotunately, there are many buildbots failing to build this PR. Checkbuildbot.python.org/all/#/release_statusfor reference

As per our buildbot policy, this PR will be reverted if the buildbot fleet is not fixed in the next 24h.@markshannon@itamarocan you please take a look?

I have opened#99976.I will close it if the issue is fixed before the deadline.

itamaro added a commit to itamaro/cpython that referenced this issue Dec 3, 2022
pythonGH-99859introduced new buildbot failures, as reported [here](python#91054 (comment)).

I was able to reproduce the failures with:

```
./python.exe -m test -v test_capi.test_watchers -m "*TestCodeObjectWatchers*" -R 3:3
```

The root cause appears to be to static events counters used in the tests,
when running the tests with repetitions (using the same interpreter state),
the counts from the first test run affected the next runs.

This fixes it by resetting the counts when adding and clearing test watchers.
@itamaro
Copy link
Contributor

itamaro commented Dec 4, 2022

this issue can be closed now!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.12 bugs and security fixes topic-C-API type-feature A feature request or enhancement
Projects
None yet
Development

No branches or pull requests

3 participants