Skip to content

Improve binding removal logic#5

Merged
oremanj merged 5 commits into
mainfrom
better-destruction
Sep 13, 2025
Merged

Improve binding removal logic#5
oremanj merged 5 commits into
mainfrom
better-destruction

Conversation

@oremanj

@oremanj oremanj commented Sep 11, 2025

Copy link
Copy Markdown
Collaborator

The previous technique of requiring pymb_remove_binding() to be called when the type object has zero references turns out to not be tenable for binding types that don't control their metaclass. The best one can do for such types is a weakref, and GC executes weakref callbacks before breaking cycles.

Create new semantics for binding removal that take care of removal automatically when either the type object is being finalized or the binding capsule is destroyed, whichever comes first. These rely on the fact that free-threaded builds use deferred refcounting for type objects, so it's only possible for a type object to be deallocated during GC. Since free-threaded GC contains a stop-the-world pause in between running finalizers/weakrefs and breaking references, we can protect concurrent binding users by unpublishing the binding during type finalization but not deallocating it until type destruction. If the binding is to be removed before its type dies, we use a capsule-in-a-cycle to defer the deallocation until the next GC pass.

@oremanj oremanj force-pushed the better-destruction branch 2 times, most recently from adcc6e9 to dad39eb Compare September 11, 2025 22:36
The previous technique of requiring `pymb_remove_binding()` to be called when the type object has zero references turns out to not be tenable for binding types that don't control their metaclass. The best one can do for such types is a weakref, and GC executes weakref callbacks before breaking cycles.

Create new semantics for binding removal that take care of removal automatically when either the type object is being finalized or the binding capsule is destroyed, whichever comes first. These rely on the fact that free-threaded builds use deferred refcounting for type objects, so it's only possible for a type object to be deallocated during GC. Since free-threaded GC contains a stop-the-world pause in between running finalizers/weakrefs and breaking references, we can protect concurrent binding users by unpublishing the binding during type finalization but not deallocating it until type destruction. If the binding is to be removed before its type dies, we use a capsule-in-a-cycle to defer the deallocation until the next GC pass.
@pablogsal

pablogsal commented Sep 12, 2025

Copy link
Copy Markdown
Member

Apart from accounting for this towards #4 , tell me if you would like me to take a look at this PR but one thing I noticed skimming at the approach (which may not be a problem but is still good to have present) is that technically speaking weakreaf callbacks may not be honoured in some situations. The most representative one is if they are involved in the transitive closure of unreachable cycles with the object they refer. For example:

import weakref
import gc

def c(*args):
    print(args)

class A:
    def __del__(self):
        print(self.x)
        print("Deleting A")

a = A()
x = weakref.ref(a, c)
a.a = a
a.x = x
del a
del x

print("Collecting")
gc.collect()
print("Collected")

this prints:

Collecting
<weakref at 0x77ff9b4b27a0; dead>
Deleting A
Collected

You can check more about it here: https://github.com/python/cpython/blob/4afa98596ea0182a9aed1556866813052c425a8e/Python/gc.c#L957-L961

Of course this may not apply to your current design, but if the weakref is exposed somehow or is under user management at some point that may be a potential trouble.

Comment thread pymetabind.h Outdated
Comment thread pymetabind.h Outdated
Comment thread pymetabind.h
Comment thread pymetabind.h
Comment thread pymetabind.h
Comment thread pymetabind.h
Comment thread pymetabind.h
@oremanj

oremanj commented Sep 12, 2025

Copy link
Copy Markdown
Collaborator Author

weakreaf callbacks may not be honoured in some situations. The most representative one is if they are involved in the transitive closure of unreachable cycles with the object they refer.

Thanks for pointing this out. Since the weakrefs I'm using here are kept alive by references that are not visible to GC, I don't think this is a hazard for the current implementation.

@pablogsal

pablogsal commented Sep 12, 2025

Copy link
Copy Markdown
Member

Create new semantics for binding removal that take care of removal automatically when either the type object is being finalized or the binding capsule is destroyed, whichever comes first. These rely on the fact that free-threaded builds use deferred refcounting for type objects, so it's only possible for a type object to be deallocated during GC.

I think it's very unlikely we will change this (and I am sure I am not pointing out something you haven't already considered) but it's technically possible we change this in the future (or even between patch versions) so this has a nonzero chance of becoming a bit brittle in the future.

Comment thread pymetabind.h Outdated
@oremanj oremanj merged commit 112de2f into main Sep 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants