1. Pypy
  2. Untitled project
  3. pypy
  4. Issues

Issues

Issue #2030 wontfix

Circular weakref callbacks never called (self._x = weakref.ref(self, callback))

Jason Madden
created an issue

It appears that if an object holds a weak reference to itself, the callback for that reference is never called. It's as if when both the object and the reference to it become garbage at the same time, the callback isn't called.

I ran into this testing ZODB under PyPy (it used this pattern in ZODB.blob.Blob).

Here's an example of what happens under CPython:

$ /opt/local/bin/python2.7
Python 2.7.9 (default, Dec 13 2014, 15:13:49)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import weakref, sys
>>> class X(object):
...     pass
...
>>> x = X()
>>> x._ref = weakref.ref(x, lambda r: sys.stderr.write('callback\n'))
>>> del x
callback
>>>

Now, under PyPy (both 2.5.1 and a relatively recent nightly), the callback doesn't get called when the reference is deleted (but we expect that):

$ pypy
Python 2.7.9 (9c4588d731b7, Mar 23 2015, 16:20:40)
[PyPy 2.5.1 with GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>> import weakref, sys
>>>> class X(object):
....     pass
....
>>>> x = X()
>>>> x._ref = weakref.ref(x, lambda r: sys.stderr.write('callback\n'))
>>>> del x
>>>>

No problem, we need to ask GC to do a collection. We've seen this before.

>>>> import gc
>>>> gc.collect()
0

Hmm, maybe a few collections?

>>>> for _ in range(100): _ = gc.collect()
>>>>

Nothing.

If we use a separate reference object with a different lifetime, then it is called immediately after the first GC:

>>>> x = X()
>>>> ref = weakref.ref(x, lambda r: sys.stderr.write('direct callback\n'))
>>>> del x
>>>> gc.collect()
direct callback
0
>>>>

Comments (3)

  1. Armin Rigo

    This works as documented by the line in the Python documentation that introduces callbacks (https://docs.python.org/2/library/weakref.html):

    """ If callback is provided and not None, and the returned weakref object is still alive, the callback will be called when the object is about to be finalized"""

    In other words, because the weakref object dies at the same time as the object it is a weak reference to, then the callback is not invoked. You can see the same effect in CPython by doing del x._ref; del x in this order, which does not invoke the callback.

    We should investigate more in-depth what the real reason for this restriction is, and if it makes sense to weaken it somehow in PyPy...

  2. Log in to comment