Issue #92 resolved

Memory leak when ffi.callback() is in a reference cycle

Simon Sapin
created an issue

CPython’s cycle collector does not seem to know about the reference held by ffi.callback() objects to their function argument, which can cause memory leaks when a cycle goes through that reference.

I don’t know if PyPy is affected, leaks are not as easy to spot without sys.getrefcount().

This test case fails on CPython with CFFI 0.6:

import gc
import sys
import cffi

def make_callback():
    container = [data]
    callback = ffi.callback('int()', lambda: len(vars(container)))
    container.append(callback)
    # Ref cycle: callback -> lambda (closure) -> container -> callback
    return callback

ffi = cffi.FFI()
data = 'something'
initial_refcount = sys.getrefcount(data)
callback = make_callback()
assert sys.getrefcount(data) == initial_refcount + 1
del callback
gc.collect()
assert sys.getrefcount(data) == initial_refcount  # Fails, data is leaked

An example of real code with this kind of cycle: https://github.com/SimonSapin/cairocffi/blob/v0.5/cairocffi/surfaces.py#L70

The issue can be worked around with weakref (which cairocffi will do), but it would be better fixed in CFFI.

http://docs.python.org/2/c-api/typeobj.html#PyTypeObject.tp_traverse seems to be related, but I’m not sure.

Comments (4)

  1. Armin Rigo

    There are 3 kinds, the 3rd one being p=ffi.new("struct foo *"), which currently owns a regular Python reference to the object p[0]. (This is a bit of a hack.) All 3 kinds should have the same solution though.

  2. Armin Rigo

    Solved in 3751a1489e7b and subsequent checkins. Only for handles and for callbacks, but these are the only two kinds of cdata objects that can possibly have a reference to a Python object --- ignoring the ctype; I'm sure someone sufficiently determined could do a cycle that involves a cdata and its ctype, but that looks unlikely to be a problem, so I'll just ignore it for now.

  3. Log in to comment