Tie lifespan of one cdata object to another

Issue #12 resolved
Aaron Iles
created an issue

Is it possible to add a mechanism to add a reference from one cdata object to another?

When creating and initialising C structures that include pointers to other C objects it's necessary to keep references to all cdata objects that are created. It would be much simpler (and less error prone) if a co-ownership relationship could be created between the C structure cdata object at the fields cdata object.

Comments (7)

  1. Armin Rigo
    • changed status to open

    Note that you can already hack it on top of cffi. The cdata objects that own memory are weakly referenceable. This makes it possible to have a WeakKeyDictionary that maps the original owner cdata objects to e.g. a list of further cdata objects.

    I'm a bit against tracking dependencies automatically as in ctypes, because it becomes a mess that works only in 90% of the cases and you are left wondering why you get segfaults in the remaining 10%.

    I would suggest a possibly cleaner solution: have a wrapper around the original cdata object. If you keep alive an instance of some class you define, instead of just the bare C structure object, then you can attach any extra cdata objects to this instance as well. This is a way to have the "correct direction" of references: not from cdata objects to Python objects, but from Python objects to cdata objects.

  2. Aaron Iles reporter

    I included both your WeakKeyDictionary and wrapper object suggestions to create a solution. The code is attached or you can read a gist of it on github at https://gist.github.com/3110849.

    I understand the reluctance to included dependency tracking in cffi. Would you consider including an optional functionality that the author had to explicitly enable in cffi? At the very least the fact that cdata objects support weak referencing and that it can be used to build a solution to this problem should be added to the documentation. I have found this exchange extremely helpful.

    Update: Yeah, that code I linked does have one issue. The wrapper object obviously can't be passed directly to foreign functions, you get a TypeError raised. Not a show stopper, but one reason why it might be nice to include support in cffi would be to allow such wrapper objects to be passed directly, rather than being manually unwrapped first.

  3. Armin Rigo

    You're missing my point: I meant *either* use a WeakKeyDictionary *or* use a wrapper, not both at the same time. For example:

    s = ffi.new("struct foo *") field1 = ffi.new("struct bar *") global_weakkey_dict[s] = (field1,) s.bar = field1

  4. Armin Rigo

    Added the WeakKeyDictionary example to the documentation. Closing this as "resolved", kind of: it doesn't take more lines of code to use a WeakKeyDictionary than it would to use some custom mechanism.

  5. Aaron Iles reporter

    The docs fail to mention that pointers returned by C functions can not be weakly referenced. There is at least one scenario when using OpenSSL where cffi created pointers are being assigned to attributes of a C allocated pointer. This scenario was also being covered by my (admittedly horribly over engineered) code snippet above.

    Still, happy with issue be mark resolved.

  6. Log in to comment