More detailed documentation for libraries that return allocated memory

Issue #251 resolved
jbarlow NA
created an issue

I found it tricky to work out to access a library function that looked this:

void generate_numbers(float **out_data, size_t *size);

where generate_numbers returns allocated memory by writing an array to *out_data:

*out_data = calloc(N, sizeof(float));  // or whatever the parameter order is
*out_data[0...N] = values
*size = allocated bytes

The solution seems unintuitive to me, especially because cffi seems picky about precisely how C initializers are specified, and sometimes one must use a different specification from the C function signature for pointers to arrays.

I am also concerned that I have might done did it in a way that leaks memory or is vulnerable to double frees.

data = ffi.new('float *[]', 1)    # does not work with float **, meaning of initializer not obvious
size = ffi.new('size_t *', 0)

library.generate_numbers(data, size)

I then wanted to access these numbers as a bytes object in Python and save them.

char_data = ffi.cast('char *', data[0])    # does not work to ffi.buffer(data) or ffi.buffer(data[0])
data_bytes = ffi.buffer(char_data, size[0])[:]

Comments (2)

  1. Armin Rigo

    From your description, it seems that you should be able to use the following:

    pdata = ffi.new('float **')
    psize = ffi.new('size_t *')
    library.generate_numbers(pdata, psize)
    

    and then use pdata[0] as a float * and psize[0] as an integer size. If you're confused by the optional initializer to ffi.new() (it defaults to 0 or NULL anyway, which is good enough here): in ffi.new('foo *', x) the x must be an initializer for foo. It can be 0 for ffi.new('size_t *', 0) but it must be ffi.NULL for ffi.new('float **', ffi.NULL), because it should be the initializer for that float * pointer. If you're using the syntax ffi.new('float *[]', x) instead, then you're making an array of float * items, and the initializer's meaning changes: it is either an integer telling the length of that array, or it is a list of array items.

    For the second part, indeed, ffi.buffer() checks the type of pointer, so you need a cast. There is no simpler solution. It might possibly go into the same thought process than https://groups.google.com/forum/?hl=en#!topic/python-cffi/tIBi88knDvs .

  2. Armin Rigo

    The first part describes some confusion with ffi.new() that is well-documented, even though it is a bit confusing. I haven't so far found a nicer solution.

    Done the second part with ffi.unpack().

  3. Log in to comment