Provide a version of FFI.new() that doesn't zero the buffer.

Issue #294 resolved
Cory Benfield
created an issue

Encountered in a Requests issue. Also tracking it on a PyOpenSSL issue.

Essentially, ffi.new("char[]", 1024 * 1024 * 1024) will allocate a 1GB byte buffer and will then zero it. This is because CFFI uses calloc to do its allocations.

In general, this is a useful property. However, for some cases it's not helpful, and would be actively better if we could avoid it. In particular, for cases where we want to receive data into a byte buffer, we don't care if it gets zeroed: that is basically a wasted extra copy. It would be great if applications that want it can opt-into an allocation of a truly empty byte buffer without resorting to binding malloc.

Comments (4)

  1. Cory Benfield reporter

    Assuming I'm understanding what should_clear_after_alloc does, then I think that's fine. I assume there's not much overhead in repeatedly using ffi.new_allocator(should_clear_after_alloc=False)(*args) instead of ffi.new(*args)?

  2. Armin Rigo

    There shouldn't be much overhead, but it was really meant to be used by taking new_dont_clear = ffi.new_allocator(should_clear_after_alloc=False) at global level, and then calling new_dont_clear(). I should make it clearer in the docs.

  3. Log in to comment