1. Python CFFI
  2. Untitled project
  3. cffi
  4. Issues
Issue #90 invalid

using StringIO as a file-obj won't cast

Alain Cady
created an issue

Dunno if it's a bug, but trying to cast a StringIO into a 'FILE *' failed with an attribute error exception. Using cStringIO leads to different exception.

CPython & Pypy behave alike, with both modules.

StringIO: Traceback (most recent call last):

  File "app_main.py", line 72, in run_toplevel
  File "cffi_stringio.py", line 10, in <module>
    c_s = ffi.cast("FILE *", s)
  File "/usr/lib64/pypy2.0/lib_pypy/cffi/api.py", line 208, in cast
    return self._backend.cast(cdecl, source)
AttributeError: StringIO instance has no attribute '__trunc__'

cStringIO: Traceback (most recent call last):

  File "app_main.py", line 72, in run_toplevel
  File "cffi_stringio.py", line 10, in <module>
    c_s = ffi.cast("FILE *", s)
  File "/usr/lib64/pypy2.0/lib_pypy/cffi/api.py", line 208, in cast
    return self._backend.cast(cdecl, source)
TypeError: unsupported operand type for int(): 'StringI'

Using Pypy 2.0.2 (same behaviour observed with 2.0_beta1)

Comments (7)

  1. Armin Rigo

    "FILE *" only works with real file objects. I'm even thinking about removing support, because it can be done more explicitly using the fileno() attribute and the C function fdopen(), as done anyway on Python 3 --- but again only for real file objects.

    If you're on Linux, I think there was some C API to open an in-memory "FILE *" object, but it's certainly not commonly used. It's probably better to just open a regular file and transfer between memory and the file. If the file is short-lived (or, Linux again, located in /dev/shm/), then it doesn't go to disk anyway.

  2. Alain Cady reporter

    There is fmemopen() indeed; but actually it is defined in POSIX.1-2008, therefore not Linux specific. As far I understand, StringIO does not provide file descriptor therefore casting isn't straightforward (I got no clue about internal data representation of StringIO).

    For now, cffi allows me to use a C library without writing any line of C, dropping "FILE *" support will enforce the need of C wrappers...

  3. Armin Rigo

    No, it's wrong. You can declare the C functions fdopen() and fclose() in the cdef, and call them from Python in order to get a "FILE *" from a file descriptor.

    For your problem, you can declare the C function fmemopen() in the cdef, and call it from Python, for example as follows. If you want to transfer data in the Python->C direction:

    s = stringio.getvalue()
    f = lib.fmemopen(s, len(s), 'r')
    # then pass 'f' to some C function
    lib.fclose(f)
    

    Or, to go in the C->Python direction, assuming you know how large a buffer you need:

    buf = ffi.new("char[]", size_of_buffer)
    f = lib.fmemopen(buf, size_of_buffer, 'w')
    # then pass 'f' to some C function
    lib.fclose(f)
    # the result is in 'buf'
    
  4. Alain Cady reporter

    I don't even think of that, my bad...

    Thanks for the code snippet. Python->C works well on CPython, but produces really strange behaviours with Pypy (lines read from C look like random garbage).

  5. Armin Rigo

    Ah, oups. fmemopen()'s first argument must point to a buffer that remains valid for as long as the file is in use. On CPython it is the case only by chance, as long as the stringio doesn't go out of scope. Updated code (always for the Python->C direction):

    s = stringio.getvalue()
    buf = ffi.new("char[]", s)    # the actual buffer
    f = lib.fmemopen(buf, len(s), 'r')
    # ...pass 'f' to some C function...
    lib.fclose(f)
    # make sure that 'buf' remains alive up to this point
    
  6. Log in to comment