Simple undocumented question

Issue #279 closed
Flávio Coelho
created an issue

Hi I was trying to wrap a simple C source code consisting of two .C Source files (files attached).

I adapted the (very) simple example from the documentation getting to this:

import os
from cffi import FFI
blastbuilder = FFI()
ffibuilder = FFI()
with open(os.path.join(os.path.dirname(__file__), "c-src/blast.c")) as f:
    blastbuilder.set_source("blast", f.read(), libraries=["c"])
with open(os.path.join(os.path.dirname(__file__), "c-src/blast.h")) as f:
    blastbuilder.cdef(f.read())
blastbuilder.compile(verbose=True)

with open('c-src/dbc2dbf.c','r') as f:
    ffibuilder.set_source("_readdbc",
                          f.read(),
                          libraries=["c"])

with open(os.path.join(os.path.dirname(__file__), "c-src/blast.h")) as f:
    ffibuilder.cdef(f.read(), override=True)

if __name__ == "__main__":
    # ffibuilder.include(blastbuilder)
    ffibuilder.compile(verbose=True)

However when it compiles it does not link both blast.o and _readdbc.o into a single .so it compiles only _readdbc, and when I try to import _readdbc, it gives me an import error, saying it can find blast. If I do it by hand, I can import _readdbc, but I don’t see any of the functions declared in dbc2dbf.c

Maybe I am doing something stupid, but I couldn't find any example that illustrated a case of wrapping similar to mine. In any case, my knowledge of C is very limited. So I apologize in advance for any stupid mistake.

Lastly, I know this not the best place to ask such a question. But maybe this little problem can become a documentation issue.

Comments (8)

  1. Flávio Coelho reporter

    Update: I managed to build the code and expose the Functions I wanted, by doing this:

    ffibuilder.set_source("_readdbc",
                              f.read(),
                              libraries=["c"],
                              sources=["c-src/blast.c"])
    

    Now I am having trouble calling a function with the following signature:

    void dbc2dbf(char** input_file, char** output_file);
    

    How do I cast such a pointer? I have Tried

    ffi.new('char[]', os.path.abspath(infile))
    

    but it gave the following error:

    python3: Bad address: Unknown error 1886221359

  2. Armin Rigo

    The signature is not enough to guess how the function is supposed to be called, in this case. It takes what might be, likely, two pointers over char *, and possibly change them to point to another char *. It's not clear to me why it would do that, given the argument names. Can you link to the documentation of the function? In any case, the ffi.new('char[]', xyz) returns a char *, whereas you need a char **. Maybe try p = ffi.new('char[]', xyz); q = ffi.new('char *[]', [p]), pass q to the function, and afterwards read ffi.string(q[0]) again to check the possibly-changed string value? That's just a rough guess; the documentation of the function is needed to tell more, or maybe an example in C that calls it.

  3. Flávio Coelho reporter

    Hi Armin,

    Thanks for the help. Here is my latest attempt, based on your suggestion:

    import os
    import _readdbc as rdbc
    from _readdbc import ffi, lib
    
    def dbc2dbf(infile, outfile):
        infile = ffi.new('char *[]', ffi.new('char[]', os.path.abspath(infile)))
        outfile = ffi.new('char *[]', ffi.new('char[]', os.path.abspath(outfile)))
        print(infile, ffi.string(infile[0]))
        lib.dbc2dbf(infile, outfile)
        print(os.path.exists(outfile))
    
    if __name__ == "__main__":
        dbc2dbf(b'/tmp/DNRJ2014.dbc', b'test.dbf')
    

    It is now giving the following error:

     File "readdbc.py", line 15, in dbc2dbf
        infile = ffi.new('char *[]', ffi.new('char[]', os.path.abspath(infile)))
    TypeError: '_cffi_backend.CDataOwn' object cannot be interpreted as an integer
    

    Have you taken a look at the C source files that I attached to the previous message? It's pretty much all I have, this C code came from an R package that I want to port to Python. In the R wrapper code this is how the dbc2dbf function is called:

    dbc2dbf <- function(input.file, output.file) {
            if( !file.exists(input.file) )
                    stop("Input file does not exist.")
            out <- .C("dbc2dbf", input = as.character(path.expand(input.file)), output = as.character(path.expand(output.file)))
            file.exists(output.file)
    }
    

    So the actual arguments are passed as strings.

  4. Flávio Coelho reporter

    I tried also like this:

    def dbc2dbf(infile, outfile):
        infile = ffi.new('char**', ffi.new('char[]', os.path.abspath(infile)))
        outfile = ffi.new('char**', ffi.new('char[]', os.path.abspath(outfile)))
        print(infile, ffi.string(infile[0]))
        lib.dbc2dbf(infile, outfile)
        print(os.path.exists(outfile))
    

    Then, it seems to go a little further, the first print gives:

    <cdata 'char * *' owning 8 bytes> b'/tmp/DNRJ2014.dbc'
    

    So we are finally getting a char**, but I still get an error:

    python3: No such file or directory: Unknown error 1340028632
    

    Even though both files exist (I checked).

  5. Armin Rigo

    ffi.new(..., ffi.new(...)) is always wrong, because the lifetime of the inner object is restricted to the outer call---in other words, as soon as the complete line is done, the inner ffi.new() is freed. You must use a variable instead.

  6. Armin Rigo

    Or use this style:

    p = ffi.new("char[]", "myfilename")
    q = ffi.new("char[]", "myfilename")
    lib.dbc2dbf([p], [q])
    

    It's the same in this case because dbc2dbf takes char ** argument for no good reason (from a C point of view): it never changes that char *, and only ever read input_file[0] and output_file[0].

  7. Flávio Coelho reporter

    I got it!!

    here is the final code:

    import os
    import _readdbc as rdbc
    from _readdbc import ffi, lib
    
    def dbc2dbf(infile, outfile):
        p = ffi.new('char[]', os.path.abspath(infile))
        q = ffi.new('char[]', os.path.abspath(outfile))
    
        lib.dbc2dbf([p], [q])
    
        print(os.path.exists(outfile))
    
    if __name__ == "__main__":
        dbc2dbf(b'/tmp/DNRJ2014.dbc', b'/tmp/output.dbf')
    

    It is really better to pass [p] and [q]

    Thanks a lot for the help!

  8. Log in to comment