1. Python CFFI
  2. Untitled project
  3. cffi
  4. Issues
Issue #97 resolved

ffi.verify fails when using chdir() and setup.py

Antonio Cuni
created an issue

Suppose to have the following foo.py file:

import cffi
import os

ffi = cffi.FFI()
ffi.cdef("""
    int foo(int);
""")

old_cwd = os.getcwd()
os.chdir('/tmp/pypyenv/site-packages')

lib = ffi.verify("""
    int foo(int x) {
        return x+1;
    }
""")

os.chdir(old_cwd)

and the following setup.py:

import os
from distutils.core import setup
import foo

foo_ext = foo.ffi.verifier.get_extension()
setup (name = 'foo',
       version = '1.0',
       description = '...',
           py_modules=['foo'],
       author = "Antonio Cuni",
       ext_modules = [foo_ext],
       )

then cffi fails to locate the module after installation:

$ mkvirtualenv -p pypy /tmp/pypyenv
$ /tmp/pypyenv/bin/pypy setup.py install
$ cd /tmp/
$ /tmp/pypyenv/bin/pypy -c 'import foo'
Traceback (most recent call last):
  File "app_main.py", line 72, in run_toplevel
  File "app_main.py", line 579, in run_it
  File "<string>", line 1, in <module>
  File "/tmp/pypyenv/site-packages/foo.py", line 12, in <module>
    lib = ffi.verify("""
  File "/home/antocuni/pypy/default/lib_pypy/cffi/api.py", line 311, in verify
    lib = self.verifier.load_library()
  File "/home/antocuni/pypy/default/lib_pypy/cffi/verifier.py", line 69, in load_library
    return self._load_library()
  File "/home/antocuni/pypy/default/lib_pypy/cffi/verifier.py", line 143, in _load_library
    return self._vengine.load_library()
  File "/home/antocuni/pypy/default/lib_pypy/cffi/vengine_gen.py", line 64, in load_library
    module = backend.load_library(self.verifier.modulefilename)
OSError: Cannot load library _cffi__gbeedbab6x9e554b22.pypy-22.so: _cffi__gbeedbab6x9e554b22.pypy-22.so: cannot open shared object file: No such file or directory

Note that the file _cffi__gbeedbab6x9e554b22.pypy-22.so is correctly installed to site-packages.

This is what happens: - verifier.locate_module calls self._vengine.find_module, passing path==None - _vengine.find_module starts to look in sys.path to find one - the first entry of sys.path is '', so filename==os.path.join('', basename) == '_cffi.....-pypy-22.so') - os.path.isfile returns True, and filename is returned.

This string is eventually passed to dlopen. However, since it does not contain a slash, it is NOT interpreted as a filename by dlopen (as confirmed by "man dlopen"), and the .so won't be loaded.

The correct fix is to make sure that the string returned by find_module always contains at least one slash (either a relative path like './_cffi...' or an absolute one).

Note that on older version of ubuntu (e.g. ubuntu10), dlopen seems to be broken and it incorrectly finds the .so even if it does not contain a slash. On newer versions, it correctly fails.

(If you wonder why I need to chdir() before verify(), it's because I'm passing additional C source files to verify(), and by chdir() I can pass relative filenames. If I used absolute filenames, the CRC would be different after setup.py install, and cffi would try to recompile the module)