Not installing on Ubuntu 18.04

Issue #11 new
Former user created an issue

Tried in Python 2.7.15 final, pip 10.0.1, on Ubuntu 18.04. Logs:

$ pip install -U cryptacular==1.4.1
Collecting cryptacular==1.4.1
    100% |████████████████████████████████| 40kB 1.1MB/s 
Requirement not upgraded as not directly required: pbkdf2 in ./venv/lib/python2.7/site-packages (from cryptacular==1.4.1) (1.3)
Requirement not upgraded as not directly required: setuptools in ./venv/lib/python2.7/site-packages (from cryptacular==1.4.1) (39.1.0)
Building wheels for collected packages: cryptacular
  Running setup.py bdist_wheel for cryptacular ... error
  Complete output from command /home/chris/Geru/geru/venv/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-HOfBud/cryptacular/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-uqEWhX --python-tag cp27:
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-2.7
  creating build/lib.linux-x86_64-2.7/cryptacular
  copying cryptacular/__init__.py -> build/lib.linux-x86_64-2.7/cryptacular
  creating build/lib.linux-x86_64-2.7/cryptacular/core
  copying cryptacular/core/__init__.py -> build/lib.linux-x86_64-2.7/cryptacular/core
  copying cryptacular/core/test_core.py -> build/lib.linux-x86_64-2.7/cryptacular/core
  creating build/lib.linux-x86_64-2.7/cryptacular/pbkdf2
  copying cryptacular/pbkdf2/test_pbkdf2.py -> build/lib.linux-x86_64-2.7/cryptacular/pbkdf2
  copying cryptacular/pbkdf2/__init__.py -> build/lib.linux-x86_64-2.7/cryptacular/pbkdf2
  creating build/lib.linux-x86_64-2.7/cryptacular/bcrypt
  copying cryptacular/bcrypt/test_bcrypt.py -> build/lib.linux-x86_64-2.7/cryptacular/bcrypt
  copying cryptacular/bcrypt/__init__.py -> build/lib.linux-x86_64-2.7/cryptacular/bcrypt
  creating build/lib.linux-x86_64-2.7/cryptacular/crypt
  copying cryptacular/crypt/test_crypt.py -> build/lib.linux-x86_64-2.7/cryptacular/crypt
  copying cryptacular/crypt/__init__.py -> build/lib.linux-x86_64-2.7/cryptacular/crypt
  running egg_info
  writing requirements to cryptacular.egg-info/requires.txt
  writing cryptacular.egg-info/PKG-INFO
  writing namespace_packages to cryptacular.egg-info/namespace_packages.txt
  writing top-level names to cryptacular.egg-info/top_level.txt
  writing dependency_links to cryptacular.egg-info/dependency_links.txt
  reading manifest file 'cryptacular.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  writing manifest file 'cryptacular.egg-info/SOURCES.txt'
  copying cryptacular/bcrypt/_bcrypt.c -> build/lib.linux-x86_64-2.7/cryptacular/bcrypt
  running build_ext
  building 'cryptacular.bcrypt._bcrypt' extension
  creating build/temp.linux-x86_64-2.7
  creating build/temp.linux-x86_64-2.7/crypt_blowfish-1.2
  creating build/temp.linux-x86_64-2.7/cryptacular
  creating build/temp.linux-x86_64-2.7/cryptacular/bcrypt
  gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DNO_BF_ASM -Icrypt_blowfish-1.2/ -I/usr/local/include/python2.7 -c crypt_blowfish-1.2/crypt_blowfish.c -o build/temp.linux-x86_64-2.7/crypt_blowfish-1.2/crypt_blowfish.o
  gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DNO_BF_ASM -Icrypt_blowfish-1.2/ -I/usr/local/include/python2.7 -c crypt_blowfish-1.2/crypt_gensalt.c -o build/temp.linux-x86_64-2.7/crypt_blowfish-1.2/crypt_gensalt.o
  gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DNO_BF_ASM -Icrypt_blowfish-1.2/ -I/usr/local/include/python2.7 -c crypt_blowfish-1.2/wrapper.c -o build/temp.linux-x86_64-2.7/crypt_blowfish-1.2/wrapper.o
  gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DNO_BF_ASM -Icrypt_blowfish-1.2/ -I/usr/local/include/python2.7 -c cryptacular/bcrypt/_bcrypt.c -o build/temp.linux-x86_64-2.7/cryptacular/bcrypt/_bcrypt.o
  In file included from /usr/local/include/python2.7/Python.h:47:0,
                   from cryptacular/bcrypt/_bcrypt.c:26:
  crypt_blowfish-1.2/crypt.h:17:10: fatal error: gnu-crypt.h: Arquivo ou diretório inexistente
   #include <gnu-crypt.h>
            ^~~~~~~~~~~~~
  compilation terminated.
  error: command 'gcc' failed with exit status 1

  ----------------------------------------
  Failed building wheel for cryptacular
  Running setup.py clean for cryptacular
Failed to build cryptacular
requests 2.18.4 has requirement certifi>=2017.4.17, but you'll have certifi 2015.9.6.2 which is incompatible.
Installing collected packages: cryptacular
  Running setup.py install for cryptacular ... error
    Complete output from command /home/chris/Geru/geru/venv/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-HOfBud/cryptacular/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-5w3u0D/install-record.txt --single-version-externally-managed --compile --install-headers /home/chris/Geru/geru/venv/include/site/python2.7/cryptacular:
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-2.7
    creating build/lib.linux-x86_64-2.7/cryptacular
    copying cryptacular/__init__.py -> build/lib.linux-x86_64-2.7/cryptacular
    creating build/lib.linux-x86_64-2.7/cryptacular/core
    copying cryptacular/core/__init__.py -> build/lib.linux-x86_64-2.7/cryptacular/core
    copying cryptacular/core/test_core.py -> build/lib.linux-x86_64-2.7/cryptacular/core
    creating build/lib.linux-x86_64-2.7/cryptacular/pbkdf2
    copying cryptacular/pbkdf2/test_pbkdf2.py -> build/lib.linux-x86_64-2.7/cryptacular/pbkdf2
    copying cryptacular/pbkdf2/__init__.py -> build/lib.linux-x86_64-2.7/cryptacular/pbkdf2
    creating build/lib.linux-x86_64-2.7/cryptacular/bcrypt
    copying cryptacular/bcrypt/test_bcrypt.py -> build/lib.linux-x86_64-2.7/cryptacular/bcrypt
    copying cryptacular/bcrypt/__init__.py -> build/lib.linux-x86_64-2.7/cryptacular/bcrypt
    creating build/lib.linux-x86_64-2.7/cryptacular/crypt
    copying cryptacular/crypt/test_crypt.py -> build/lib.linux-x86_64-2.7/cryptacular/crypt
    copying cryptacular/crypt/__init__.py -> build/lib.linux-x86_64-2.7/cryptacular/crypt
    running egg_info
    writing requirements to cryptacular.egg-info/requires.txt
    writing cryptacular.egg-info/PKG-INFO
    writing namespace_packages to cryptacular.egg-info/namespace_packages.txt
    writing top-level names to cryptacular.egg-info/top_level.txt
    writing dependency_links to cryptacular.egg-info/dependency_links.txt
    reading manifest file 'cryptacular.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'cryptacular.egg-info/SOURCES.txt'
    copying cryptacular/bcrypt/_bcrypt.c -> build/lib.linux-x86_64-2.7/cryptacular/bcrypt
    running build_ext
    building 'cryptacular.bcrypt._bcrypt' extension
    creating build/temp.linux-x86_64-2.7
    creating build/temp.linux-x86_64-2.7/crypt_blowfish-1.2
    creating build/temp.linux-x86_64-2.7/cryptacular
    creating build/temp.linux-x86_64-2.7/cryptacular/bcrypt
    gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DNO_BF_ASM -Icrypt_blowfish-1.2/ -I/usr/local/include/python2.7 -c crypt_blowfish-1.2/crypt_blowfish.c -o build/temp.linux-x86_64-2.7/crypt_blowfish-1.2/crypt_blowfish.o
    gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DNO_BF_ASM -Icrypt_blowfish-1.2/ -I/usr/local/include/python2.7 -c crypt_blowfish-1.2/crypt_gensalt.c -o build/temp.linux-x86_64-2.7/crypt_blowfish-1.2/crypt_gensalt.o
    gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DNO_BF_ASM -Icrypt_blowfish-1.2/ -I/usr/local/include/python2.7 -c crypt_blowfish-1.2/wrapper.c -o build/temp.linux-x86_64-2.7/crypt_blowfish-1.2/wrapper.o
    gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DNO_BF_ASM -Icrypt_blowfish-1.2/ -I/usr/local/include/python2.7 -c cryptacular/bcrypt/_bcrypt.c -o build/temp.linux-x86_64-2.7/cryptacular/bcrypt/_bcrypt.o
    In file included from /usr/local/include/python2.7/Python.h:47:0,
                     from cryptacular/bcrypt/_bcrypt.c:26:
    crypt_blowfish-1.2/crypt.h:17:10: fatal error: gnu-crypt.h: Arquivo ou diretório inexistente
     #include <gnu-crypt.h>
              ^~~~~~~~~~~~~
    compilation terminated.
    error: command 'gcc' failed with exit status 1

    ----------------------------------------
Command "/home/chris/Geru/geru/venv/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-HOfBud/cryptacular/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-5w3u0D/install-record.txt --single-version-externally-managed --compile --install-headers /home/chris/Geru/geru/venv/include/site/python2.7/cryptacular" failed with error code 1 in /tmp/pip-install-HOfBud/cryptacular/

Can you help me?

Comments (24)

  1. Guido Stevens

    Confirmed also on Ubuntu 17.04, python3.6 with python3.6 backports and latest updates installed. I'm running a fully scripted install (Docker + buildout) which used to work flawlessly.

    Stackoverflow suggests the latest python3.6.5 is to blame. The following seems to point in the same direction:

    Getting distribution for 'cryptacular==1.4.1'.
    In file included from /usr/include/python3.6m/Python.h:39:0,
                     from cryptacular/bcrypt/_bcrypt.c:26:
    crypt_blowfish-1.2/crypt.h:17:23: fatal error: gnu-crypt.h: No such file or directory
    compilation terminated.
    error: Setup script exited with error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
    

    Searching for gnu-crypt.h in Ubuntu gives no results. After symlinking /usr/include/gnu-crypt.h -> /usr/include/crypt.h:

    Getting distribution for 'cryptacular==1.4.1'.
    In file included from cryptacular/bcrypt/_bcrypt.c:27:0:
    crypt_blowfish-1.2/ow-crypt.h:27:14: error: conflicting types for ‘crypt_r’
     extern char *crypt_r(__const char *key, __const char *setting, void *data);
                  ^
    In file included from crypt_blowfish-1.2/crypt.h:17:0,
                     from /usr/include/python3.6m/Python.h:39,
                     from cryptacular/bcrypt/_bcrypt.c:26:
    /usr/include/crypt.h:60:14: note: previous declaration of ‘crypt_r’ was here
     extern char *crypt_r (const char *__key, const char *__salt,
                  ^
    error: Setup script exited with error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
    
  2. Guido Stevens

    Update: cd /usr/include && sudo mv crypt.h gnu-crypt.h makes it possible to at least install cryptacular again. I'd advise anybody reading this and contemplating that workaround to revert that change immediately after compiling cryptacular.

  3. Guido Stevens

    This is a python3.6 specific problem, that shows up in my Ubuntu 16.04 install only because I'm running python3.6 backports there.

    Clean reproduction on Ubuntu 18.04:

    • docker run -it ubuntu:bionic

    • apt-get update && apt-get install -y gcc python3-dev python3-pip && pip3 install cryptacular

        x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DNO_BF_ASM -Icrypt_blowfish-1.2/ -I/foo/include -I/usr/include/python3.6m -c cryptacular/bcrypt/_bcrypt.c -o build/temp.linux-x86_64-3.6/cryptacular/bcrypt/_bcrypt.o
        In file included from /usr/include/python3.6m/Python.h:39:0,
                         from cryptacular/bcrypt/_bcrypt.c:26:
        crypt_blowfish-1.2/crypt.h:17:23: fatal error: gnu-crypt.h: No such file or directory
        compilation terminated.
    
  4. Daniel Holth repo owner

    It's strange that python would affect crypt. Maybe it's necessary to edit the bundled bcrypt implementation so that it includes crypt.h instead of gnu-crypt.h (ideally based on a preprocessor flag). Some system crypt support bcrypt without the extension module. That might be another option, to disable the c code and use the pure python part of cryptacukar to talk to crypt().

  5. Guido Stevens

    This worked fine in January with python3.6. I suspect that since then there's been a subtle change in the dist/packaging toolchain in Python which now causes this error to be thrown on recompiling the C extensions. I only ran into this because I accidentally cleared my buildout eggs cache.

    As to including crypt.h, the vendored bcrypt provides its own but apparently also wants to include the systemwide crypt.h as gnu-crypt.h. IOW there's a potential name clash built into the way this monkey patching is done. When I symlink system crypt.h into gnu-crypt.h the compiler still breaks, apparently because it then still loads the systemwide crypt.h on places where it expects the vendored override to be loaded. All seems pretty messy to me.

  6. Daniel Holth repo owner

    Another option would be to stop compiling wrapper.c (uses system crypt for non-bcrypt support) and directly call _crypt_blowfish_rn from the python extension.

  7. Daniel Holth repo owner

    I don't have 18.04

    Let me know if adding undef_macros = [ '__GLIBC__' ] to setup.py. A line in wrapper.c #if defined(__GLIBC__) && defined(_LIBC) only includes crypt.h and crypt-gnu.h if __GLIBC__ and _LIBC are set.

    It might be a bug in Ubuntu's Python that extensions are compiled with these macros set...

          Extension('cryptacular.bcrypt._bcrypt',
              sources = [
              'crypt_blowfish-1.2/crypt_blowfish.c',
              'crypt_blowfish-1.2/crypt_gensalt.c',
              'crypt_blowfish-1.2/wrapper.c',
              'cryptacular/bcrypt/_bcrypt.c',
              # how do I compile .S with distutils?
              # 'crypt_blowfish-1.0.3/x86.S', 
              ],
              define_macros = [ ('NO_BF_ASM', None) ],
              undef_macros = [ '__GLIBC__' ], 
              include_dirs = [ 
              'crypt_blowfish-1.2/',
              ]
          )
    
  8. Daniel Holth repo owner

    I tried the new Ubuntu. Try defining __SKIP_GNU and removing crypt.h from crypt_blowfish-1.2/. This causes it to compile for me. New versions of Python.h #include crypt.h.

              define_macros = [ ('NO_BF_ASM', None), ('__SKIP_GNU', 1) ],
    
  9. Log in to comment