Unable to install numpy with pypy3 on MacOS

Issue #2942 resolved
hunch created an issue

Hi,

I am trying to install numpy with pip_pypy3, but I am getting this lengthy error. The last section of the error is

...
...

    compile options: '-Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/local/Cellar/pypy3/6.0.0/libexec/include -c'
    cc: _configtest.c
    cc -pthread -arch x86_64 _configtest.o -o _configtest
    ld: library not found for -lgcc_s.10.4
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    failure.
    removing: _configtest.c _configtest.o _configtest.o.d
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/22/cp1phkbd42q8dj21sdk2z7m80000gn/T/pip-install-w4eoqdu5/numpy/setup.py", line 415, in <module>
        setup_package()
      File "/private/var/folders/22/cp1phkbd42q8dj21sdk2z7m80000gn/T/pip-install-w4eoqdu5/numpy/setup.py", line 407, in setup_package
        setup(**metadata)
      File "/private/var/folders/22/cp1phkbd42q8dj21sdk2z7m80000gn/T/pip-install-w4eoqdu5/numpy/numpy/distutils/core.py", line 171, in setup
        return old_setup(**new_attr)
      File "/usr/local/Cellar/pypy3/6.0.0/libexec/site-packages/setuptools/__init__.py", line 143, in setup
        return distutils.core.setup(**attrs)
      File "/usr/local/Cellar/pypy3/6.0.0/libexec/lib-python/3/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/local/Cellar/pypy3/6.0.0/libexec/lib-python/3/distutils/dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "/usr/local/Cellar/pypy3/6.0.0/libexec/lib-python/3/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/private/var/folders/22/cp1phkbd42q8dj21sdk2z7m80000gn/T/pip-install-w4eoqdu5/numpy/numpy/distutils/command/install.py", line 62, in run
        r = self.setuptools_run()
      File "/private/var/folders/22/cp1phkbd42q8dj21sdk2z7m80000gn/T/pip-install-w4eoqdu5/numpy/numpy/distutils/command/install.py", line 36, in setuptools_run
        return distutils_install.run(self)
      File "/usr/local/Cellar/pypy3/6.0.0/libexec/lib-python/3/distutils/command/install.py", line 549, in run
        self.run_command('build')
      File "/usr/local/Cellar/pypy3/6.0.0/libexec/lib-python/3/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/local/Cellar/pypy3/6.0.0/libexec/lib-python/3/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/private/var/folders/22/cp1phkbd42q8dj21sdk2z7m80000gn/T/pip-install-w4eoqdu5/numpy/numpy/distutils/command/build.py", line 47, in run
        old_build.run(self)
      File "/usr/local/Cellar/pypy3/6.0.0/libexec/lib-python/3/distutils/command/build.py", line 135, in run
        self.run_command(cmd_name)
      File "/usr/local/Cellar/pypy3/6.0.0/libexec/lib-python/3/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/local/Cellar/pypy3/6.0.0/libexec/lib-python/3/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/private/var/folders/22/cp1phkbd42q8dj21sdk2z7m80000gn/T/pip-install-w4eoqdu5/numpy/numpy/distutils/command/build_src.py", line 148, in run
        self.build_sources()
      File "/private/var/folders/22/cp1phkbd42q8dj21sdk2z7m80000gn/T/pip-install-w4eoqdu5/numpy/numpy/distutils/command/build_src.py", line 159, in build_sources
        self.build_library_sources(*libname_info)
      File "/private/var/folders/22/cp1phkbd42q8dj21sdk2z7m80000gn/T/pip-install-w4eoqdu5/numpy/numpy/distutils/command/build_src.py", line 292, in build_library_sources
        sources = self.generate_sources(sources, (lib_name, build_info))
      File "/private/var/folders/22/cp1phkbd42q8dj21sdk2z7m80000gn/T/pip-install-w4eoqdu5/numpy/numpy/distutils/command/build_src.py", line 375, in generate_sources
        source = func(extension, build_dir)
      File "numpy/core/setup.py", line 667, in get_mathlib_info
        raise RuntimeError("Broken toolchain: cannot link a simple C program")
    RuntimeError: Broken toolchain: cannot link a simple C program

    ----------------------------------------
Command "/usr/local/Cellar/pypy3/6.0.0/bin/pypy3 -u -c "import setuptools, tokenize;__file__='/private/var/folders/22/cp1phkbd42q8dj21sdk2z7m80000gn/T/pip-install-w4eoqdu5/numpy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /private/var/folders/22/cp1phkbd42q8dj21sdk2z7m80000gn/T/pip-record-kycren62/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/22/cp1phkbd42q8dj21sdk2z7m80000gn/T/pip-install-w4eoqdu5/numpy/

System version: MacOS 10.14.2; pip-18.1; setuptools-40.6.3; pypy3(6.0.0). I can install cython successfully. Any help would be great. Thanks.

Comments (9)

  1. mattip

    This seems like a problem with clang. There is something similar at this link

    Can you compile numpy on cpython from source?

    python3 -mvenv /tmp/test_numpy
    source /tmp/test_numpy/bin/activate
    pip install --ignore-installed -vv --no-binary :all: numpy
    

    If that succeeds, perhaps you can compare the output to see what changed

  2. champagne_alexandre

    Hello hunch and mattip,

    I'm having what I think is the exact same issue (same OS, same error messages I think - but those logs are long...), from what I can read. See my issue (Issue #2949).

    I just tried the command you suggested, mattip, and from what I can see it executed without any error message. See the attached log file. The output differs starting from pretty much the first line(s), i.e. when using pypy we immediately get

      Running setup.py install for numpy ... error
    

    while with CPython no such line appear.

    Do you have any other suggestion? I'm not really used to compiling Python (as a matter of fact, I'm no expert at compiling things in general).

    Edit : sorry, forgot the link to the log file https://drive.google.com/open?id=1tD-24GaqWTjYMl4iRpRwCtkOqyTWJRg9

  3. mattip

    It seems to be some macOS confusion about the compiler. I don't know anything about macOS, maybe someone else has a clue?

  4. Václav Slavík

    This appears to be a bug in macOS 10.14 and/or latest Xcode, where the libgcc_s.10.4 stub doesn't exist anymore in /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib/ (only libgcc_s.1.tbd does), but some compiler flags nevertheless trigger its use — that's according to this comment on an unrelated issue.

    Per a related discussion, a workaround is to temporarily make libgcc_s.10.4.dylib exist, e.g. like this:

    ln -s /usr/lib/libSystem.B.dylib /usr/local/lib/libgcc_s.10.4.dylib
    pip install numpy
    rm /usr/local/lib/libgcc_s.10.4.dylib
    
  5. Andrew Stepanov

    Try installing with

    $ MACOSX_DEPLOYMENT_TARGET=10.14 pip install numpy
    

    I don't think this is PyPy's fault. You get the exact same error when building from source on CPython.

  6. Alejandro SI

    I’ve tried
    $ MACOSX_DEPLOYMENT_TARGET=10.14 pip install numpy
    successfully and silently, but when running “import numpy” from pypy shell, it failed and the error thrown was the following below. Any idea?

    (virtualenv_con_pypy3_5_6_0_0) Alejandro:virtualenv_con_pypy3_5_6_0_0 alejandrosantillaniturres$ python

    Python 3.5.3 (fdd60ed87e94, Apr 24 2018, 06:10:18)

    [PyPy 6.0.0 with GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.1)] on darwin

    Type "help", "copyright", "credits" or "license" for more information.

    import numpy

    Traceback (most recent call last):

    File "/Users/alejandrosantillaniturres/Desktop/programming/python/virtualenv_con_pypy3_5_6_0_0/site-packages/numpy/core/__init__.py", line 40, in <module>

    from . import multiarray

    File "/Users/alejandrosantillaniturres/Desktop/programming/python/virtualenv_con_pypy3_5_6_0_0/site-packages/numpy/core/multiarray.py", line 12, in <module>

    from . import overrides

    File "/Users/alejandrosantillaniturres/Desktop/programming/python/virtualenv_con_pypy3_5_6_0_0/site-packages/numpy/core/overrides.py", line 6, in <module>

    from numpy.core._multiarray_umath import (

    ImportError: dlopen(/Users/alejandrosantillaniturres/Desktop/programming/python/virtualenv_con_pypy3_5_6_0_0/site-packages/numpy/core/_multiarray_umath.pypy3-60-darwin.so, 6): Symbol not found: _PyStructSequence_InitType2

    Referenced from: /Users/alejandrosantillaniturres/Desktop/programming/python/virtualenv_con_pypy3_5_6_0_0/site-packages/numpy/core/_multiarray_umath.pypy3-60-darwin.so

    Expected in: flat namespace

    in /Users/alejandrosantillaniturres/Desktop/programming/python/virtualenv_con_pypy3_5_6_0_0/site-packages/numpy/core/_multiarray_umath.pypy3-60-darwin.so

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):

    File "<stdin>", line 1, in <module>

    File "/Users/alejandrosantillaniturres/Desktop/programming/python/virtualenv_con_pypy3_5_6_0_0/site-packages/numpy/__init__.py", line 142, in <module>

    from . import core

    File "/Users/alejandrosantillaniturres/Desktop/programming/python/virtualenv_con_pypy3_5_6_0_0/site-packages/numpy/core/__init__.py", line 71, in <module>

    raise ImportError(msg)

    ImportError:

    IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

    Importing the multiarray numpy extension module failed. Most

    likely you are trying to import a failed build of numpy.

    Here is how to proceed:

    • If you're working with a numpy git repository, try `git clean -xdf`

    (removes all files not under version control) and rebuild numpy.

    • If you are simply trying to use the numpy version that you have installed:

    your installation is broken - please reinstall numpy.

    • If you have already reinstalled and that did not fix the problem, then:

    1. Check that you are using the Python you expect (you're using /Users/alejandrosantillaniturres/Desktop/programming/python/virtualenv_con_pypy3_5_6_0_0/bin/python),

    and that you have no directories in your PATH or PYTHONPATH that can

    interfere with the Python and numpy versions you're trying to use.

    2. If (1) looks fine, you can open a new issue at

    https://github.com/numpy/numpy/issues. Please include details on:

    • how you installed Python

    • how you installed numpy

    • your operating system

    • whether or not you have multiple versions of Python installed

    • if you built from source, your compiler versions and ideally a build log

    Note: this error has many possible causes, so please don't comment on

    an existing issue about this - open a new one instead.

    Original error was: dlopen(/Users/alejandrosantillaniturres/Desktop/programming/python/virtualenv_con_pypy3_5_6_0_0/site-packages/numpy/core/_multiarray_umath.pypy3-60-darwin.so, 6): Symbol not found: _PyStructSequence_InitType2

    Referenced from: /Users/alejandrosantillaniturres/Desktop/programming/python/virtualenv_con_pypy3_5_6_0_0/site-packages/numpy/core/_multiarray_umath.pypy3-60-darwin.so

    Expected in: flat namespace

    in /Users/alejandrosantillaniturres/Desktop/programming/python/virtualenv_con_pypy3_5_6_0_0/site-packages/numpy/core/_multiarray_umath.pypy3-60-darwin.so

    quit()

  7. mattip

    You cannot use PyPy3 5.6 with the latest NumPy release. Even PyPy3 7.1.1 release will fail with this version of numpy. You either need to use an older version of numpy, or a PyPy nightly.

    NumPy is now checking the latest PyPy in its CI, so this error should not repeat going forward.

  8. Alejandro SI

    Good advice!
    I’ve tried with:

    MACOSX_DEPLOYMENT_TARGET=10.14 pip install numpy==1.14.1

    It installed silently and when importing I only had a warning. Maybe it works. If not, I will go down in versions of numpy.

    Thank you mattip!

    Python 3.5.3 (fdd60ed87e94, Apr 24 2018, 06:10:18)[PyPy 6.0.0 with GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.1)] on darwinType "help", "copyright", "credits" or "license" for more information.

    import numpy

    /Users/alejandrosantillaniturres/Desktop/programming/python/virtualenv_con_pypy3_5_6_0_0/lib-python/3/importlib/_bootstrap.py:223: UserWarning: builtins.type size changed, may indicate binary incompatibility. Expected 872, got 416

    return f(*args, **kwds)

  9. Log in to comment