multiarray import fails - Symbol not found: _aheapsort_bool on macOS

Create issue
Issue #2994 resolved
S. Moon created an issue

macOS Version: 10.13.6 Apple LLVM version 9.0.0 (clang-900.0.39.2) Target: x86_64-apple-darwin17.7.0

(numpy package was installed through pip, and not built from source)

Steps to repro: 1. Install numpy on a fresh pypy3.5 / 3.6 installation (Affects at least version PyPy3.6 v7.1.0-beta and PyPy3.5 v7.0.0, and also the latest 3.6 nightly - pypy-c-jit-96343-de061d87e39c-osx64) 2. import numpy

Logs:

Traceback (most recent call last):
  File "/Users/user/Downloads/pypy3.6-v7.1.0-osx64/site-packages/numpy/core/__init__.py", line 40, in <module>
    from . import multiarray
  File "/Users/user/Downloads/pypy3.6-v7.1.0-osx64/site-packages/numpy/core/multiarray.py", line 12, in <module>
    from . import overrides
  File "/Users/user/Downloads/pypy3.6-v7.1.0-osx64/site-packages/numpy/core/overrides.py", line 6, in <module>
    from numpy.core._multiarray_umath import (
ImportError: dlopen(/Users/user/Downloads/pypy3.6-v7.1.0-osx64/site-packages/numpy/core/_multiarray_umath.pypy3-71-darwin.so, 6): Symbol not found: _aheapsort_bool
  Referenced from: /Users/user/Downloads/pypy3.6-v7.1.0-osx64/site-packages/numpy/core/_multiarray_umath.pypy3-71-darwin.so
  Expected in: dynamic lookup

This does not repro on Linux builds of the same pypy versions.

Comments (13)

  1. Julian Berman

    From some debugging with @mattip , this looks like the relevant build output (via running python setup.py build_ext in the numpy repo):

    ld: warning: ignoring file build/temp.macosx-10.14-x86_64-2.7/libnpymath.a, file was built for archive which is not the architecture being linked (x86_64): build/temp.macosx-10.14-x86_64-2.7/libnpymath.ald: warning: ignoring file build/temp.macosx-10.14-x86_64-2.7/libnpysort.a, file was built for archive which is not the architecture being linked (x86_64): build/temp.macosx-10.14-x86_64-2.7/libnpysort.a
    
  2. Julian Berman

    Would note that CPython seems to compile only for i386. Comparing objdumps, from CPython:

    ⊙  objdump -x build/temp.macosx-10.14-intel-2.7/libnpysort.a | head -n30                                                       jberman@USNYHJBERMANMB2
    In archive build/temp.macosx-10.14-intel-2.7/libnpysort.a:
    In nested archive quicksort.o:
    
    i386:     file format mach-o-i386
    i386
    architecture: i386, flags 0x00000011:
    HAS_RELOC, HAS_SYMS
    start address 0x00000000
     MACH-O header:
       magic:      0xfeedface
       cputype:    0x7 (I386)
       cpusubtype: 0x3 (X86_ALL)
       filetype:   0x1
       ncmds:      0x4
       sizeocmds:  0x4b4
       flags:      0x2000
       version:    1
    

    vs.:

    ⊙  objdump -x buildold/temp.macosx-10.14-x86_64-2.7/libnpysort.a | head -n30                                                   jberman@USNYHJBERMANMB2
    In archive buildold/temp.macosx-10.14-x86_64-2.7/libnpysort.a:
    
    quicksort.o:     file format mach-o-x86-64
    rw-r--r-- 0/0  61312 Dec 31 19:00 1969 quicksort.o
    architecture: i386:x86-64, flags 0x00000011:
    HAS_RELOC, HAS_SYMS
    start address 0x0000000000000000
     MACH-O header:
       magic:      0xfeedfacf
       cputype:    0x1000007 (X86_64)
       cpusubtype: 0x3 (X86_ALL)
       filetype:   0x1
       ncmds:      0x4
       sizeocmds:  0x1b8
       flags:      0x2000
       version:    2
    
  3. Julian Berman

    Sorry so, crossposting here from #2949 -- the 2 things that should be needed here are exporting MACOSX_DEPLOYMENT_TARGET=10.14 andbrew uninstall binutils if you have that installed -- you may also need to`rm -rf ~/Library/Caches/pip if you have cached broken wheels built -- @Siyoon Moon can you confirm whether those two things fix this for you, or if not pastebin your output after doing those?

  4. Julian Berman

    I think the above should solve this for anyone encountering it, so closing this out, but if anyone still has an issue please do file another ticket with some detail!

  5. S. Moon reporter

    Sorry for the lack of a reply - you mentioned the wrong person so it looks like the notification went to limbo. I just tried this and it seems to work; thanks a lot. This is a rather unobvious “fix” for an average user - any chance this can be made more of a straightforward installation?

  6. S. Moon reporter

    I’m not sure; I also nuked the pip cache as instructed. I should have probably tried with just the deployment target.

  7. mattip

    Did you also have the problems with brew uninstall binutils?

    I think nuking the cache was the right thing to do, since then the MACOSX_DEPLOYMENT_TARGET=10.14 setting could affect compilation.

  8. S. Moon reporter

    Not sure what you mean by problem - I also removed binutils, although I’m a bit unsure if that could have contributed to the problem.

  9. Log in to comment