Cannot pip install rpy2 with latest R 3.4.0 (unsupported option '-fopenmp') on macOS

Issue #403 open
Clemens Brunner
created an issue

Installing rpy2 with the latest R 3.4.0 doesn't work on macOS. Apparently, R 3.4.0 was built using the '-fopenmp' flag with Clang 4.0.0, which is not supplied by Apple. Therefore, pip install rpy2 gives the following error:

    gcc -bundle -undefined dynamic_lookup -L/usr/local/anaconda/lib -L/usr/local/anaconda/lib -arch x86_64 build/temp.macosx-10.7-x86_64-3.6/./rpy/rinterface/_rinterface.o -L/usr/local/lib -L/usr/local/anaconda/lib -Lbuild/temp.macosx-10.7-x86_64-3.6 -L/usr/local/lib -lpcre -llzma -lbz2 -lz -licucore -lm -liconv -lr_utils -o build/lib.macosx-10.7-x86_64-3.6/rpy2/rinterface/_rinterface.cpython-36m-darwin.so -fopenmp -F/Library/Frameworks/R.framework/.. -framework R
    clang: error: unsupported option '-fopenmp'
    error: command 'gcc' failed with exit status 1

Everything worked fine with R 3.3.3 (which was built with the system Clang).

Comments (37)

  1. Laurent Gautier

    Disclaimer: I don't have a dev OS X to test anything of this.

    Could this is be because R was compiled with gcc rather than clang ? Did you try with gcc (easiest might be to install it with homebrew)

  2. Laurent Gautier
    • changed status to open

    An alternative might be to work out precompiled binaries for OS X but since I currently do not have access to a dev environment with OS X for rpy2, and since this would likely open up additional complexity because of R versions, this would have to be contributed by someone.

  3. Clemens Brunner reporter

    @pkulkarni2110 as a workaround, you can continue to use R 3.3.3, which is the last version that works with rpy2. @Laurent Gautier I have a Mac, but setting up the tools that were used to build R 3.4.0 could be a PITA. Unless I could just use the versions in Homebrew - I saw that they have LLVM version 4.0.0, but gcc is at 7.1.0 (and gcc@6 is 6.3.0 and not 6.1.0), so not sure it will be that easy. IMO it was quite an unfortunate decision from the R devs not to use Apple provided tools.

  4. Clemens Brunner reporter

    @Laurent Gautier either way this situation is not ideal, because it might entail having two pip packages for macOS depending on which R version you want to connect to. I don't think that this is even possible. Unless of course rpy2 built with the newer compiler versions also works with older R versions built with Apple-provided stuff.

  5. pkulkarni2110

    I installed R ver 3.3. I installed XQuartz as it was mentioned in the R webpage to install in macOs. I cannot certainly say it was helped to resolve the issue in anyway. Yes it is not ideal situation but it did work for temporarily.

  6. Clemens Brunner reporter

    Installing XQuartz is not necessary. I agree that this is just a temporary workaround - all future versions of R will be built with the newer tools, so eventually rpy2 will need to support it. MacOS High Sierra will likely ship with these updated build tools, so when rpy2 is built for this platform everything should hopefully work with R 3.4.0 and higher. Still, the question remains how rpy2 will handle older macOS versions. I guess we'd have to try if building with the newer tools produces a package that works both with R 3.4.0 as well as older versions such as R 3.3.3. I can try to build it and see if it works when I have time.

  7. Laurent Gautier

    @Clemens Brunner : I might be missing something here. How is this a support issue with rpy2 rather than an inconvenience of the R binaries on OS X ? rpy2 is getting its compilation flags (-fopenmp, presumably among other flags, here) from the installed R binary.

  8. Clemens Brunner reporter

    Because rpy2 doesn't work with R 3.4.0 or later, since setup.py requires a clang version that supports the -fopenmp flag. Since rpy2 gets the compiler flags from the R binary, is it possible to disable the -fopenmp flag if it is found? This would probably solve the issue.

  9. Clemens Brunner reporter

    Besides that, it is quite a bit of work to install the latest clang. I just tried it using Homebrew - and rpy2 installs fine! Here's what I did:

    brew install --with-toolchain llvm
    export PATH="/usr/local/opt/llvm/bin:$PATH"
    export LDFLAGS="-L/usr/local/opt/llvm/lib -Wl,-rpath,/usr/local/opt/llvm/lib"; pip3 install rpy2
    

    I'm not sure if we should expect rpy2 users on macOS to run through these steps (especially since the first step takes a long time because it compiles llvm from source).

  10. Laurent Gautier

    It might install, and may be even run, but that not something I'd like to hardcode in setup.py (which would mean support it). We can leave your workaround, someone may want to contribute "wheel" binaries for OS X, but otherwise building from source will require the toolchain the R binary itself was built with.

  11. Clemens Brunner reporter

    You mean the workaround using R < 3.4.0? Since I have the toolchain already setup I could provide a binary wheel for macOS. Two issues I see here:

    1. The binary wheel works only with R >= 3.4.0 - can pip determine which R version the user has installed and decide which wheel to install?
    2. I don't know how to create a binary wheel, maybe you could give me some instructions.
  12. Clemens Brunner reporter

    OK, but so far the source wheels worked and you only need a binary wheel for macOS using R >= 3.4.0 and all sorts of Python versions. The minimum should be Python 3.6 here, so this would be only one wheel if patch version levels are not important (i.e. Python 3.6 works like 3.6.1, similarly for R 3.4.0 and 3.4.1 etc). Let me know if you need anything from me to create this binary wheel.

  13. David Grunderson

    To piggyback of this issue, I have also been experiencing this same bug, though I am using R version 3.4 and Python version 2.7. would there be any way to get rpy2 to function with Python 2.7? I have been experiencing a great number of difficulties in trying to install rpy2, despite having tried pip, easy_install, and downloading the source tar. Despite my R being up to date and located in the PATH environmental variable, I have consistently received the same error:

      clang: error: unsupported option '-fopenmp'
        error: command 'cc' failed with exit status 1
    

    I have tried numerous approaches to solving this, including downloading XCode for OSX, as well as importing architecture flags, such as:

    env ARCHFLAGS="-arch i386 -arch x86_64" pip install rpy2
    

    I have also tried to installing the latest GCC compilers, but that made no change on the error yielded during rp2's installation process. Then I tried implementing commands suggested by Clemens Brunner above, and found that they worked perfectly, but only for Python 3.4. Is there any other approach you would recommend in order for rpy2 to function with Python 2.7?

  14. Clemens Brunner reporter

    Another much simpler option is to remove the -fopenmp linker flag (at least on macOS, I have no idea if this flag works on Windows). I just added the following lines to setup.py after line 272:

    if "-fopenmp" in unknown:  # remove linker argument
        unknown.remove("-fopenmp")
    

    Lo and behold, everything works just fine with the latest R 3.4.1 (all tests pass). But of course I'm not using OpenMP.

  15. Dan Li

    I have followed @ Clemens Brunner and run the following three command. I installed llvm, set the environment in .bash_profile, and source it to activate the change. I installed rpy2 in python3. But I still got the same error.

    brew install --with-toolchain llvm
    export PATH="/usr/local/opt/llvm/bin:$PATH"
    export LDFLAGS="-L/usr/local/opt/llvm/lib -Wl,-rpath,/usr/local/opt/llvm/lib"; pip3 install rpy2
    

    I also tried to install OpenMPI in http://thecoatlessprofessor.com/programming/openmp-in-r-on-os-x/#after-3-4-0. But it didn't help.

    It is so annoying. Can anyone give me some help?

  16. Clemens Brunner reporter

    @Dan Li try uninstalling and then re-installing by ignoring the cache:

    pip3 uninstall rpy2
    export PATH="/usr/local/opt/llvm/bin:$PATH"
    export LDFLAGS="-L/usr/local/opt/llvm/lib -Wl,-rpath,/usr/local/opt/llvm/lib"
    pip3 install --no-cache-dir rpy2
    
  17. Dan Li

    Sorry, I still get the error.

        clang: error: unsupported option '-fopenmp'
        error: command 'gcc' failed with exit status 1
    
        ----------------------------------------
    Command "/Users/danli/anaconda/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/gn/y3k01wts6yvg9z_m0zlqnllm0000gn/T/pip-build-viF4iB/rpy2/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/gn/y3k01wts6yvg9z_m0zlqnllm0000gn/T/pip-XQWVG4-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/gn/y3k01wts6yvg9z_m0zlqnllm0000gn/T/pip-build-viF4iB/rpy2/
    
  18. Dan Li

    Hi, Clemens Brunner. Here is the info.

    DanmatoMacBook-Pro:paper danli$ which clang
    /usr/local/opt/llvm/bin/clang
    DanmatoMacBook-Pro:paper danli$ clang --version
    clang version 4.0.1 (tags/RELEASE_401/final)
    Target: x86_64-apple-darwin16.7.0
    Thread model: posix
    InstalledDir: /usr/local/opt/llvm/bin
    
  19. Clemens Brunner reporter

    Somehow the build process still uses gcc instead of clang. I don't know why this worked for me out of the box, but you could either symlink gcc to clang (and g++ to clang++), or explicitly tell pip to use clang (this is actually where I'll stop, because I have no experience with distutils, but you could try to go from here or here). Alternatively, you could try the simpler option I mentioned above (add the two lines to setup.py and install via python3 setup.py install). Lastly, if you install the package directly without pip, you could also try python3 setup.py install --compiler=clang.

  20. Clemens Brunner reporter

    BTW, although Apple has updated clang in High Sierra, they have disabled the fopenmp option. So unfortunately this means that installing rpy2 on a Mac via pip still doesn't work out of the box (if the latest R version is installed).

  21. Sanjiv Das

    I have tried everything to install rpy2 on High Sierra running R3.4.1 and Python3.6 (anaconda). Btw, downgrading to R3.3.3 also does not work. I get the following error:

    unable to execute 'x86_64-apple-darwin13.4.0-ar': No such file or directory error: command 'x86_64-apple-darwin13.4.0-ar' failed with exit status 1

    I think its related to the same issue in this thread, but if not, and there is hope for a fix, please let me know. I ran into this problem only after upgrading OSX to High Sierra.

  22. Log in to comment