Issue #141 resolved

clang: error: unknown argument: '-mno-fused-madd'

ryh
created an issue
pip install cffi                                                     
Downloading/unpacking cffi
  Downloading cffi-0.8.2.tar.gz (197kB): 197kB downloaded
  Running setup.py egg_info for package cffi
    OS/X: confusion between 'cc' versus 'gcc' (see issue 123)
    will not use '__thread' in the C code

Downloading/unpacking pycparser (from cffi)
  Downloading pycparser-2.10.tar.gz (206kB): 206kB downloaded
  Running setup.py egg_info for package pycparser

Installing collected packages: cffi, pycparser
  Running setup.py install for cffi
    OS/X: confusion between 'cc' versus 'gcc' (see issue 123)
    will not use '__thread' in the C code
    building '_cffi_backend' extension
    cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -mno-fused-madd -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/usr/local/Cellar/libffi/3.0.13/lib/libffi-3.0.13/include -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c c/_cffi_backend.c -o build/temp.macosx-10.9-intel-2.7/c/_cffi_backend.o
    clang: error: unknown argument: '-mno-fused-madd' [-Wunused-command-line-argument-hard-error-in-future]
    clang: note: this will be a hard error (cannot be downgraded to a warning) in the future
    error: command 'cc' failed with exit status 1
    Complete output from command /Users/Bigfish/dev/python/CairoSVG/.venv/bin/python -c "import setuptools;__file__='/Users/Bigfish/dev/python/CairoSVG/.venv/build/cffi/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/0n/xrqqr8c957d_0mz0y2rp_wz00000gn/T/pip-5zVZJi-record/install-record.txt --single-version-externally-managed --install-headers /Users/Bigfish/dev/python/CairoSVG/.venv/include/site/python2.7:
    OS/X: confusion between 'cc' versus 'gcc' (see issue 123)

will not use '__thread' in the C code

running install

running build

running build_py

creating build

creating build/lib.macosx-10.9-intel-2.7

creating build/lib.macosx-10.9-intel-2.7/cffi

copying cffi/__init__.py -> build/lib.macosx-10.9-intel-2.7/cffi

copying cffi/api.py -> build/lib.macosx-10.9-intel-2.7/cffi

copying cffi/backend_ctypes.py -> build/lib.macosx-10.9-intel-2.7/cffi

copying cffi/commontypes.py -> build/lib.macosx-10.9-intel-2.7/cffi

copying cffi/cparser.py -> build/lib.macosx-10.9-intel-2.7/cffi

copying cffi/ffiplatform.py -> build/lib.macosx-10.9-intel-2.7/cffi

copying cffi/gc_weakref.py -> build/lib.macosx-10.9-intel-2.7/cffi

copying cffi/lock.py -> build/lib.macosx-10.9-intel-2.7/cffi

copying cffi/model.py -> build/lib.macosx-10.9-intel-2.7/cffi

copying cffi/vengine_cpy.py -> build/lib.macosx-10.9-intel-2.7/cffi

copying cffi/vengine_gen.py -> build/lib.macosx-10.9-intel-2.7/cffi

copying cffi/verifier.py -> build/lib.macosx-10.9-intel-2.7/cffi

running build_ext

building '_cffi_backend' extension

creating build/temp.macosx-10.9-intel-2.7

creating build/temp.macosx-10.9-intel-2.7/c

cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -mno-fused-madd -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/usr/local/Cellar/libffi/3.0.13/lib/libffi-3.0.13/include -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c c/_cffi_backend.c -o build/temp.macosx-10.9-intel-2.7/c/_cffi_backend.o

clang: error: unknown argument: '-mno-fused-madd' [-Wunused-command-line-argument-hard-error-in-future]

clang: note: this will be a hard error (cannot be downgraded to a warning) in the future

error: command 'cc' failed with exit status 1

in virtualenv 1.10.1 Python 2.7.5 pip 1.4.1 setuptools 0.9.8 Mac OS X 10.9.2 Apple LLVM version 5.1 (clang-503.0.38) (based on LLVM 3.4svn)

Comments (15)

  1. Lisandro Dalcin

    Here you have a fix, as well as a better implementation of ask_supports_thread()

    diff -r ddbfc6f0300b setup.py
    --- a/setup.py  Sat Mar 15 07:53:51 2014 +0100
    +++ b/setup.py  Sat Mar 15 12:55:03 2014 +0300
    @@ -41,26 +41,31 @@
                 #
                 resultlist[:] = res
    
    +if sys.platform =='darwin':
    +    from distutils import sysconfig, ccompiler
    +    sysconfig_customize_compiler = sysconfig.customize_compiler
    +    def customize_compiler(compiler):
    +        print compiler
    +        sysconfig_customize_compiler(compiler)
    +        if sys.platform == 'darwin':
    +            while '-mno-fused-madd' in compiler.compiler:
    +                compiler.compiler.remove('-mno-fused-madd')
    +            while '-mno-fused-madd' in compiler.compiler_so:
    +                compiler.compiler_so.remove('-mno-fused-madd')
    +            while '-mno-fused-madd' in compiler.linker_so:
    +                compiler.linker_so.remove('-mno-fused-madd')
    +    sysconfig.customize_compiler = customize_compiler
    +    ccompiler.customize_compiler = customize_compiler
    +
     def ask_supports_thread():
    -    if sys.platform == "darwin":
    -        sys.stderr.write("Note: will not use '__thread' in the C code\n")
    -        sys.stderr.write("This is for OS/X-specific reasons: confusion "
    -                         "between 'cc' versus 'gcc' (see issue 123)\n")
    -        return
    -    import distutils.errors
    -    from distutils.ccompiler import new_compiler
    -    compiler = new_compiler(force=1)
    -    try:
    -        compiler.compile(['c/check__thread.c'])
    -    except distutils.errors.CompileError:
    -        sys.stderr.write("Note: will not use '__thread' in the C code\n")
    -        sys.stderr.write("The above error message can be safely ignored\n")
    +    from distutils.core import Distribution
    +    config = Distribution().get_command_obj('config')
    +    ok = config.try_compile('__thread int some_threadlocal_variable_42;')
    +    if ok:
    +        define_macros.append(('USE__THREAD', None))
         else:
    -        define_macros.append(('USE__THREAD', None))
    -    try:
    -        os.unlink('c/check__thread.o')
    -    except OSError:
    -        pass
    +        sys.stderr.write("the above error message can be safely ignored;\n")
    +        sys.stderr.write("will not use '__thread' in the C code\n")
    
     def use_pkg_config():
         _ask_pkg_config(include_dirs,       '--cflags-only-I', '-I', sysroot=True)
    
  2. ryh reporter

    Lisandro Dalcin looks great,but there is more, while install gevent (dep:greenlet) it output clang: error: unknown argument: '-fno-tree-dominator-opts' [-Wunused-command-line-argument-hard-error-in-future]

    should we submit the issue to distutils? or had to PATCH every package's setup.py one by one?

  3. Lisandro Dalcin

    I think this is not a distutils issue. It is related to how system python was built in OS X. What OS X version are you running? Is -fno-tree-dominator-opts listed in the Makefile (use distutils.sysconfig.get_makefile_filename() to get full path) ?

  4. Armin Rigo

    "better implementation of ask_supports_thread()": are you sure it fixes properly the issue 123? I'm not keen to accept something which might break a previously-resolved issue if I can't test it myself. Please confirm that everything still works on OS X 10.7...

  5. Lisandro Dalcin

    Armin Rigo I do not have access to an OS X 10.7 box, but the original code that you "fixed" (by simply disabling the test for __thread) was plain wrong. When you create a new compiler instance with new_compiler(), it just contains the default name for the C compiler in a POSIX platform (i.e. cc), instead of the compiler actually used to build Python. My proposed implementation of ask_supports_thread() will use the right compiler command, setup the same way as the build_ext distutils command.

    Do any of you have OS X 10.7 to confirm my code works as expected?

    PS: My previous patch as a bogus "print compiler" line, sorry about that.

  6. Lisandro Dalcin

    I've found an issue with my patch in homebrewed Python 3. I've tried to attach the patch, but I cannot make it succeed. Is attaching files supposed to work here? Here you have the update.

    diff -r ddbfc6f0300b setup.py
    --- a/setup.py  Sat Mar 15 07:53:51 2014 +0100
    +++ b/setup.py  Mon Mar 17 00:28:44 2014 +0300
    @@ -41,26 +41,31 @@
                 #
                 resultlist[:] = res
    
    +if sys.platform =='darwin':
    +    from distutils import sysconfig, ccompiler
    +    sysconfig_customize_compiler = sysconfig.customize_compiler
    +    def customize_compiler(compiler):
    +        sysconfig.get_config_vars()
    +        sysconfig_customize_compiler(compiler)
    +        if sys.platform == 'darwin':
    +            while '-mno-fused-madd' in compiler.compiler:
    +                compiler.compiler.remove('-mno-fused-madd')
    +            while '-mno-fused-madd' in compiler.compiler_so:
    +                compiler.compiler_so.remove('-mno-fused-madd')
    +            while '-mno-fused-madd' in compiler.linker_so:
    +                compiler.linker_so.remove('-mno-fused-madd')
    +    sysconfig.customize_compiler = customize_compiler
    +    ccompiler.customize_compiler = customize_compiler
    +
     def ask_supports_thread():
    -    if sys.platform == "darwin":
    -        sys.stderr.write("Note: will not use '__thread' in the C code\n")
    -        sys.stderr.write("This is for OS/X-specific reasons: confusion "
    -                         "between 'cc' versus 'gcc' (see issue 123)\n")
    -        return
    -    import distutils.errors
    -    from distutils.ccompiler import new_compiler
    -    compiler = new_compiler(force=1)
    -    try:
    -        compiler.compile(['c/check__thread.c'])
    -    except distutils.errors.CompileError:
    -        sys.stderr.write("Note: will not use '__thread' in the C code\n")
    -        sys.stderr.write("The above error message can be safely ignored\n")
    +    from distutils.core import Distribution
    +    config = Distribution().get_command_obj('config')
    +    ok = config.try_compile('__thread int some_threadlocal_variable_42;')
    +    if ok:
    +        define_macros.append(('USE__THREAD', None))
         else:
    -        define_macros.append(('USE__THREAD', None))
    -    try:
    -        os.unlink('c/check__thread.o')
    -    except OSError:
    -        pass
    +        sys.stderr.write("the above error message can be safely ignored;\n")
    +        sys.stderr.write("will not use '__thread' in the C code\n")
    
     def use_pkg_config():
         _ask_pkg_config(include_dirs,       '--cflags-only-I', '-I', sysroot=True)
    
  7. Armin Rigo

    As a general rule, adding custom logic in setup*.py to work around a general bug on a specific platform is a no-no for me. I would much rather point people to the general place that describes the bug, describes known workarounds, and pushes forward to get this fixed. If you can give me a link to such a place, I can put it on the doc page.

    The ask_support_thread() part looks good to me, accepting this.

  8. Lisandro Dalcin

    Your rule is OK and I understand your concerns, but the thing is that end users experience a lot of trouble to figure out the issue and eventually fix it (when they can).

    The definitive fix for OS X is to ask users to find the installed Makefiles on the Python versions they want to use, e.g:

    $ python -c 'from distutils import sysconfig; print(sysconfig.get_makefile_filename())'
    /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config/Makefile
    

    and then edit such Makefile to remove any offending compiler flag from the CFLAGS variable. Of course, editing the Makefile requires root access.

    Alternatively, users can set empty CFLAGS in the environment, and it will still work, but then you risk removing important compiler flags (e.g, -fwrapv or -Os) :

    CFLAGS= python setup.py build
    

    Is there anything I can do to convince you to do something about it? What would be acceptable for you? Of course you can add docs and links describing the issue, but this just push the problem on user's shoulders, and we are talking about a problem that can be fixed with a few lines of code.

    How about at least the following: Let's write a function similar to ask_supports_thread() that just tries to compile a simple code, e.g, config.try_compile('int some_variable_42;'), if that fails, we can print in the console a helpful error message saying that the compiler is not working, and asking the users to either edit the Makefile or set an empty CFLAGS environment variable. What do you think?

  9. Mikhail Korobov

    Lisandro: accepted answer from http://stackoverflow.com/questions/22313407/clang-error-unknown-argument-mno-fused-madd-python-package-installation-fa works fine for me.

    The problem could be fixed for cffi using a few lines of code, but then every other Python project that uses C/C++ compiler should also add this workaround to its setup.py. Some of these projects could be inactive, so this will take time, and some of them may not get updated ever. So it seems better to fix the issue on user's side, once for all packages.

    Setting CFLAGS (as suggested at stackoverflow) or using Python installed using homebrew fixes the issue.

  10. Armin Rigo

    I'm fine with the try_compile() solution, and if that fails, explain what the problem is to the user: e.g. how to tweak CFLAGS (setting it empty or otherwise to a value that makes sense for clang). We can explain the problem and point for example to the stackoverflow question as one possible answer. That seems a much better fix than randomly removing flags on one particular platform (which will likely be fixed anyway soon, we hope).

  11. Log in to comment