Problem when building python module with intel

[Davide Del Vento]

I'm building shtns with intel compiler v16.0.0 and its corresponding MKL v11.3.0

To do so, I'm setting

export CC=icc

and configuring with

./configure --enable-mkl --enable-python --prefix=$MY_INSTALL_DIR

Running make works fine as far as the build is concerned, but fail at link stage with

gcc -pthread -shared build/temp.linux-x86_64-2.7/shtns_numpy_wrap.o sht_init.o sht_fly.o sht_omp.o sht_mem.o -l-mkl -lrt -lm -lpython2.7 -o build/lib.linux-x86_64-2.7/_shtns.so /usr/bin/ld: cannot find -l-mkl

(I've removed some -L specific to my install to make the problem more evident).

Manually running:

icc -pthread -shared build/temp.linux-x86_64-2.7/shtns_numpy_wrap.o sht_init.o sht_fly.o sht_omp.o sht_mem.o -mkl -lrt -lm -lpython2.7 -o build/lib.linux-x86_64-2.7/_shtns.so

Fixes the issue, I wish why autotool did not pick that by itself.

[Davide Del Vento]

Also, not the -pthread, possibly related with issue#12

[Nathanaël Schaeffer]

Hi, thanks for the report.

When building SHTns for python, there are actually two build systems involved.

• The first relies on autotools, and follows your instruction about using icc.

• The second is the python build system, Distutils, and I think you cannot specify a compiler; it will always use the compiler used for compilation of Python...

The recommended way is then to stick to gcc if you build for python... If your method of manually running icc works, it is good to know anyway.

Also, the -pthread is also coming from the way your python was build, and I do not know a way to override it (same as issue#12)

[Davide Del Vento]

Thank you for your explanation.

I am not an expert in Distutils, but I use it extensively to install python packages on our supercomputers. Quite often, it follows my "nudges" and uses icc/ifortran instead of gcc/gfortran, even though the python in use is the one I compiled from source with gcc. Most notably numpy/scipy are compiled with Intel. So it is possible (but I don't know exactly how, from a developer point of view).

I'm fine to stick to my workaround for my installs, but I suggest that you consider looking into this. For a "high performance" library such as SHTns, intel compiler (and especially intel v16) provides quite a boost of performance compared to gnu (moreover, MKL is much easier to use with the Intel compiler than with gcc)

Cheers, Davide

[Ankit Barik]

Hello,

I'm having the same problem. The cause seems to be the MKL library, not the compiler. More specifically, in setup.py, the libraries are split based on '-l' which leads to the following library list: ['-mkl', 'rt', 'm'].

This, in turn, is linked by placing '-l' in front of each of the entries leading to the link '-l-mkl' which is invalid, it should simply be '-mkl'. That's why running the command manually works fine, but not with the setup.py. In addition, the installation also works fine with intel compilers but without MKL.

Cheers, Ankit

[Ankit Barik]

The solution I found is as follows (probably a more elegant solution exists):

The libraries are detected automatically using autotools, hence, including them in "cargs" instead of separately in "libs" seems to do the job fine.

Thus, the sections of the code setup.py.in would look like:

cargs = "@OPENMP_CFLAGS@ @LIBS@"
largs = "@LDFLAGS@"

shtns_module = Extension('_shtns', sources=['shtns_numpy_wrap.c'],
        extra_objects=shtns_o, depends=shtns_o,
        extra_compile_args=cargs.split(),
        extra_link_args=largs.split(),
        library_dirs=libdir,
#       libraries=libslist,
        include_dirs=[numpy_inc])

omitting 'libraries= ' all together.

Hope this helps.

Cheers, Ankit

[Nathanaël Schaeffer]

Hello Ankit,

Thank you very much for the feedback. I think I used the "libraries" parameter for a reason, but I don't remember why. That's why I propose the following patch. Could you try it ?

largs = "@LDFLAGS@"
libs = "@LIBS@"
libslist = libs.replace('-l','').split()    # transform to list of libraries
if '-mkl' in libslist:
    libslist.remove('-mkl')     # this is not a library...
    cargs += ' -mkl'            # ...but rather a compilation option for icc

[Ankit Barik]

Hello Nathanael,

Yes, that does seem like a better solution.

I tried it out - it builds and installs the extension fine for both with and without mkl.

In addition, I noticed an additional problem, when you use gcc with mkl, you get additional options -Wl, --start-group and -Wl, --end-group. I tried the following patch and it works with all possible compiler and library combinations: icc + mkl, icc + fftw, gcc + mkl, gcc + fftw.

if '-mkl' in libslist:
        libslist.remove('-mkl')     # this is not a library...
        cargs += ' -mkl'            # ...but rather a compilation option for icc

if '-Wl,--start-group' in libslist:
        libslist.remove('-Wl,--start-group')     # this is not a library...
        cargs += ' -Wl,--start-group'            # ...but rather a compilation option for gcc

if '-Wl,--end-group' in libslist:
        libslist.remove('-Wl,--end-group')     # this is not a library...
        cargs += ' -Wl,--end-group'            # ...but rather a compilation option for gcc

[Nathanaël Schaeffer]

Thank you. In addition to compiling did you try to actually import the compiled shtns module from python ?

[Ankit Barik]

Hello Nathanael,

Sorry for the late reply, got a bit busy with a paper (you might be happy to hear that it uses shtns and XHELLS).

You're right, importing the thing doesn't work that well. In particular, after applying the patch above and trying to import, I observe the following:

• gcc + fftw: no problem
• gcc + mkl: no problem
• icc + fftw: causes the following error

ImportError                               Traceback (most recent call last)
<ipython-input-1-e2f467cc64ae> in <module>()
----> 1 import shtns

/home/ankit/soft/shtns/shtns.py in <module>()
     28                 fp.close()
     29             return _mod
---> 30     _shtns = swig_import_helper()
     31     del swig_import_helper
     32 else:

/home/ankit/soft/shtns/shtns.py in swig_import_helper()
     20             fp, pathname, description = imp.find_module('_shtns', [dirname(__file__)])
     21         except ImportError:
---> 22             import _shtns
     23             return _shtns
     24         if fp is not None:

ImportError: /home/ankit/.local/lib/python2.7/site-packages/_shtns.so: undefined symbol: __svml_cos4

which I fixed by using adding -lsvml -lirc -limf to libs. I don't know if anyone else faced this problem.

• icc + mkl: After adding -lsvml -lirc -limf to libs to take care of the above error, I get the following error:

In [1]: import shtns
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-e2f467cc64ae> in <module>()
----> 1 import shtns

/home/ankit/soft/shtns/shtns.py in <module>()
     28                 fp.close()
     29             return _mod
---> 30     _shtns = swig_import_helper()
     31     del swig_import_helper
     32 else:

/home/ankit/soft/shtns/shtns.py in swig_import_helper()
     20             fp, pathname, description = imp.find_module('_shtns', [dirname(__file__)])
     21         except ImportError:
---> 22             import _shtns
     23             return _shtns
     24         if fp is not None:

ImportError: /home/ankit/.local/lib/python2.7/site-packages/_shtns.so: undefined symbol: fftw_plan_many_dft_r2c

but if I add -lmkl_intel_lp64 -lmkl_sequential -lmkl_core to libs, it gets fixed and the module imports fine.

Could you think of a clever patch that could take care of the above problems? The brute force way would be to check for icc and mkl and include the relevant options, but that's not very elegant I think.