problem to run swig demo(Attempting to use an MPI routine before initializing MPICH)

[jay]

Hello:

I got following error when I compile the swig demo by virtualenv

/usr/bin/ld: /software/python/3.6.3/lib/libpython3.6m.a(abstract.o): relocation R_X86_64_32S against `_Py_NotImplementedStruct' can not be used when making a shared object; recompile with -fPIC /software/python/3.6.3/lib/libpython3.6m.a: error adding symbols: Bad value collect2: error: ld returned 1 exit status make: *** [_helloworld.cpython-36m-x86_64-linux-gnu.so] Error 1

the python version is python 3.6.3. It seems there is a similar problem here: https://stackoverflow.com/questions/50179694/usr-local-lib-libpython3-6m-a-error-adding-symbols-bad-value-collect2-error#

Do you have some clues about this?

Thanks a lot for your help.

[Lisandro Dalcin]

I your Python was not built with a shared libpython, then I guess you have to modify things in the demo makefile to make it work. Did you build Python yourself? Please note that the most common setup is to have shared Python library, that is the case for distro-provided Linux packages, Miniconda/Anadonca distributions, etc. However, AFAICR, if you build Python yourself from a tarball downloaded from python.org, you get a static lib with default configure flags.

[jay]

Thanks for your reply! I attached the Makefile that I used, maybe you could you give me some clues? thanks a lot for your help.

/////////////makefile//////////////

.PHONY: default
default: build test clean

PYTHON = python
PYTHON_CONFIG = ${PYTHON} ../python-config
MPI4PY_INCLUDE = ${shell ${PYTHON} -c 'import mpi4py; print( mpi4py.get_include() )'}


SWIG = swig
SWIG_PY = ${SWIG} -python
.PHONY: src
src: helloworld_wrap.c
helloworld_wrap.c: helloworld.i
    ${SWIG_PY} -I${MPI4PY_INCLUDE} -o $@ $<

MPICC = mpicc
CFLAGS = -fPIC ${shell ${PYTHON_CONFIG} --includes}
LDFLAGS = -shared ${shell ${PYTHON_CONFIG} --libs} -L/software/python/3.6.3/lib
SO = ${shell ${PYTHON_CONFIG} --extension-suffix}
.PHONY: build
build: _helloworld${SO}
_helloworld${SO}: helloworld_wrap.c
    ${MPICC} ${CFLAGS} -I${MPI4PY_INCLUDE} -o $@ $< ${LDFLAGS}


MPIEXEC = mpiexec
NP_FLAG = -n
NP = 5
.PHONY: test
test: build
    ${MPIEXEC} ${NP_FLAG} ${NP} ${PYTHON} test.py


.PHONY: clean
clean:
    ${RM} helloworld_wrap.c helloworld.py* _helloworld${SO}

[jay]

Hello, Lisandro Dalcin:

I compiled the demo successfully, but there are following errors:

Attempting to use an MPI routine before initializing MPICH Attempting to use an MPI routine before initializing MPICH ...

it seems the communicator instance can not be sent to c code, can you give me some clues about this? (python/3.5.1 Cray Linux Environment mpi4py/2.0.0)

Thanks for your help!

[Lisandro Dalcin]

That most provably means that your build environment is messed up. You linked the SWIG demo with an MPI library that is not the same as the one mpi4py uses, or maybe you are using static libraries libraries. You should run ldd on the mpi4py extension module MPI.so and the SWIG module, and check they are linked against the same MPI libraries.

[jay]

Got it, it is indeed different on my env, thanks a lot for your help!