Installing slepc4py
Hi,
I'm using Ubuntu 12.04 LTS When I try to install slepc4py using pip, this is the debug log I get:
/usr/local/bin/pip run on Wed May 14 10:55:49 2014 Downloading/unpacking https://bitbucket.org/slepc/slepc4py/downloads/slepc4py-3.4.tar.gz Downloading from URL https://bitbucket.org/slepc/slepc4py/downloads/slepc4py-3.4.tar.gz Running setup.py (path:/tmp/pip-IFVaAN-build/setup.py) egg_info for package from https://bitbucket.org/slepc/slepc4py/downloads/slepc4py-3.4.tar.gz Traceback (most recent call last): File "<string>", line 17, in <module> File "/tmp/pip-IFVaAN-build/setup.py", line 248, in <module> main() File "/tmp/pip-IFVaAN-build/setup.py", line 245, in main run_setup() File "/tmp/pip-IFVaAN-build/setup.py", line 117, in run_setup ext_modules = get_ext_modules(Extension), File "/tmp/pip-IFVaAN-build/setup.py", line 77, in get_ext_modules petsc4py_includes = [petsc4py.get_include()] AttributeError: 'module' object has no attribute 'get_include' Complete output from command python setup.py egg_info: Traceback (most recent call last):
File "<string>", line 17, in <module>
File "/tmp/pip-IFVaAN-build/setup.py", line 248, in <module>
main()
File "/tmp/pip-IFVaAN-build/setup.py", line 245, in main
run_setup()
File "/tmp/pip-IFVaAN-build/setup.py", line 117, in run_setup
ext_modules = get_ext_modules(Extension),
File "/tmp/pip-IFVaAN-build/setup.py", line 77, in get_ext_modules
petsc4py_includes = [petsc4py.get_include()]
AttributeError: 'module' object has no attribute 'get_include'
Similar problem when I use python seup.py build Any help would be appreciated
Thanks, Kim
Comments (35)
-
-
reporter - attached screen.log
-
reporter Thanks for the reply, I just installed the newer petsc4py by directly linking it to the download. Now when I try to install slepc4py, I get that output on the attached file
-
Have you installed petsc in /usr/lib ? From the log I'm seing the following:
SLEPC_DIR: /usr/lib/slepc PETSC_DIR: /home/kim/Downloads/petsc-3.4.4 PETSC_ARCH: arch-linux2-c-debug
So you likely have PETSC_DIR and PETSC_ARCH in your environment pointing to a the PETSc source build directory, however SLEPc seems to be installed in prefix /usr/lib/slepc.
How did you get PETSc and SLEPc? Do the PETSc and SLEPc versions match? While "pip" could eventually fail, using setup.py should succeed, unless you have broken dependencies.
-
reporter To install PETSc and SLEPc I used the method on their websites, using ./configure
I changed PETSC_DIR to /usr/lib/petsc but still get the same error when I use pip. When I use setup.py, I get the attached error
-
reporter - attached out.txt
-
Did you switch to use C++ only builds of PETSc and SLEPc? Is this a pip install?
-
reporter Sorry, I'm really new to all of this stuff. How would you "switch to use C++ only builds of PETSc and SLEPc" ? And no, I used
python setup.py build
Using pip would give me the same error as the first log attachment
-
My question was related to how you built PETSc and SLEPc. I guess you only want SLEPc and PETSc to use them from Python, right? In such a case, please remove your PETSc and SLEPc install from /usr/lib, remove PETSC_DIR PETSC_ARCH and SLEPC_DIR from your environment variables, make 100% sure you do not have any other PETSc/SLEPc/petsc4py/slepc4py lying around, and try the following (please use the same install order as below):
pip install --user http://ftp.mcs.anl.gov/pub/petsc/release-snapshots/petsc-lite-3.4.4.tar.gz pip install --user http://www.grycap.upv.es/slepc/download/distrib/slepc-3.4.4.tar.gz pip install --user https://bitbucket.org/petsc/petsc4py/downloads/petsc4py-3.4.tar.gz pip install --user https://bitbucket.org/slepc/slepc4py/downloads/slepc4py-3.4.tar.gz
-
reporter Thanks a lot, everything was installed, I think it was because I had some other petsc. Just one thing I'm having trouble with. I tried running ex1.py in the slepc4py demo folder but it gives me an ImportError: No module named PETSc
-
Oh, were you using Anaconda Python? Perhaps it does not play well with the --user option to pip. If you use Anaconda, then the --user option is not really important (but it is if you are using the OS-provided Python, as you risk making a mess on system locations, like underneath /usr/... )
What happens if you enter the Python prompt and execute "from petsc4py import PETSc" and "from slepc4py import SLEPc"?
-
reporter Yes I am using Anaconda Python.
Here is what happens:
from petsc4py import PETSc Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/kim/.local/lib/python2.7/site-packages/petsc4py/PETSc.py", line 3, in <module> PETSc = ImportPETSc(ARCH) File "/home/kim/.local/lib/python2.7/site-packages/petsc4py/lib/init.py", line 29, in ImportPETSc return Import('petsc4py', 'PETSc', path, arch) File "/home/kim/.local/lib/python2.7/site-packages/petsc4py/lib/init.py", line 64, in Import module = imp.load_module(fullname, fo, fn, stuff) ImportError: libmpi.so.0: cannot open shared object file: No such file or directory from slepc4py import SLEPc Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/kim/.local/lib/python2.7/site-packages/slepc4py/SLEPc.py", line 4, in <module> SLEPc = ImportSLEPc(ARCH) File "/home/kim/.local/lib/python2.7/site-packages/slepc4py/lib/init.py", line 33, in ImportSLEPc PETSc = ImportPETSc(arch) File "/home/kim/.local/lib/python2.7/site-packages/petsc4py/lib/init.py", line 29, in ImportPETSc return Import('petsc4py', 'PETSc', path, arch) File "/home/kim/.local/lib/python2.7/site-packages/petsc4py/lib/init.py", line 63, in Import fo, fn, stuff = imp.find_module(name, pathlist) ImportError: No module named PETSc
-
Oh, the actual problem is that the dynamic linker cannot find the MPI libraries. This sould be easy to fix, try the following:
export LD_LIBRARY_PATH=/usr/lib/openmpi/lib
or whatever is the place you have Open MPI installed ("mpicc -show" should tell you)
Then try again to import PETSc,
PS: Did you install openmpi yourself? In general, installing in /usr/lib is a very bad idea. Also, I believe mpich and mpi4py are available in anaconda, it would be better to cleanup your system and try to use MPI from Anaconda,
-
reporter I had OpenMPI before but then deleted it. To use MPI from Anaconda, do you just change the LS_LIBRARY_PATH to mpich folder in Anaconda?
I'm now getting this:
from petsc4py import PETSc
Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/kim/.local/lib/python2.7/site-packages/petsc4py/PETSc.py", line 3, in <module> PETSc = ImportPETSc(ARCH) File "/home/kim/.local/lib/python2.7/site-packages/petsc4py/lib/init.py", line 29, in ImportPETSc return Import('petsc4py', 'PETSc', path, arch) File "/home/kim/.local/lib/python2.7/site-packages/petsc4py/lib/init.py", line 64, in Import module = imp.load_module(fullname, fo, fn, stuff) ImportError: libpetsc.so: cannot open shared object file: No such file or directory
-
Well, if you use MPICH from Anaconda, you will not need to set LD_LIBRARY_PATH. However, you need to rebuild everything from scratch, that is, run "pip uninstall petsc slepc petsc4py slepc4py", and next install them again.
-
reporter Okay so I uninstalled them and am trying to install again. Do I use the same method as what you posted above? I'm not sure if I have to specify to use the anaconda mpich or how to do that.
I tried installing petsc like before and I get this:
Incomplete BLAS install; Perhaps blas package is installed - but blas-dev/blas-devel is required?
-
Just make sure that "mpicc -show" works and produces god output (the -I and -L flags should point to paths within your Anaconda prefix install). After that, just install the packages one by one, this time without the --user option:
pip install http://ftp.mcs.anl.gov/pub/petsc/release-snapshots/petsc-lite-3.4.4.tar.gz pip install http://www.grycap.upv.es/slepc/download/distrib/slepc-3.4.4.tar.gz pip install https://bitbucket.org/petsc/petsc4py/downloads/petsc4py-3.4.tar.gz pip install https://bitbucket.org/slepc/slepc4py/downloads/slepc4py-3.4.tar.gz
and next let me know if it worked.
-
reporter mpicc -show looks good. I get the same error as I just posted above when I do
pip install http://ftp.mcs.anl.gov/pub/petsc/release-snapshots/petsc-lite-3.4.4.tar.gz
I'll attach the log
-
reporter - attached tmpvBtfpX
-
OK, for some reason you are not using MPICH from anaconda, but some MPI you have installed in /usr/local, look at the log:
configure options: PETSC_ARCH=linux-x86_64-python --with-shared-libraries=1 --with-debugging=0 --with-c2html=0 --with-cc=/usr/local/bin/mpicc --with-cxx=/usr/local/bin/mpicxx --with-fc=/usr/local/bin/mpif90
Please fix your PATH environ variable until "which mpicc" points to the Anaconda one.
-
BTW, what OS are you running on? If you are in Linux, please use your system package manager to install blas, lapack, blas-devel and lapack-devel packages.
PS: yes, you are in Linux, as I can see in the logs.
-
reporter Is there a way to manually make with-cc point to the anaconda mpich?
This is what I have:
kim@kim-Macmini:~$ which mpicc /home/kim/anaconda/bin/mpicc kim@kim-Macmini:~$ mpicc -show gcc -I/home/kim/anaconda/include -L/home/kim/anaconda/lib -Wl,-rpath,/home/kim/anaconda/lib -lmpich -lopa -lmpl -lrt -lpthread kim@kim-Macmini:~$ echo $PATH /home/kim/anaconda/bin:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
but it still goes to /usr/local/bin
Yes, I'm on Linux ubuntu 12.04 LTS
-
The paths that PETSc uses should came from mpi4py. Can you run this in the Python prompt? What do you get?
import mpi4py print mpi4py import pprint pprint.pprint(mpi4py.get_config())
-
reporter >>> import mpi4py >>> print mpi4py <module 'mpi4py' from '/home/kim/anaconda/lib/python2.7/site-packages/mpi4py/__init__.pyc'> >>> import pprint >>> pprint.pprint(mpi4py.get_config()) {'mpicc': '/home/kim/anaconda/bin/mpicc', 'mpicxx': '/home/kim/anaconda/bin/mpicxx', 'mpif77': '/home/kim/anaconda/bin/mpif77', 'mpif90': '/home/kim/anaconda/bin/mpif90'}
Also, I'm still getting a blas error after I installed the blas and lapack packages
-
reporter So I reinstalled anaconda and now it shows up in the compilers for petsc but I'm still getting the blas error
-
reporter - attached tmpt8N8Ej
-
Oh, now It makes sense. About blas and lapack, you should install them from the system package manager, after that the PETSc installation should go fine
sudo apt-get install libblas-dev liblapack-dev
-
reporter I did that and got the same error as the log above. Not sure why its not finding it..
-
reporter So, I managed to get them installed again by using ./configure for PETSc and SLEPc then pip install for petsc4py and slepc4py. I'm pretty sure its using the anaconda mpicc.
This is what happens when I import:
>>> from petsc4py import PETSc Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/kim/anaconda/lib/python2.7/site-packages/petsc4py/PETSc.py", line 3, in <module> PETSc = ImportPETSc(ARCH) File "/home/kim/anaconda/lib/python2.7/site-packages/petsc4py/lib/__init__.py", line 29, in ImportPETSc return Import('petsc4py', 'PETSc', path, arch) File "/home/kim/anaconda/lib/python2.7/site-packages/petsc4py/lib/__init__.py", line 64, in Import module = imp.load_module(fullname, fo, fn, stuff) ImportError: /usr/local/lib/libmpich.so.12: undefined symbol: MPL_large_writev
-
Do you have LD_LIBRARY_PATH set in your environment? Is it pointing to /usr/local? The problem here is that your PETSc libraries seem to link against MPICH libraries in /usr/local, and not the ones in Anaconda. Does the file /usr/local/lib/libmpich.so.12 actually exists in your system? Why it is there? Is it an old MPICH2 install with --prefix=/usr/local?
Can you try to
export LD_LIBRARY_PATH=/home/kim/anaconda/lib:$LD_LIBRARY_PATH
and try to " from petsc4py import PETSc" again?
-
reporter I think it was there from a previous install. I still get the same error when I set the LD_LIBRARY_PATH to anaconda
-
So, I can only guess that somehow the MPI libraries in /usr/local were picked in the PETSc and/or petsc4py install step. At this point, I'm a bit lost. The only thing I can suggest is to use apt-get to remove any traces of system-provided MPICH2 and/or Open MPI, and start from scratch building/installing PETSc, SLEPc, petsc4py, slepc4py.
Please note that this file /usr/local/lib/libmpich.so.12 is likely not from an apt-get install, but an MPI that you installed with --prefix=/usr/local. In such a case, you need to manually remove any files related to MPICH. Doing this by hand could be really tedious, because you have stuff from MPICH in /usr/local/bin, /usr/local/lib, /usr/local/etc, /usr/local/share, etc... One way to clean MPICH from /usr/local os to download the tarball with the exact same version you installed, ./configure --prefix=/usr/local, make && make install, and finally execute "make uninstall", this should hopefully clean all the files. If you still have around the source tree that you used to install MPICH, then go there directly and execute "make uninstall".
I'm sorry, but I think your system is likely a mess. In previous comments in this issue I saw you had stuff installed with prefix /usr/lib, now I see stuff in /usr/local...
I general, you should NEVER install underneath /opt/, /usr/ or even /usr/local. In my personal desktop, I have the convention of installing stuff using as prefix
/home/devel/<package>/<version>
. At this point, I think you should cleanup your system from custom installs, also remove any MPI you installed with apt-get, and once everything is clean, building PETSc, SLEPc, and **py packages should work out of the box. -
reporter Thanks a lot for your help and advice, I will keep it in mind in the future.
I cleaned up everything and installed then uninstalled mpich in /usr/local. Installed petsc/slepc/petsc4py/slepc4py again and tried ex1.py and it works. Thanks again and sorry for the trouble.
-
OK, good to know that you could make it work. I'm still wondering what was wrong in your system, but anyway you could clean things up and make it work.
-
- changed status to invalid
-
assigned issue to
- Log in to comment
Any chance you have an old version of petsc4py lying around? Please try to install using the following three separate pip invocations (the --user is optional, but highly recommended):