is concurrent installation of real and complex petsc4py modules possible?
The structure of the PETSc installation makes it possible to install different versions and different configurations side by side. You can choose which one you want to work with at run time (or build time) using PETSC_DIR, choosing between real and complex number support, for instance.
Is it feasible to maintain a similar kind of concurrent installation for petsc4py? Installation of the modules themselves could be easy, e.g. rename as /usr/lib/python3/dist-packages/petsc4py_real and petsc4py_complex, perhaps provide petsc4py as a symlink to the preferred configuration. Then each of "import petsc4py_real", "import petsc4py_complex", "import petsc4py" would run in python.
But then the internals of the module still refer to petsc4py. So while "import petsc4py_complex" will present get_config(), get_include() and init(), and petsc4py_complex.init() will run without error, at the same time it does not load up petsc4py_complex.PETSc.
So only "import petsc4py" runs correctly, though its petsc configuration can be switched by swapping a petsc4py symlink in the python module dir between petsc4py_real and petsc4py_complex.
Would there be a straightforward way of activating petsc4py_real and petsc4py_complex modules so that different configurations can be used by different python applications (one might use import petsc4py_complex as petsc4py
for instance) at the same time?
Comments (16)
-
-
reporter Thanks Lisandro. The config step doesn't seem to be set up for PETSC_ARCH. e.g. conf/baseconf.py: petscversion_h = os.path.join(petsc_dir, 'include', 'petscversion.h'), petsc_arch is not invoked between petsc_dir and include. _get_petsc_version doesn't take petsc_arch as an option.
I'm trying to work with the Debian pybuild system, which calls setup.py config/build/install, each in turn.
-
Can you please provide a reproducible example of how things break? My suggestions apply for the case of you using a PETSc source tree where you have multiple archs built under a single $PETSC_DIR, and not for prefix-installations of PETSc (which maybe is the Debian case?). In any case, I'm totally fine with changing things to make this possible.
Can you reproduce the following command line? (of course, adjust PETSC_ARCH to fit your case):
$ PETSC_ARCH=arch-linux2-c-debug:arch-test python setup.py config running config ---------------------------------------------------------------------- PETSC_DIR: /home/devel/petsc/dev ---------------------------------------------------------------------- PETSC_ARCH: arch-linux2-c-debug * scalar-type: real * precision: double * language: CONLY * compiler: mpicc * linker: mpicc ---------------------------------------------------------------------- PETSC_ARCH: arch-test * scalar-type: complex * precision: double * language: CXXONLY * compiler: mpicxx * linker: mpicxx ----------------------------------------------------------------------
-
reporter You're right, Debian uses the prefix installation, so PETSC_ARCH is not used after installation (PETSC_ARCH="" in the installed petscconf.h). PETSC_DIR is /usr/lib/petscdir/petsc3.10/x86_64-linux-gnu-real or /usr/lib/petscdir/petsc3.10/x86_64-linux-gnu-complex.
If you interpret that as base PETSC_DIR + PETSC_ARCH then config says:
$ PETSC_DIR=/usr/lib/petscdir/petsc3.10 PETSC_ARCH=x86_64-linux-gnu-real:x86_64-linux-gnu-complex python3 setup.py config running config ---------------------------------------------------------------------- PETSC_DIR: /usr/lib/petscdir/petsc3.10 error: [Errno 2] No such file or directory: '/usr/lib/petscdir/petsc3.10/include/petscversion.h'
The full PETSC_DIR does work, but of course only handles one configuration at a time
$ PETSC_DIR=/usr/lib/petscdir/petsc3.10/x86_64-linux-gnu-real python3 setup.py config running config ---------------------------------------------------------------------- PETSC_DIR: /usr/lib/petscdir/petsc3.10/x86_64-linux-gnu-real ---------------------------------------------------------------------- PETSC_ARCH: * scalar-type: real * precision: double * language: CONLY * compiler: mpicc * linker: mpicc ----------------------------------------------------------------------
$ PETSC_DIR=/usr/lib/petscdir/petsc3.10/x86_64-linux-gnu-complex python3 setup.py config running config ---------------------------------------------------------------------- PETSC_DIR: /usr/lib/petscdir/petsc3.10/x86_64-linux-gnu-complex ---------------------------------------------------------------------- PETSC_ARCH: * scalar-type: complex * precision: double * language: CONLY * compiler: mpicc * linker: mpicc ----------------------------------------------------------------------
-
reporter We’ve now got a client program that can be built against PETSc with either real or complex number support. Dolfinx is the next-generation version of Dolfin for the FEniCS suite (https://github.com/FEniCS/dolfinx). It can be built for either real or complex numbers.
Ideally we’d want any system to be able to choose the real or complex number build for dolfinx without having to uninstall/reinstall the petsc4py packages. PETSc itself enables that via prefix installations and PETSC_DIR. It could be a good time therefore to revisit this Issue to see if we can do the same with petsc4py (controlled via PETSC_DIR rather than PETSC_ARCH).
-
This should be rather trivial to implement. What I’m not sure is the “interface” to ask
setup.py
to do it. Right now, you build for multiply arches withPETSC_ARCH=arch-a:arch-b:arch-c
. We could also supportPETSC_DIR=prefix-a:prefix-b:prefix-c
. But then, how do we let users select the version they want to use at runtime? -
reporter I think it's reasonable to expect users to set
PETSC_DIR
as an environment variable if they want to select the non-default installation at runtime. That makes it the same kind of handling from the user’s perspective as what you’ve currently got in place withPETSC_ARCH
Another option is to provide a switch inside petsc4py itself allowing progammatic change of build, something like
import petsc4py petsc4py.petsc_dir='/usr/lib/petscdir/petsc3.10/x86_64-linux-gnu-complex/'
where the value of
petsc4py.petsc_dir
would have to be one of the available builds, perhaps accessible bypetsc4py.list_available_petsc_dir()
But perhaps this second option is not so viable, since I imagine multiple PETSc inits would be a problem. -
reporter Unless (for that second option), you can adapt the mechanism you suggested with arch, to use something like
import petsc4py petsc4py.init(petsc_dir="/usr/lib/petscdir/petsc3.10/x86_64-linux-gnu-complex/")
instead of
import petsc4py petsc4py.init(arch="arch-complex")
-
The more I think about this, the less I like it. Your request is definitely non-standard. Multiple PETSC_ARCH is something that PETSc supports and has been carefully designed for, so it is natural for petsc4py to support it. But multiple prefix installs are a different beast. This should be best handled with environment modules (as done in Fedora), but IIRC that’s not an available option on Debian/Ubuntu.
In the same way that you install PETSc in two different prefixes, petsc4py should be installed in two different places (maybe inside each PETSC_DIR?), and then install a
*.pth
file in python site-packages to implement the logic of appending tosys.path
using the value of thePETSC_DIR
environ var. And most likely you will have to do something similar for Dolfinx. -
reporter We can make something like that work with a petsc4py_real, petsc4py_complex installation, alongside a petsc4py dummy module which selects the one needed according to the value of PETSC_DIR. I’ve been working on a selective import with a similar mechanism for h5py, selecting between serial and mpi build (cf. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=944769 )
Installing actually inside the PETSC_DIR is an interesting proposition. Not sure if it makes it too messy updating PYTHONPATH or keeping the list of available petsc4pys up to date to make it workable, but can look into it. Maybe a symlink from PYTHONPATH/petsc4py/<some_petsc_build> to the appropriate subdir in PETSC_DIR can make it work.
-
I hate these dummy modules. IMHO,
*.pth
files are a far better option.Try the following bash script on your side (minimal PETSc build, no MPI, no Fortran, assumes blas/lapack installed on the system, just to make it quick to build). As you can see, this is quite trivial to implement, it is robust and follow Python standard practice, it does not require changes to petsc4py, just some boring work on the packaging side (which I’m not interested in contributing to, sorry). Same approach could be implemented for any other Python package like Dolfinx wanting to support multiple PETSc builds. Now I’m done for good!
Disclaimer: Only tested on Fedora 31 with PETSc/petsc4py 3.12.
#!/bin/bash set -eu # For testing purposes, install stuff at $HOME ROOTDIR=$HOME # Download, build and install PETSc real/complex variants wget -q http://ftp.mcs.anl.gov/pub/petsc/release-snapshots/petsc-lite-3.12.4.tar.gz tar -zxf petsc-lite-3.12.4.tar.gz cd petsc-3.12.4 VARIANT=real export PETSC_DIR=$PWD export PETSC_ARCH=arch-$VARIANT ./configure --prefix=$ROOTDIR/petscdir/petsc3.12/x86_64-linux-gnu-$VARIANT --with-fc=0 --with-mpi=0 --with-scalar-type=$VARIANT make make install unset PETSC_DIR unset PETSC_ARCH VARIANT=complex export PETSC_DIR=$PWD export PETSC_ARCH=arch-$VARIANT ./configure --prefix=$ROOTDIR/petscdir/petsc3.12/x86_64-linux-gnu-$VARIANT --with-fc=0 --with-mpi=0 --with-scalar-type=$VARIANT make make install unset PETSC_DIR unset PETSC_ARCH cd .. # Download, build and install petsc4py real/complex variants wget -q https://bitbucket.org/petsc/petsc4py/downloads/petsc4py-3.12.0.tar.gz tar -xzf petsc4py-3.12.0.tar.gz cd petsc4py-3.12.0 VARIANT=real export PETSC_DIR=$ROOTDIR/petscdir/petsc3.12/x86_64-linux-gnu-$VARIANT python3 setup.py clean --all python3 setup.py install --install-lib=$PETSC_DIR/lib unset PETSC_DIR VARIANT=complex export PETSC_DIR=$ROOTDIR/petscdir/petsc3.12/x86_64-linux-gnu-$VARIANT python3 setup.py clean --all python3 setup.py install --install-lib=$PETSC_DIR/lib unset PETSC_DIR cd .. # Install petsc4py.pth file in user site-packages (~/.local/...) site_packages=$(python3 -m site --user-site) petsc4py_pth=$site_packages/petsc4py.pth mkdir -p $site_packages printf "import sys, os; " > $petsc4py_pth printf "p = '%s'; " $ROOTDIR/petscdir/petsc3.12/x86_64-linux-gnu-real >> $petsc4py_pth printf "p = os.path.join(os.getenv('PETSC_DIR') or p, 'lib'); " >> $petsc4py_pth printf "p in sys.path or sys.path.append(p);\n" >> $petsc4py_pth # Test petsc4py installations # PETSC_DIR not set python3 -c "from petsc4py import PETSc; print(PETSc); print(PETSc.ScalarType)" # PETSC_DIR set to real variant VARIANT=real export PETSC_DIR=$ROOTDIR/petscdir/petsc3.12/x86_64-linux-gnu-$VARIANT python3 -c "from petsc4py import PETSc; print(PETSc); print(PETSc.ScalarType)" # PETSC_DIR set to complex variant VARIANT=complex export PETSC_DIR=$ROOTDIR/petscdir/petsc3.12/x86_64-linux-gnu-$VARIANT python3 -c "from petsc4py import PETSc; print(PETSc); print(PETSc.ScalarType)" # Remember to cleanup your user site-package after testing! # rm $(python3 -m site --user-site)/petsc4py.pth
-
Note that the
petsc4py.pth
file need to have a single line with semicolon-separated statements! -
reporter Thanks Lisandro. I’m not familiar with .pth, I’ll take a close look.
-
reporter Seems to be working fine, working now in Debian petsc4py 3.12.0-6. Debian already defines /usr/lib/petsc as the default PETSc, so I used
import sys, os; petsc_dir = '/usr/lib/petsc'; petsc_dir = os.path.join(os.getenv('PETSC_DIR') or petsc_dir, 'lib/python3/dist-packages'); petsc_dir in sys.path or sys.path.append(petsc_dir);
Likewise for slepc4py.
I guess we can close this Issue, unless it’s worth working petsc4py.pth and installation under PETSC_DIR into the main source code.
-
I don’t think it is worth it to add these pth files to petsc4py, they are more or less trivial, and its contents will depend on specific packaging details of Linux distros.
-
- changed status to closed
- Log in to comment
Maybe the following simple approach currently implemented will fit your needs?
Install petsc4py this way:
Then, at runtime, you just
export PETSC_ARCH=arch-you-want
, and thenfrom petsc4py import PETSc
will give you what you asked for.If you do not like environment variables, you can also try:
If this is what you had in mind and enough for you, please close this issue. Otherwise, let's continue the discussion... And remember: never EVER complain about all this not being documented, or you risk getting back a "Pull requests very welcome!". Oh, hold on, it is documented, though admittedly obscure and hard to find: https://www.mcs.anl.gov/petsc/petsc4py-current/docs/apiref/petsc4py-module.html#init