Build Fails With Homebrew Installed PETSc 3.4.4

[Kyle Mandli]

I am trying to put together an "easy" set of instructions for someone on Mac OS X to get PETSc and petsc4py to install. My current tact has been to use homebrew and pip in consort however I get the following error when attempting to install petsc4py:

$> pip install petsc4py
Downloading/unpacking petsc4py
  Running setup.py egg_info for package petsc4py

Requirement already satisfied (use --upgrade to upgrade): numpy in ./src/envs/claw/lib/python2.7/site-packages (from petsc4py)
Installing collected packages: petsc4py
  Running setup.py install for petsc4py
    gcc-4.2 not found, using clang instead
    PETSC_DIR:   /usr/local/Cellar/petsc/3.4.4
    PETSC_ARCH:  arch-darwin-c-opt
    scalar-type: real
    precision:   double
    language:    CONLY
    compiler:    mpicc
    linker:      mpicc
    building 'PETSc' extension
    mpicc -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -O -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -O2 -DNDEBUG -g -O3 -DPETSC_DIR=/usr/local/Cellar/petsc/3.4.4 -D__INSDIR__=petsc4py-3.4/ -I/usr/local/Cellar/open-mpi/1.8.1/include -I/usr/local/Cellar/petsc/3.4.4/arch-darwin-c-opt/include -Isrc/include -I/Users/mandli/src/envs/claw/lib/python2.7/site-packages/numpy/core/include -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/PETSc.c -o build/temp.macosx-10.6-intel-2.7/arch-darwin-c-opt/src/PETSc.o
    In file included from /usr/local/Cellar/petsc/3.4.4/arch-darwin-c-opt/include/petscbag.h:4:0,
                     from /usr/local/Cellar/petsc/3.4.4/arch-darwin-c-opt/include/petsc.h:5,
                     from src/petsc4py.PETSc.c:300,
                     from src/PETSc.c:3:
    /usr/local/Cellar/petsc/3.4.4/arch-darwin-c-opt/include/petscsys.h:2338:117: error: expected ';', ',' or ')' before '*' token
     PETSC_STATIC_INLINE PetscErrorCode PetscSegBufferGetInts(PetscSegBuffer seg,PetscInt count,PetscInt *PETSC_RESTRICT *slot) {return PetscSegBufferGet(seg,count,(void**)slot);}
                                                                                                                         ^
    In file included from /Users/mandli/src/envs/claw/lib/python2.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1761:0,
                     from /Users/mandli/src/envs/claw/lib/python2.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:17,
                     from /Users/mandli/src/envs/claw/lib/python2.7/site-packages/numpy/core/include/numpy/arrayobject.h:4,
                     from src/include/petsc4py/numpy.h:11,
                     from src/petsc4py.PETSc.c:304,
                     from src/PETSc.c:3:
    /Users/mandli/src/envs/claw/lib/python2.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
     #warning "Using deprecated NumPy API, disable it by " \
      ^
    error: command 'mpicc' failed with exit status 1
    Complete output from command /Users/mandli/src/envs/claw/bin/python -c "import setuptools;__file__='/Users/mandli/src/envs/claw/build/petsc4py/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-managed --record /var/folders/zm/sqxw5tfd2x94r4md8rdrcdyr0000gn/T/pip-pIVdqa-record/install-record.txt --install-headers /Users/mandli/src/envs/claw/bin/../include/site/python2.7:
    running install

running build

running build_src

running build_py

running build_ext

gcc-4.2 not found, using clang instead

PETSC_DIR:   /usr/local/Cellar/petsc/3.4.4

PETSC_ARCH:  arch-darwin-c-opt

scalar-type: real

precision:   double

language:    CONLY

compiler:    mpicc

linker:      mpicc

building 'PETSc' extension

mpicc -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -O -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -O2 -DNDEBUG -g -O3 -DPETSC_DIR=/usr/local/Cellar/petsc/3.4.4 -D__INSDIR__=petsc4py-3.4/ -I/usr/local/Cellar/open-mpi/1.8.1/include -I/usr/local/Cellar/petsc/3.4.4/arch-darwin-c-opt/include -Isrc/include -I/Users/mandli/src/envs/claw/lib/python2.7/site-packages/numpy/core/include -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/PETSc.c -o build/temp.macosx-10.6-intel-2.7/arch-darwin-c-opt/src/PETSc.o

In file included from /usr/local/Cellar/petsc/3.4.4/arch-darwin-c-opt/include/petscbag.h:4:0,

                 from /usr/local/Cellar/petsc/3.4.4/arch-darwin-c-opt/include/petsc.h:5,

                 from src/petsc4py.PETSc.c:300,

                 from src/PETSc.c:3:

/usr/local/Cellar/petsc/3.4.4/arch-darwin-c-opt/include/petscsys.h:2338:117: error: expected ';', ',' or ')' before '*' token

 PETSC_STATIC_INLINE PetscErrorCode PetscSegBufferGetInts(PetscSegBuffer seg,PetscInt count,PetscInt *PETSC_RESTRICT *slot) {return PetscSegBufferGet(seg,count,(void**)slot);}

                                                                                                                     ^

In file included from /Users/mandli/src/envs/claw/lib/python2.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1761:0,

                 from /Users/mandli/src/envs/claw/lib/python2.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:17,

                 from /Users/mandli/src/envs/claw/lib/python2.7/site-packages/numpy/core/include/numpy/arrayobject.h:4,

                 from src/include/petsc4py/numpy.h:11,

                 from src/petsc4py.PETSc.c:304,

                 from src/PETSc.c:3:

/Users/mandli/src/envs/claw/lib/python2.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]

 #warning "Using deprecated NumPy API, disable it by " \

  ^

error: command 'mpicc' failed with exit status 1

----------------------------------------
Command /Users/mandli/src/envs/claw/bin/python -c "import setuptools;__file__='/Users/mandli/src/envs/claw/build/petsc4py/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-managed --record /var/folders/zm/sqxw5tfd2x94r4md8rdrcdyr0000gn/T/pip-pIVdqa-record/install-record.txt --install-headers /Users/mandli/src/envs/claw/bin/../include/site/python2.7 failed with error code 1 in /Users/mandli/src/envs/claw/build/petsc4py
Storing complete log in /Users/mandli/.pip/pip.log

I am rather suspicious that it ends up defaulting to Clang but I am not sure if this is really the issue. I should also mention that mpi4py is working fine. Any thoughts?

[Lisandro Dalcin]

My guess is that PETSc was built with GCC some time ago, then you upgraded OS X and/or Xcode and GCC is no longer available. Then when trying to build petsc4py with clang bad things happen. Note that the error does not originate in petsc4py sources, but in a PETSc header file that uses the PETSC_RESTRICT definition.

Could you try to brew uninstall and next brew install PETSc? That way you will get a fresh build of PETSc using clang.

[Kyle Mandli]

With your comment it occured to me that PETSc was built via brew with my installed gcc 4.8 (also through brew). The installed python (stock apple) was built with 4.2 which is probably where the problem is coming in. In this case then, what's the best why to tell the petsc4py install to use gcc 4.8 rather than 4.2?

[Aron Ahmadia]

Kyle, I will humbly submit that hashdist is the correct solution to this problem. Can you give me more details about packages you're trying to solve, and on what operating systems?

[Kyle Mandli]

Really would like an easy to install stack for new comers to PyClaw that includes all the parallel packages relevant. This does not stop just here as I would also like to include a number of other numerical packages to provide sort of a "toolbox" for computation. This would ideally be used for both educational and research use.

As for the platform, the mac is first because I have one in front of me, linux is next. Windows users I am not too concerned with (probably suggest a virtual machine).

[Aron Ahmadia]

That's easy to do, and our current stack already covers PETSc/MPI/petsc4py and PyClaw on three operating systems (via Cygwin for Windows).

All we need is more testing :)

[Lisandro Dalcin]

Kyle, are you using MPICH from brew? In that case, PETSc should pick the "mpicc" compiler wrapper, then petsc4py should follow PETSc. Can you check the status of this? Look at $PETSC_DIR/$PETSC_ARCH/conf/petscvariables for the PCC definition.

[Kyle Mandli]

It used open-mpi rather than mpich2. That being said it is clear from the file you pointed to that things are confused about which compiler to use. I am going to try and install everything again being careful to direct the compilers being used and will update when I finish.

[Kyle Mandli]

So I successfully reinstalled the tool-chain using mpich2 and everything seems to be working. I think it was an incompatible compiler somewhere in the compilation chain that was making the problem.

[Lisandro Dalcin]

Glad to know that things are working for you. Managing compiler upgrades and old builds is something that I really do not know how to do within brew (other that manually uninstall and reinstall).