docker with IPOpt

[Evan Cummings]

Hello,

I'm trying to create a docker which includes IPOpt, and am getting a segfault when running a test script.

I am wondering, do you have a docker that includes IPOpt?

[Simon Funke]

Sorry I do not have a docker script, it would be very useful to add though. Did you base your docker script on https://bitbucket.org/dolfin-adjoint/virtual/src ? Can you share what you have so far?

Regarding debugging, maybe you can run the testscript with gdb to get an idea why it crashes?

[Evan Cummings]

I have tried basing the Dockerfile from dolfin-adjoint, and fenics-stable, with both giving the same error. However, the L_BFGS_B method does work. I am not familiar with the `testscript gdb'.

Here's what I have (based from fenics-stable):

#FROM quay.io/dolfinadjoint/dolfin-adjoint:latest
FROM quay.io/fenicsproject/stable
#FROM ubuntu:16.04
MAINTAINER pf4d <[e-mail]@gmail.com>


#RUN apt-get update && \
#    apt-get -y install software-properties-common

USER root

#RUN add-apt-repository ppa:fenics-packages/fenics && \
RUN apt-get update && \
    apt-get -y install python-pip \
                       python-shapely \
                       htop \
                       vim \
                       git \
                       curl \
                       wget \
                       libsm6 \
                       libglu1-mesa && \
    apt-get -y dist-upgrade


# get the latest python packages :
RUN pip install --upgrade colored \
                          termcolor \
                          pyproj \
                          tifffile



# install libadjoint :
RUN git clone -b libadjoint-2016.1.0 https://bitbucket.org/dolfin-adjoint/libadjoint && \
    cd libadjoint && \
    mkdir build; cd build && \
    cmake -DCMAKE_INSTALL_PREFIX="/usr/local" .. && \
    make install && \
    cd ../.. && \
    rm -r libadjoint


# install dolfin-adjoint :
RUN git clone -b dolfin-adjoint-2016.1.0 https://bitbucket.org/dolfin-adjoint/dolfin-adjoint
ENV PYTHONPATH /home/fenics/dolfin-adjoint:$PYTHONPATH


# install ipopt with metis and mumps, still need HSL :
RUN cd /tmp && \
    curl -O http://www.coin-or.org/download/source/Ipopt/Ipopt-3.12.6.tgz && \
    tar -xvf Ipopt-3.12.6.tgz && \
    cd Ipopt-3.12.6 && \
    cd ThirdParty/Metis && \
    ./get.Metis && \
    cd ../../ && \
    cd ThirdParty/Mumps && \
    ./get.Mumps && \
    cd ../../ && \
    ./configure --with-blas="-lblas -llapack" --with-lapack="-llapack" --prefix="/usr/local" && \
    make install && \
    cd ../ && \
    rm -r Ipopt-3.12.6

#./configure --with-blas="-lblas -llapack" --with-lapack="-llapack" --with-hsl-incdir="/usr/local/include" --with-hsl-lib="/usr/local/lib" --with-mumps-incdir="/usr/include" --with-mumps-lib="/usr/lib" --prefix="/usr/local"'


# install pyipopt :
RUN cd /tmp && \
    git clone https://github.com/pf4d/pyipopt.git && \
    cd pyipopt && \
    python setup.py build && \
    python setup.py install && \
    cd ../ && \
    ldconfig && \
    rm -r pyipopt


# install basemap for matplotlib :
RUN wget http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/basemap-1.0.7/basemap-1.0.7.tar.gz && \
    tar -xzvf basemap-1.0.7.tar.gz && \
    cd basemap-1.0.7/geos-3.3.3/ && \
    export GEOS_DIR=/usr/local/ && \
    ./configure --prefix=$GEOS_DIR && \
    make && \
    make install && \
    cd .. && \
    python setup.py install && \
    cd .. && \
    rm -r basemap-1.0.7 && \
    rm basemap-1.0.7.tar.gz


# install gmsh-dynamic 2.10.1 :
RUN wget https://www.dropbox.com/s/hp64kx6wh790sf6/gmsh.tgz?dl=1 -O gmsh.tgz && \
    tar -xzvf gmsh.tgz && \
    cd gmsh-2.10.1-dynamic-svn-Linux && \
    cd gmshpy && \
    python setup.py install && \
    ldconfig && \
    cd ../.. && \
    rm gmsh.tgz
ENV PATH /home/fenics/gmsh-2.10.1-dynamic-svn-Linux/bin:$PATH
ENV LD_LIBRARY_PATH /home/fenics/gmsh-2.10.1-dynamic-svn-Linux/lib:$LD_LIBRARY_PATH


# install cslvr :
RUN git clone https://github.com/pf4d/cslvr
ENV PYTHONPATH /home/fenics/cslvr:$PYTHONPATH


# finally, cleanup :
RUN apt-get clean && \
    rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

[Simon Funke]

Thanks.

I have now added IpOpt to the official Docker files - but I do get the same segmentation fault as you do.

Running the optimization_pyipopt test with the dev-dolfin-adjoint Docker image results in this:

Program received signal SIGSEGV, Segmentation fault.
__GI___libc_free (mem=0xfff907aa00027100) at malloc.c:2952
2952    malloc.c: No such file or directory.
(gdb) bt
#0  __GI___libc_free (mem=0xfff907aa00027100) at malloc.c:2952
#1  0x00007fffe757179b in Ipopt::MumpsSolverInterface::InitializeStructure (this=0x1bf04c0, dim=21, nonzeros=42, ia=0x1c20a40, ja=0x1c20af0) at IpMumpsSolverInterface.cpp:312
#2  0x00007fffe7555217 in Ipopt::TSymLinearSolver::InitializeStructure (this=0x1bf52f0, sym_A=...) at IpTSymLinearSolver.cpp:344
#3  0x00007fffe75547c3 in Ipopt::TSymLinearSolver::MultiSolve (this=0x1bf52f0, sym_A=..., rhsV=std::vector of length 1, capacity 1 = {...}, solV=std::vector of length 1, capacity 1 = {...}, 
    check_NegEVals=true, numberOfNegEVals=0) at IpTSymLinearSolver.cpp:169
#4  0x00007fffe7544b2f in Ipopt::StdAugSystemSolver::MultiSolve (this=0x1bf5380, W=0xe3fb50, W_factor=1, D_x=0x1c22be0, delta_x=0, D_s=0x1c1ebb0, delta_s=0, J_c=0x1b67190, D_c=0x0, 
    delta_c=0, J_d=0x1c17af0, D_d=0x0, delta_d=0, rhs_xV=std::vector of length 1, capacity 1 = {...}, rhs_sV=std::vector of length 1, capacity 1 = {...}, 
    rhs_cV=std::vector of length 1, capacity 1 = {...}, rhs_dV=std::vector of length 1, capacity 1 = {...}, sol_xV=std::vector of length 1, capacity 1 = {...}, 
    sol_sV=std::vector of length 1, capacity 1 = {...}, sol_cV=std::vector of length 1, capacity 1 = {...}, sol_dV=std::vector of length 1, capacity 1 = {...}, check_NegEVals=true, 
    numberOfNegEVals=0) at IpStdAugSystemSolver.cpp:222
#5  0x00007fffe7481f42 in Ipopt::AugSystemSolver::Solve (this=0x1bf5380, W=0xe3fb50, W_factor=1, D_x=0x1c22be0, delta_x=0, D_s=0x1c1ebb0, delta_s=0, J_c=0x1b67190, D_c=0x0, delta_c=0, 
    J_d=0x1c17af0, D_d=0x0, delta_d=0, rhs_x=..., rhs_s=..., rhs_c=..., rhs_d=..., sol_x=..., sol_s=..., sol_c=..., sol_d=..., check_NegEVals=true, numberOfNegEVals=0)
    at IpAugSystemSolver.hpp:104
#6  0x00007fffe74e9a3e in Ipopt::LowRankAugSystemSolver::Solve (this=0x1bf5480, W=0x1c251c0, W_factor=1, D_x=0x1c22be0, delta_x=0, D_s=0x1c1ebb0, delta_s=0, J_c=0x1b67190, D_c=0x0, 
    delta_c=0, J_d=0x1c17af0, D_d=0x0, delta_d=0, rhs_x=..., rhs_s=..., rhs_c=..., rhs_d=..., sol_x=..., sol_s=..., sol_c=..., sol_d=..., check_NegEVals=true, numberOfNegEVals=0)
    at IpLowRankAugSystemSolver.cpp:170
#7  0x00007fffe7514737 in Ipopt::PDFullSpaceSolver::SolveOnce (this=0x1bf5560, resolve_with_better_quality=false, pretend_singular=false, W=..., J_c=..., J_d=..., Px_L=..., Px_U=..., 
    Pd_L=..., Pd_U=..., z_L=..., z_U=..., v_L=..., v_U=..., slack_x_L=..., slack_x_U=..., slack_s_L=..., slack_s_U=..., sigma_x=..., sigma_s=..., alpha=1, beta=0, rhs=..., res=...)
    at IpPDFullSpaceSolver.cpp:518
#8  0x00007fffe75124ae in Ipopt::PDFullSpaceSolver::Solve (this=0x1bf5560, alpha=-1, beta=0, rhs=..., res=..., allow_inexact=true, improve_solution=false) at IpPDFullSpaceSolver.cpp:216
#9  0x00007fffe75263d7 in Ipopt::QualityFunctionMuOracle::CalculateMu (this=0x1bf6750, mu_min=9.9999999999999994e-12, mu_max=1000.0000495969518, 
    new_mu=@0x7fffffffcd00: 1.4588114271222651e-316) at IpQualityFunctionMuOracle.cpp:226
#10 0x00007fffe746be5d in Ipopt::AdaptiveMuUpdate::UpdateBarrierParameter (this=0x1bf6910) at IpAdaptiveMuUpdate.cpp:394
#11 0x00007fffe74ad658 in Ipopt::IpoptAlgorithm::UpdateBarrierParameter (this=0x1bf6a50) at IpIpoptAlg.cpp:489
#12 0x00007fffe74ac7f8 in Ipopt::IpoptAlgorithm::Optimize (this=0x1bf6a50, isResto=false) at IpIpoptAlg.cpp:317
#13 0x00007fffe74135d1 in Ipopt::IpoptApplication::call_optimize (this=0x1b88230) at IpIpoptApplication.cpp:885
#14 0x00007fffe74125e2 in Ipopt::IpoptApplication::OptimizeNLP (this=0x1b88230, nlp=..., alg_builder=...) at IpIpoptApplication.cpp:772
#15 0x00007fffe741229c in Ipopt::IpoptApplication::OptimizeNLP (this=0x1b88230, nlp=...) at IpIpoptApplication.cpp:735
#16 0x00007fffe7411dd9 in Ipopt::IpoptApplication::OptimizeTNLP (this=0x1b88230, tnlp=...) at IpIpoptApplication.cpp:714
#17 0x00007fffe741d41d in IpoptSolve (ipopt_problem=0x1b88b90, x=0x1b88ae0, g=0x0, obj_val=0x7fffffffdb68, mult_g=0x1b95bb0, mult_x_L=0x1b78c20, mult_x_U=0x1b89450, user_data=0x1bef760)
    at IpStdCInterface.cpp:272
#18 0x00007fffe785a7d6 in solve (self=<optimized out>, args=<optimized out>) at src/pyipoptcoremodule.c:663
#19 0x00000000004c41da in PyEval_EvalFrameEx ()
#20 0x00000000004c994f in PyEval_EvalFrameEx ()
#21 0x00000000004c22e5 in PyEval_EvalCodeEx ()
#22 0x00000000004c2089 in PyEval_EvalCode ()
#23 0x00000000004f1e6f in ?? ()
#24 0x00000000004ec802 in PyRun_FileExFlags ()
#25 0x00000000004eaf81 in PyRun_SimpleFileExFlags ()
#26 0x000000000049e36a in Py_Main ()
#27 0x00007ffff7811830 in __libc_start_main (main=0x49dcd0 <main>, argc=2, argv=0x7fffffffe238, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, 
    stack_end=0x7fffffffe228) at ../csu/libc-start.c:291
#28 0x000000000049dbf9 in _start ()
(gdb) 

[Simon Funke]

The problem was that IpOpt was compiled against its own Mumps version, but at runtime it started using the mumps library that comes with PETSc.

I have now fixed this in commits a567086075a7a677b0e2645ff699b12268a46dea and 8bb1e00dc94bae79e862c747c797aed77024322a.

With this the optimization-pyipopt test runs fine in serial. Running it in parallel still. This will require some more thought, but Ill close this issue for now.