pybind11 Expression broken: parameters list/set confusion

[Marco Morandini]

Right now demo/documented/poisson/python does not work with pybind. If I understand correctly it boils down to the fact that somewhere some parameters (params['build']['include_dirs'], params['build']['libs'] and params['build']['lib_dirs']) are generated as set, but this is not accepted by dijitso.params.as_str_tuple. Relaxing the requirement there to accept also sets makes ufl unahappy: in ufl/ufl/utils/sorting.py there is

warning("Applying str() to a metadata value of type {0}, don't know if this is safe.".format(type(value).__name__))

A patch like the following papers around the issue, but I'm not convinced it is the right approach:

--- /home/marco/Programmi/Dolphin/src_from_dorsal/dolfin/python/dolfin/./jit/jit.py     2017-10-12 14:10:09.077094838 +0200                                    
+++ ./jit/jit.py        2017-10-13 17:47:26.034746703 +0200                                                                                                    
@@ -133,9 +133,9 @@

     # Set compiler/build options
     params = dijitso.params.default_params()
-    params['build']['include_dirs'] = d["include_dirs"]
-    params['build']['libs'] = d["libraries"]
-    params['build']['lib_dirs'] = d["library_dirs"]
+    params['build']['include_dirs'] = list(d["include_dirs"])
+    params['build']['libs'] = list(d["libraries"])
+    params['build']['lib_dirs'] = list(d["library_dirs"])

     name = cpp_data['name']
     if name not in ('subdomain', 'expression'):
--- /home/marco/Programmi/Dolphin/src_from_dorsal/dolfin/python/dolfin/./fem/form.py    2017-10-12 14:10:09.077094838 +0200
+++ ./fem/form.py       2017-10-13 17:48:40.940211462 +0200
@@ -36,10 +36,10 @@
         import pkgconfig
         d = pkgconfig.parse('dolfin')
         if form_compiler_parameters is None:
-            form_compiler_parameters = {"external_include_dirs": d["include_dirs"]}
+            form_compiler_parameters = {"external_include_dirs": list(d["include_dirs"])}
         else:
             # FIXME: add paths if dict entry already exists
-            form_compiler_parameters["external_include_dirs"] = d["include_dirs"]
+            form_compiler_parameters["external_include_dirs"] = list(d["include_dirs"])

         ufc_form = ffc_jit(form, form_compiler_parameters=form_compiler_parameters,
                            mpi_comm=mesh.mpi_comm())

[Jan Blechta]

@MarcoMorandini I can't reproduce it with current master, demo_poisson.py runs just fine. Maybe it is something dependent on the configuration. What is value of your external_include_dirs?

[Prof Garth Wells]

Not sure what's up here; Poisson demo runs on the CI and is the go-to test problem and we haven't had any other reports.

[Marco Morandini]

Don't know why, but on my build pkgconfig.parse('dolfin')["include_dirs"] is a set, and not a list. Thus, the error. I.e. I get this:

marco@pao:~> python3
Python 3.4.6 (default, Mar 22 2017, 12:26:13) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pkgconfig
>>> d = pkgconfig.parse('dolfin')
>>> print(d)
defaultdict(<class 'set'>, {'define_macros': {('HAS_SLEPC', ''), ('HAS_PARMETIS', ''), ('DOLFIN_VERSION', '2017.2.0.dev0'), ('NDEBUG', ''), ('HAS_CHOLMOD', ''), ('HAS_SCOTCH', ''), ('HAS_ZLIB', ''), ('HAS_TRILINOS', ''), ('HAS_PETSC4PY', ''), ('DOLFIN_LA_INDEX_SIZE', '4'), ('HAS_MPI', ''), ('HAS_PETSC', ''), ('DOLFIN_SIZE_T', '8'), ('HAS_HDF5', ''), ('HAS_UMFPACK', '')}, 'include_dirs': {'/home/marco/local/Fenics/include/eigen3', '/home/marco/local/Fenics/lib/python3.4/site-packages/ffc/backends/ufc', '/home/marco/local/Fenics/include/suitesparse', '/home/marco/local/Fenics/include/trilinos', '/home/marco/local/Fenics/lib64/python3.4/site-packages/petsc4py/include', '/home/marco/local/Fenics/include'}, 'library_dirs': {'/home/marco/local/Fenics/lib64', '/home/marco/local/Fenics/lib', '/home/marco/Programmi/Vtk/ParaView-5.4.0-Qt5-OpenGL2-MPI-Linux-64bit/lib'}, 'libraries': {'anasazi', 'belosepetra', 'tpetra', 'petsc', 'galeri-xpetra', 'zoltan', 'stratimikos', 'tpetraclassic', 'teuchosremainder', 'ModeLaplace', 'aztecoo', 'teuchosnumerics', 'stratimikosamesos', 'tpetraext', 'slepc', 'amesos2', 'belostpetra', 'tpetrainout', 'xpetra-sup', 'teuchoskokkoscompat', 'stratimikosml', 'kokkoscontainers', 'stratimikosaztecoo', 'z', 'teko', 'belos', 'mpi', 'kokkostsqr', 'mpi_cxx', 'thyraepetra', 'boost_timer', 'stratimikosifpack', 'anasaziepetra', 'thyracore', 'thyratpetra', 'teuchoscore', 'muelu', 'epetra', 'tpetraclassicnodeapi', 'tpetraclassiclinalg', 'zoltan2', 'muelu-adapters', 'anasazitpetra', 'triutils', 'ifpack2', 'stratimikosbelos', 'ifpack', 'amesos', 'teuchoscomm', 'epetraext', 'muelu-interface', 'xpetra', 'dl', 'thyraepetraext', 'dolfin', 'isorropia', 'teuchoskokkoscomm', 'm', 'hdf5', 'kokkoscore', 'tpetrakernels', 'galeri-epetra', 'teuchosparameterlist', 'ml', 'kokkosalgorithms', 'rtop', 'ifpack2-adapters'}})
>>> type(d["include_dirs"])
<class 'set'>
>>> 

Perhaps a configuration error on my side?

[Chris Richardson]

Can you find out what version of pkgconfig you have? Mine says 1.2.2 - maybe try pip3 install -U pkgconfig to update?

[Prof Garth Wells]

Side note: we'll remove the dependency on the pkgconfig module at some point. It's very lightweight so easy for us to implement.

[Marco Morandini]

Version 1.1.0

You are right, after upgrading pkgconfig type(d["include_dirs"]) returns <class 'list'> and not <class 'set'> as with 1.1.0.

I think this could be confusing to all the users that, like me, relies on their distribution pkgconfig, whatever version it may be; perhaps the workaround proposed at the top of the thread could avoid them some headache.

[Jan Blechta]

Maybe the version check with error message would be more sensible than any workarounds.

[Prof Garth Wells]

Issue is with pkgconfig module, which will eventually be replaced.