Binding generation issue on ubuntu16.04

[Former user]

Hi,

I tried to use cppyy to generate python binding module like https://github.com/jclay/cppyy-knearestneighbors-example. On py2.7 there's no error during building. However, when I try to use the binding module, there's something that doesn't work very well.

from cppyy.gbl import std
import knn_example
from knn_example import NearestNeighbors, Point

Point.__repr__ = lambda self: repr(str(self.x) + ", " + str(self.y))

knn = NearestNeighbors()
points = [Point(2,0), Point(1,0), Point(0,10), Point(5,5), Point(2,5)]
knn.points = std.vector[Point](points)
result = knn.nearest(Point(2.1, 5.3), 4)
print(list(result))

I try to run this script, but in

result = knn.nearest(Point(2.1, 5.3), 4)

I got the following error.

IncrementalExecutor::executeFunction: symbol '_ZN16NearestNeighbors7nearestE5Pointi' unresolved while linking symbol '__cf_9'!
You are probably missing the definition of NearestNeighbors::nearest(Point, int)
Maybe you need to load the corresponding shared library?
Error in <TClingCallFunc::make_wrapper>: Failed to compile
  ==== SOURCE BEGIN ====
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wformat-security"
__attribute__((used)) extern "C" void __cf_9(void* obj, int nargs, void** args, void* ret)
{
   if (ret) {
      new (ret) (vector<Point>) (((NearestNeighbors*)obj)->nearest(*(Point*)args[0], *(int*)args[1]));
      return;
   }
   else {
      ((NearestNeighbors*)obj)->nearest(*(Point*)args[0], *(int*)args[1]);
      return;
   }
}
#pragma clang diagnostic pop
  ==== SOURCE END ====
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: vector<Point> NearestNeighbors::nearest(Point, int k) =>
    ValueError: nullptr result where temporary expected

It's weird that I need to explictly load the "knn_lib" library to avoid that error.

Maybe the LINK_LIBRARIES argument in cppyy_add_bindings() of CMakeLists.txt didn't work?

Here is the environment:

System:

Ubuntu 16.04 (Windows 10 subsystem)

Package:

cppyy (1.4.5)

cppyy-backend (1.7.4)

cppyy-cling (6.15.2.5)

CPyCppyy (1.6.4)

[wlav]

You're right of course, an explicit load should not be needed. I haven't had time to look into it yet (working on, hopefully, the last bits of the MS Windows port), but here's another example that takes a different tack and extended the cmake fragments: https://github.com/camillescott/cppyy-bbhash

[Lycen Yao]

I forgot to sign in when I post this issue. Thanks for your reply, wlav. The cppyy-bbhash example looks nice, especially the selection file part.

Edit: The bbhash example works fine only when the whole library is header-only. Otherwise calling methods still throws missing definition error

[Camille Scott]

@LycenYao I initially did a simpler header only library compiled into a static library to simplify things in the example case. I'll have an example with a full shared library out this week, hopefully.

[Camille Scott]

I now have a working version for the knn example. The issue was more or less that CMake needs to be given the LINK_WHAT_YOU_USE directive for linking the cppyy lib against the c++ shared lib; otherwise, ld doesn't detect that the cppyy lib needs those symbols, and it drops the shared lib (LINK_WHAT_YOU_USE controls the --as-needed/--no-as-needed gcc/ld flags). As you can see, this is all working under continuous integration now as well, so portability is good.

Cheers, Camille

[Lycen Yao]

@camillescott LINK_WHAT_YOU_USE do work. I didn't know that Ubuntu16.04 use --as-needed flag in default before. Thank you for your help.

[wlav]

Yes, thanks a lot Camille!

[wlav]

Clarified.

[William Hart]

@Camille Scott I’ve been working on adapting your knn example for my coek code over the weekend (see https://gitlab.com/coopr/coek/tree/cppyy). Unfortunately, I’m running into the same issue as here. This uses the knn example logic, but it reorganizes your code a bit so all the cppyy logic is in a coek/cppyy subdirectory. (This required some minor edits to your cmake Find* files.)

I’m not sure how to create a small example. Is there an obvious issue in the coek/cppyy/CMakeLists.txt file? Any recommendation how to encapsulate this example? Could I create a separate issue?

WRT the error, I’m pulling in classes from three header files. But the VariableArray class in coek/src/expr/varray.hpp is the one generating the error.

[William Hart]

I seem to have resolved this issue by cleaning up some *.so files that were masking my local build. I think this was getting cppyy or cling confused.

[Camille Scott]

Glad you got it figured out! I’ll note that for templates, I was forced to move a lot of implementation into .cc files and explicitly declare specializations to avoid intermittent missing symbols in my bindings.