Revise installation and build process

[Bradley Dice]

From roadmap planning session with @vramasub and @bdice.

TBB and CMake

We would like to move from CMake to a pure Cython install, while retaining our usage of TBB. We need to make sure that we can find TBB in Python. Should we have users provide a directory? Could we make it a submodule?

[Joshua Anderson]

TBB is fairly simple to find. You need to find a) the tbb headers and b) the tbb shared libraray. Headers go on the include path and the library needs to be included at link time. I do not know what standard practice for this is in setup.py type builds, surely there must be some good example to follow somewhere.

To use setup.py, you'll also need to set C++11 compiler flags, and find the numpy include directory. I have yet to find a good way to do this for GSD's setup.py build - at least in connection with pypi. Each method I have tried has resulted in problems for one user or another. Please let me know if you find a preferred solution for this so that I can apply it to gsd.

You will probably want some sort of environment variable (TBB_ROOT or TBB_INCLUDE/TBB_LINK) to specify the directories to look in. Conda builds will need to set these to ensure that the conda-provided tbb is located.

[Malcolm Ramsay]

I don't know if it fits into your planning of re-configuring the build system but I thought I would bring scikit-build to your attention, which supports cmake compilation in the setup.py file. When using pip >= 10, which brings support for PEP 518 (the pyproject.toml file), the build dependencies (cmake + numpy) can be specified in the in the pyproject.toml file which will be installed before running the setup.py file. The best explanation of the pyproject.toml file is by Brett Cannon.

I was able to make freud pip installable on osx where tbb is pre-installed with minimal changes. Even if you do completely go the Cython route, the pyproject.toml might be suitable for finding the numpy include directory, assuming requiring pip >=10 is acceptable.

Adding set(CMAKE_OSX_DEPLOYMENT_TARGET 10.9) to CMakeLists.txt to fix a bug in scikit-build (fixed in next release 0.7.2)

# setup.py
from skbuild import setup
setup(name="freud")

# pyproject.toml
[build-system]
requires = ["setuptools", "wheel", "scikit-build", "cmake", "numpy", "cython"]

[Vyas Ramasubramani]

Thanks for the info Malcolm! I've actually discussed this possibility with the scikit-build devs before (see here). Using scikit-build is not part of our immediate plans, but it is something that we might be interested in in the future. At the moment our needs are relatively simple, and overall the package installation process contains more complexity than we would like, so that is the primary goal at the moment. However, to make it easier to install across a wider range of systems I agree that scikit-build could be a sensible route going forward.

We are currently aiming to move to a more Pythonic solution than our current installation process, which is to use CMake to call cythonize from the command line. We'd like to be able to follow the standard setup.py paradigm for Python packages, and we do want the package to be pip-installable so long as TBB is available. With that goal in mind, the first step we're trying to take is just ripping out CMake and simplifying the project structure to work nicely with Cython. Once that is done and the code is more streamlined, we'll probably revisit whether or not scikit-build would be useful for making the code more broadly usable.

Thanks for the snippet! Bradley and I actually just discussed the need for a MACOSX_DEPLOYMENT_TARGET= yesterday and I just added it to the most recent set of commits on the pure_cython branch. Feel free to give that a shot and see if it works the way you'd like for now.

[Vyas Ramasubramani]

Pull request #191 moved us to a pure Cython installation and correctly finds TBB.