Using CMake build system

Issue #77 resolved
OgreT
created an issue

I would suggest to use CMake as a build system. Then you can also skip the outdated Boost header and use your own Boost version. Also, it's much easier to create Coin for multiple platforms.

Comments (56)

  1. Thomas Moeller

    A few people would agree and Christian already started working on it (see pull request #17). However, ensuring correct builds on all supported platforms appears to be far from trivial. I'm certain Christian would appreciate some help from CMake experts. ;-)

    Thomas

  2. Bastiaan Veelo

    I have done some reading on CMake, and I think it should fit the bill. After all, KDE moved from Autotools to CMake long ago (including Windows using various compilers) which is something. Our current Autotools build system is outdated, fragile (update to newer versions got backed out) and patchy, a pain to maintain (search for Bootstrapping) and Windows builds are a masterpiece of complexity relying on hacks. Because of this, Coin does no longer build on Linux at this moment.

    Migration to CMake is starting to look like a vital step forward. In addition, we could get automated builds and testing hosted for free which is not only nice to have, but it will also help in the migration to CMake for verification on a great variety of platforms.

    So how should we go about it? Ideally, we should support the same configure options as we did previously. There may be some knowlege worth having in the am2cmake script that was used to convert KDE. There is likely to be much manual labour involved. @Christian B├╝hler: How was pull request #17 devised? Did you look at the configure script, MS project files, or something else?

    Maybe @Roy Walmsley should start a cmake branch that we can write pull requests for, and collectively bring to perfection before it is merged into main? Including README et al and removal of Autotools support?

    In the mean time, should we address any of the build issues that are coming in? It may be difficult to verify completeness as long as the original build system is broken.

  3. Bastiaan Veelo

    Note about CMake versions: newest release is currently 3.1.1, but it would be best to be compatible with versions that ship with recent Linux distributions. Ubuntu LTS is at 14.04 and it ships CMake 2.8.12.2. Pull request #17 has:

    cmake_minimum_required(VERSION 2.8.12)
    

    which looks good. On Windows it might be a good idea (when implementing cmake build) to install 2.8.12 and not the latest to ensure compatibility.

  4. Roy Walmsley

    I have now uploaded Christian's files into the branch. I had to do it manually as I couldn't change the destination. Hence pull request #17 was declined.

    I have CMake 2.8.12.2 installed on my Windows system. My IDE is Visual Studio 2008. I ran CMake through the "configure" and "generate" steps. It went through OK although I haven't inspected the result seriously.

    Roy

  5. Thomas Moeller

    I'm so happy there is progress with CMake. I tried to understand and follow the changes from Alexey in the automake setup, including the ones from the simacros repository. I had to give up after a while.

    After this is transition I think we should try "releasing" the first version of Coin 4. It seems without it people will still use the binary packages of Coin 3.1.3 from the download section.

  6. Roy Walmsley

    Hi all. Yes, sorry - definitely pushed the wrong button somewhere!

    I considered trying to roll it back but by then I had already implemented Bastiaan's pull request #32 on my system. So it looked like it was going to get very messy. Hence I left it at default.

    Do you want me to recreate a new CMake branch?

    Apologies once again,

    Roy

  7. Thomas Moeller

    Hi Roy, it would be great to get either the CMake branch or the files back on default. I couldn't clone the repository from Bastiaan (unresolved commit in a linked repo) but would like to test CMake and help where I can.

    Edit: After taking a closer look I believe it would be better (easier) to just add the CMake files to the default branch. There doesn't seem to be a conflict with the existing build system / projects.

  8. Roy Walmsley

    Hi Thomas,

    At the moment the CMake branch is back. Again not sure how it happened. Does it seem to be up to date for you? Or is it causing you a problem? It effectively has pull request #32 incorporated so has Bastiaan's latest update.

    Roy

  9. Bastiaan Veelo

    @Thomas Moeller: I thought it would be a good idea to remove Autotools completely after CMake is 100% good, including updating all READEs and related files. Although it would work to do all of this in default, I think it will be easier to see what changes relate to this effort later on when we do all this in a branch and merge it when it is done in one single operation.

    You should be able to get the CMake branch after pulling from the Coin3D/coin repository. Then you can change to it and test and contribute. Maybe there is an option to pull to get new branches, at least that branch was included when I cloned.

    Strange error you got when cloning my repo, I don't understand it...

  10. Roy Walmsley

    I've started to play with CMake and Doxygen ...

    It's easy to verify it can find Doxygen. My next problem is autogeneration of 'doxyfile' - i.e. the file that is usually generated by the configure process. So I have created a partial 'coin.doxygen.cmake.in' file. My problem now is to automate the variables.

    First question on this is: Where is the one and only place we should be specifying things like the package, version number, project, etc?

    Roy

  11. Bastiaan Veelo

    Where is the one and only place we should be specifying things like the package, version number, project, etc?

    Project is specified on line two of the toplevel CMakeLists.txt, I'd say the version numbers go on line 3.

  12. Roy Walmsley

    Bastiaan, response to your comments:

    Putting the version (and any other relevant one time only information) at the top of the CMakeLists.txt file was exactly my thought.

    I used the FindDoxygen command. I didn't look for a FIndCoind3D! That's good. It might be useful for building other repositories dependent on Coin3D, e.g. SoQT, etc.

    Roy

  13. Roy Walmsley

    I've just committed into the CMake branch a first pass at automating the documentation build. I have left the actual call to Doxygen commented out, but I have run the CMake to generate the intermediate 'Doxyfile'. I have then manually run Doxygen, using the generated 'Doxyfile' and successfully generated some HTML which seems to be OK.

    I would appreciate it if you can check this yourselves.

    Roy

  14. OgreT reporter

    I've updated the CMake files. https://bitbucket.org/Coin3D/coin/pull-request/34/updated-cmake-files

    What's new?

    First I added a possibility to create one big build project like the MSVC solutions in build directory. Instead of creating a lot of small libraries, I collect all required files in a list and add them to one project.

    Second, I update the documentation creation. The file coin_doxygenfiles.cmake.inc has been deleted, because I create the content in all subscripts. Also, I add support for HHC on Windows.

    The last point is the installation. I install headers, the library and the documentation.

  15. Roy Walmsley

    I ran CMake with all options checked. No problems. I ran Doxygen on the output doxyfile - ahh... now we have issues. The first view messages in a very, very long log are shown below.

    Start of Doxygen Log after Pull Request 34.png

    Roy

  16. OgreT reporter

    @Roy Walmsley Yes, there are two reasons for this:

    1. The doxygen configuration is not compatible to HTML workshop. The menu on the left side (GENERATE_TREEVIEW) is not available for compressed HTML files. It is possible to disable this message by using the configure command for this option, too.

    2. See https://bitbucket.org/Coin3D/coin/pull-request/34/updated-cmake-files/diff#comment-5147868 (commit https://bitbucket.org/Coin3D/coin/commits/1b33857771a4dd7139e7f09ea5df09a54dc53d8c)

  17. Thomas Moeller

    The pull request #35 contains a change that sets the GENERATE_TREEVIEW based on the HHC option, thereby addressing the first issue. I also agree with the comment from Bastiaan linked to by transporter. But this means a lot of include files still have to be added to the COIN_DOCUMENTATION_FILES variable in various CMakeList.txt files.

  18. Roy Walmsley

    With the latest update I ran CMake first, and then doxygen.

    My first observation was going to be that CMake has both "BUILD_DOCUMENTATION" and "COIN_BUILD_DOCUMENTATION" options listed (I didn't investigate). I ran with all options set to default (that is what I would want, anyway). When I ran Doxygen I didn't get the tree view warning like above. But on opening the index file I didn't get the tree view!!

    Then I decided to clean all the output and try again. Just as well. Now I had a much better list of options. And I had the tree view in the Doxygen output. Yes, the log file shows things need to be done. But I am pleased with progress so far. It's looking good.

    So that brings me to ask if there is a way to automatically clean all old content at the start of a CMake build?

    Roy

  19. Bastiaan Veelo

    if there is a way to automatically clean all old content at the start of a CMake build?

    If you build out-of-source as described in INSTALL, you can just empty the build directory completely.

  20. Roy Walmsley

    After the last CMake pull request #38 I have a three comments:

    1) There are a lot of header files missing. I'm working on that and should be able to commit later today. 2) When I look at the 'Doxygen' file output by CMake it has all the input files listed twice! (with the exception of 'tidbits' which is added separately to all the rest. It looks like the "COIN_DOCUMENTATION_FILES" variable is being built twice. Does anyone else see this? 3) When I try to compile (VS 2008) the CMake output I have an include file problem. CMake determines I don't have "inttypes.h" so it creates it. When compiling a source file that includes "inttypes,h" I get the message "Cannot open include file 'stdint,h': No such file or directory." Now, looking at the "inttypes.h" header file suggests issues with subsequent defines. I note that, for example, CMake does not search for "stdint.h". Has anyone else got this sort of problem?

    Roy

  21. Bastiaan Veelo

    1) There are a lot of header files missing.

    You mean regarding the Doxygen run? These are internal and not part of the API. I have an option in the works to [in|ex]clude internal documentation, which should address these warnings. What is your approach?

  22. Bastiaan Veelo

    Note that a lot of Doxygen warnings are due to my suggestion to use the list of sources both for compilation and for documentation. This pulls in internal sources, which should not be documented publicly. As these sources use internal headers that are not in the list, you get missing declarations. My take on this is to put the internal sources (.cpp and .h) in an additional list, and add them to EXCLUDE_FILES in the Doxyfile for public documentation, or to INPUT for complete documentation. This approach is of course debatable: we could also revert to the original with two separate lists for compilation and documentation, but I expect the list of internal sources to be much shorter than the list of public sources and it has the advantage of the option to document everything.

    Roy, let me know if you want me to complete this pull request, or you prefer your own fixes.

    Bastiaan.

  23. Roy Walmsley

    Bastiaan,

    I think I like the idea of having the option to document only the public stuff or else everything. Users of Coin who only need to know about the public API need only the public documentation. Developers, possibly modifying Coin, might want complete documentation. Hence I like the option approach.

    This also has the advantage that the majority of files are only listed once. It is only those files getting special treatment that are listed twice. So that makes maintenance easier.

    Having expressed my view I am open to suggestions. As long as we sort it out I don't really mind which way we go. I could easily undo my local commit. In which case would sending you a copy of my files help you at all?

    Roy

  24. Bastiaan Veelo

    OK I will continue working on this then.

    In which case would sending you a copy of my files help you at all?

    I think I am fine, but if you want and you use TortoiseHg I think you can easily mail a change set. My address is like Firstname@Lastname.net.

  25. Bastiaan Veelo

    Can we define issues depending on other issues? Would be nice to link issues to the CMake branch or to this issue. In that category:

    • Need option for linking with superglu. (Don't we?)
    • Is the link hack still needed? What is its purpose? See LinkHackSources in Makefile.am and all-<dir>-cpp.cpp.
    • fonts directory support for extractfont (see Makefile.am).
  26. Bastiaan Veelo

    2) When I look at the 'Doxygen' file output by CMake it has all the input files listed twice! (with the exception of 'tidbits' which is added separately to all the rest. It looks like the "COIN_DOCUMENTATION_FILES" variable is being built twice. Does anyone else see this?

    Yes. Every time I press "Generate" in cmake-gui that list is appended to, not replaced. I think this is because it is built using CACHE INTERNAL. This may also be why cmake-gui gets a lot slower each run until I delete CMakeCache.txt. I doubt we are building our lists the way we should...

  27. Roy Walmsley

    Bastiaan,

    Linking issues is a nice feature. When I was professionally employed we used JIRA as our issue tracker. It did just that so we could create a series of issues, setting one as the master and others as children linked to it. Then it was easy to view the master and see the progress of the children. Sadly, I can't see any such provision in BitBucket.

    • Yes, I should think we do need the option for linking with superglue; and simage; and .... all the rest. Let's just sort out Coin first, then look at the other repositories. Although it is good to keep them in mind as we progress.
    • I've never looked at the 'Makefile.am' files before. I would guess that they had two builds - a regular one and a compact one (for when you wanted something smaller). I would think we only need the regular build so could dispense with the link hack. I would just want to check that all-<dir>-cpp.cpp isn't used anywhere else.
    • Again I have never looked at 'extractfont.cpp'. This is a separate utility that would seem to be more appropriate to a 'tools' repository, perhaps 'IvTools'.

    Roy

  28. Roy Walmsley

    Bastiaan,

    With respect to the issue of duplication of the input file list in doxygen I didn't say that I was deleting the cache every time I built. I used the menu item in the cmake-gui; I didn't separately clean the binary output directory. I never actually tried running CMake multiple times without deleting the cache.

    I'll read up on the CACHE INTERNAL attributes and see if that sheds any light. Also, I noted that there is an 'unset' command. I imagine that is also worth a look.

    Roy

  29. OgreT reporter

    I could not follow your problems with doxygen. But you can force cmake to delete the doxygen output before building the documentation and the doxygen file before cmake configuration. To fix the double COIN_DOCUMENTATION_FILES use, just add an unset on top of the main script.

    It would be helpful to delete all unused files (all-<dir>-cpp.cpp ?) so it is possible to to use the file-GLOB command. If cmake is working, you can also delete all makefiles - cmake create new files for building - and visual studio solutions.

    I'm really happy that you invest so much time in cmake. Of course this speeds up cross platform building.

  30. Mario Koerner

    I read the discussion above with interest. I like the idea of using CMake very much, because the VS project folders for the Windows files were always a bit hard to understand (and probably hard to maintain when making bigger changes).

    So I tried to build Coin using CMake and VS 2008 (Express). The configuration and project file generation worked fine, but it did not compile due to missing or duplicate macro definitions.

    I had a look at it and made some changes to the CMake branch:

    • Determine a suitable integer data type, when standard int types are not available
    • Removed compiler define HAVE_WINDOWS_H, because it is already defined in config.h (causes duplicate definition otherwise)
    • Fixed inconsistent new line characters in CMakeLists.txt
    • Added header file checks for stdint.h, stddef.h and sys/types.h
    • Added auto-linking pragmas, because they were available in old MSC builds
    • Fixed order of include directories (the cmake code for simage linking added the Coin install directory with highest priority - so compilation would always fail when a header file changes)
    • Install runtime targets (e.g. coin.dll) into "bin" folder (instead of "lib" folder)

    With these changes, I can successfully build, install and use Coin with CMake and VS 2008 (32 Bit, Dll version). I will submit a pull request with my changes. It would be great if you could review my changes (I'm writing CMake config files for the first time) and merge them to the CMake branch in the main repository.

    I also compared the compiler options generated with CMake (after my chages) with the old VS project in the build\msvc9 directory. There are some minor differences, but I think they are acceptable:

    • CMake sets compiler define "_MBCS". The old project neither set _UNICODE nor _MBCS, so I think it's ok to keep _MBCS set.
    • The old project set compiler define "_VC80_UPGRADE=0x0710". This seems to be set by MSVC when converting a project from an older version, but it's not well documented what it does. I think it's ok to not set it with CMake.
    • CMake sets compiler option /TP. I think this is ok, because Coin is written in C++.
    • CMake compiles with /O2 (maximize speed) and /Ob2 (allow inlining for all functions), while the old project used /Ox (full optimization) and /Ob1 (allow expansion only of functions marked inline). /Ox and /O2 seem to be very similar (see here), using /Ob2 seems also fine to me.

    I hope that my changes help to make progress on the CMake support in Coin. Is it still planned to merge the CMake branch to default? I think it still needs some work (e.g. to derive all defines from config.h correctly), but it's already working quite nicely. Which major efforts do you still see?

  31. Bastiaan Veelo

    Yes, the CMake branch will be merged eventually, when it is polished enough. It will get polish with use, so I would encourage using the CMake branch and help test it. Roy is doing a great job keeping both branches in synch, so they just differ in build procedures.

    Bastiaan.

  32. Thomas Moeller

    I've successfully built Coin on OS X and Windows for a while using the CMake branch. Therefore I agree with Mario and Wei regarding merging to default (main branch). It seems we are ready for CMake and going to main would simplify things.

  33. Wei Hong

    I just tried the CMake branch. I successfully made a build with visual studio 2013. My prototype works fine so far. I will run our unit tests later.

  34. Mario Koerner

    I would also vote for merging CMake to default, because functional development under Windows on the default branch is already quite hard. I only work on the CMake branch, which means quite some manual work to get functional changes also for the default branch. We also use the CMake build (Windows, VS 2013/2017) in a professional environment for quite a while now and did dot find major issues.

    The only thing we should probably do is to adapt the INSTALL and README files and state that the preferred way of building coin is to use CMake now.

    Does anybody know whether the old configure scripts are still working under Linux?

    @Fabio Rossi: Which examples do you mean? The examples folder in the Coin source code?

  35. Log in to comment