cmake for blazemark?

[Mario Emmenlauer]

Dear Klaus and all,

I love the new cmake based build instructions for blaze!

I also use blazemark a lot, and I would like to continue using it for testing different compilers and flags on different platforms. This would be much easier using cmake. I want to mention that the current build script is a very nice, disciplied work and certainly has its beauty. But it seems a significant effort to maintain, and it might become more and more challenging to extend it over time?

Can you help me understand if the "hard-coded" shell script is required? Are the different benchmarks very different in their flags or compile options? If not, I think it should be possible with reasonable effort to use cmake instead?

In my (currently naiive) understanding, it should be possible to make two cmake loops over the different benchmarks and libraries, and shrink the build quite a lot. cmake can also generate the required headers. Actually the headers might not even be required with cmake, because it can quite easily add the relevant prepocessor definitions to the build.

What do you think, would this be an acceptable change? I can certainly help implement it, in case there is a chance to get it adopted?

All the best,

Mario

[Klaus Iglberger]

Hi Mario!

Thanks a lot for the proposal. Although you have rated it as a major enhancement we have to admit that from our current point of view it of minor importance. We don't expect that more than a handful of people are using the Blazemark. Also, since most of the frameworks that we compare in the Blazemark are not actively developed anymore (Blitz++, FLENS, Boost uBLAS, Armadillo, MTL4) we have already thought about removing the Blazemark in the Blaze 4.0 release (whenever this will happen). Therefore in the end it is mainly for our own convenience (and for yours, apparently).

You are absolutely correct that from the maintainability point of view a cmake approach would be better. Unfortunately, currently we are not willing to invest time into changing this aspect, since the Blazemark is pretty stable (there have been no changes in the last 2 years) and there is many (much) more important issues (reduction operations, N-dimensional data structures, solvers, sparse matrix extensions, etc). You are certainly free to improve things, but please keep in mind that we would still have to perform the necessary review (which, given the size of the change, would also require quite some time).

We will leave this issue open, though, since the idea is sound and we have not yet decided about the future of the Blazemark. Therefore there is the potential that we revisit this issue at a later time. Thanks again for proposing the idea,

Best regards,

Klaus!

[Mario Emmenlauer]

Hi Klaus,

thanks for the very helpful reply, its appreciated! I think the priority was set automatically, sorry for that, its certainly not major.

Personally I like blazemark a lot, and right now its quite helpful for me.

Can you help me understand a bit better if the blazemark build would be difficult to batch into a loop? In my understanding, the configure/Makefile are very homogeneous for the individual tests. From a first look it seems relatively straightforward to automate this in cmake. I'll probably give it a try and let you know how it goes.

All the best, Mario

[Klaus Iglberger]

Hi Mario!

The most important feature of the current setup (i.e. the one that we care about most) is to be able to build individual benchmarks, e.g.

make dvecdvecadd

Building all tests at once is never used explicitly, but only within our CI environment. I don't see how the loop approach would help with building individual benchmarks, but if it does, great.

Best regards,

Klaus!

[Klaus Iglberger]

Hi Mario!

Since the Blazemark is deprecated since the release of Blaze 3.5, this issue is unfortunately obsolete. Thus we will close the issue as “wontfix”. Still, thanks for proposing this enhancement,

Best regards,

Klaus!