Create test coverage reports

[Martin Sandve Alnæs]

It would be very useful if we could make test coverage reports:

• "local" coverage, i.e. for dolfin source code when running dolfin tests
• cross-product coverage, i.e. how much of ffc, uflacs, ufl, dijitso, instant is covered when running dolfin tests

Local test coverage in each project by itself is of course also useful.

In combination, these reports could help us delete unused code, and of course increase test coverage.

For inspiration: dijitso uses a pytest plugin and even contains a hack to get separate coverage reports per process when running with mpirun.

[Johannes Ring]

Bamboo now generates test coverage reports for dijitso, DOLFIN, FFC, FIAT and UFL. HTML reports are available as artifacts for each build, like for instance for DOLFIN: http://fenics-bamboo.simula.no:8085/browse/DOL-DODO/latest/artifact.

The coverage reports are also uploaded to coveralls.io. This allows us to keep track of changes in coverage over time. The FEniCS projects are available on https://coveralls.io/bitbucket/fenics-project.

We can also add coverage badges if desired:

dijitso
DOLFIN
FFC
FIAT
UFL

Note: I haven't looked at cross-product coverage yet.

[Prof Garth Wells]

Nice!

[Martin Sandve Alnæs]

Great, lets add the coverage badges to have something to stretch towards...

I think there's a lot of python code that's tested via the dolfin tests, in total we may be a little better than this but it's best to have good coverage locally per product.

[Marie Elisabeth Rognes]

Interesting!

-- Marie

[Prof Garth Wells]

Could we add the coverage tests to the README and close this issue?

[Johannes Ring]

Yes, I will take a look.

[Johannes Ring]

I have added a coverage badge for FIAT now: https://bitbucket.org/fenics-project/fiat. If it looks okay, I can add the same to the other repos.