Switch unit test framework works to googltest and pytest

[Prof Garth Wells]

See discussion thread at:

https://www.mail-archive.com/fenics%40fenicsproject.org/msg01285.html

[Martin Sandve Alnæs]

Our summer intern will look at this in July.

[Anders Logg (Chalmers)]

As part of this work, I'd like to add the following. Runnint the tests for DOLFIN is really expensive, which is a pain for developers who need to run the tests frequently. Not even make run_quicktest is very fast. We need to go through the tests and make sure that they run fast, for example by not using excessively large mesh sizes. The new testing framework should print out a summary of timings for different tests so that we may target tests that are too expensive. When testing demos, perhaps we could extend expensive demos with command-line arguments that when set will choose a smaller mesh, so that the testing framework may run a demo by demo_foo --quick.

[Prof Garth Wells]

I agree that some of the unit tests are too slow. It would be nice if the unit tests could be run in parallel.

[Johan Hake]

With py.test you can run the tests in parallel (threaded) and it supports annotating the tests with labels such as fast, slow aso.

[Martin Sandve Alnæs]

Aslak has worked on transitioning the unit/ directory to pytest and his work is in a branch. I'll see to completing and merging it and coordinate with Johannes for the buildbot changes.

His internship is over though, so he won't do the rest of the dolfin tests.

(He might work a few more hours for us later in august but that will be to help finish the dolfin python3 support.)

UFL is now using py.test, and UFLACS uses py.test and google test, working on all the buildbots.

[Martin Sandve Alnæs]

We're getting some more help on this, hopefully Aslak can finish at least the <dolfin>/test/unit/ part. Here's the intermediate work for now, rebased on latest master:

https://bitbucket.org/fenics-project/dolfin/branch/aslakbergersen/topic-pytest-cleanedupbranch

[Martin Sandve Alnæs]

Does anyone mind if we reorganize the tests into cpp/{foo,bar}, python/{foo,bar} instead of foo/{cpp,python}, bar/{cpp,python}?

In fact, moving "foo"/"bar" into the filenames and using a flat structure unit/python/test_{foo,bar}_*.py would be much easier to work with. In particular it would make it easier to share fixtures across all tests, allowing e.g. initializing unit square mesh and CG1 function space once to reuse across test files. With py.test we can use a filename pattern to select tests, e.g. "py.test -k test_la_" could run test_la_*.py.

[Prof Garth Wells]

@martinal Fine with me.

[Martin Sandve Alnæs]

Closing this and creating more specific issues for work left to do.

[Prof Garth Wells]

Removing milestone: 1.5 (automated comment)