test_symbolic_geometry_foo tests are very slow

[Prof Garth Wells]

The JIT for the test_symbolic_geometry_foo tests is very slow. Do we really need so much JIT? Is it testing DOLFIN or FFC/UFL?

[Martin Sandve Alnæs]

The advantages of having the tests in dolfin is that they cover the full pipeline with both ffc and uflacs. I wrote them for the symbolic geometry in ufl and uflacs generation but found several bugs in ffc as a result, proving the value. It's very costly to write perfect unit tests of every single piece of code through the assembly and code generation pipeline, and also much more work to maintain such tests than more highlevel tests like these.

[Martin Sandve Alnæs]

Constructive ways forward:

1) A general approach to dealing with slow tests: Marking slow tests with pytest and make cmake-targets running only fast tests. We could even have three levels: fast, normal, and slow. This needs buy-in from all the devs and continuous small improvements over time to stay useful.

2) A general performance improvement for jit: Finish use of ctypes instead of swig in ffc.

3) Specific iteration on this particular test: reduce number of test cases. Maybe move some of the tests to more low-level code generation tests in uflacs/ffc. I have a framework for jitless testing of code pieces in uflacs using a combination of pytest and gtest. This could be used more.

Doing 1-2 is much more valuable than 3, which I can't realistically promise to do now.

[Martin Sandve Alnæs]

For (1), see https://bitbucket.org/fenics-project/dolfin/issues/619/framework-for-dealing-with-fast-and-slow

[Prof Garth Wells]

Throwing right out there, could we do JIT in memory, maybe requiring LLVM? I notice a big difference in test speed on an HDD vs an SSD.

[Lawrence Mitchell]

FWIW, our experience is that once you get swig out the way, jitting is noticeably faster (even on disk). We've looked at LLVM in-memory jitting in passing, but haven't got actually got any working code: I think clang is available in library form which might be good enough, unless you want to just write llvm bitcode directly from the form-compiler.

[Martin Sandve Alnæs]

Based on my rough tests some time ago getting swig out of the way will at least halve jit time for most forms.

Not compiling elements again for each form would be the next step.

Both these tasks have issues in ffc.

[Anders Logg (Chalmers)]

Removing milestone: 1.7 (automated comment)