FFC reference data update is not childproof

[Anders Logg (Chalmers)]

Regression tests were broken on master because the ffc-reference-data repository was out of sync. Various other problems turned up after some use. A new approach is attempted. (text edited by martinal)

[Martin Sandve Alnæs]

Since git status tells me the ffc-reference-data directory is not part of the repository, I removed that directory and ran cd regression; python test.py. This gives

Looking for reference data ancestor of 45359bf88a4cc57cbd4a8f2aebc57a5ffb4b54ad
./download-references: line 35: cd: ffc-reference-data: No such file or directory

Then I did:

mkdir ffc-reference-data
python test.py

and get this instead:

Looking for reference data ancestor of 45359bf88a4cc57cbd4a8f2aebc57a5ffb4b54ad
Unable to find reference data ancestor
Download reference data failed
Running regression tests with -r auto
...

The reason I removed this dir is because the tests "pass" even though the download fails, i.e. before I did this I got:

Validating generated code (53 header files found)
-------------------------------------------------
Missing reference for /home/martinal/dev/fenics/ffc/test/regression/ffc-reference-data/current/r_quadrature_O/AdaptivePoisson.h
...

but it still tells me "Regression tests OK" at the end.

Can we please please please get back the simple, clear, and actually working concept of just including the reference data where it belongs, in the repository? I severely doubt that anyone will take the time to actually make this system failproof.

[Martin Sandve Alnæs]

The downloading part is fixed here martinal/fix-regression-downloading if that's the proper solution. Most header files differ now, so it seems the tests have been broken by not running properly for a while.

[Martin Sandve Alnæs]

The diffs are signatures only, so that could be local ufl changes since I've added a new type.

I don't understand the reference data dir, what are ffc-reference-data-{0,1,2,3,4,5}?

Why is there a recursive symbolic link in there?

martinal@martinal-mac:~/dev/fenics/ffc/test/regression$ ls ffc-reference-data/ffc-reference-data-c745f186c302fff0ef6592af4a509ab25a396da5/ffc-reference-data-cbb7a1a4366eabe33c91936432c1000bd92aacca ls: cannot access ffc-reference-data/ffc-reference-data-c745f186c302fff0ef6592af4a509ab25a396da5/ffc-reference-data-cbb7a1a4366eabe33c91936432c1000bd92aacca: Too many levels of symbolic links

Will the reference data scripts really check out all previously stored reference data in my working directory? The ffc-reference-data/ directory is already 446 M, many times as large as the entire compressed history of references that git kept for us before.

Why do I have to make an additional commit to ffc after upload_references, to include the new line in ffc-reference-data.log? I don't see what purpose that serves other than cluttering the history.

[Martin Sandve Alnæs]

What do I do with merge conflicts in ffc-reference-data.log?

[Martin Sandve Alnæs]

I'm fixing this broken design by simply storing a file in ffc with the commit id from ffc-reference-data.

This has a number of positive properties. Scripts dont depend on behaviour of git rev-list. No sym-links necessary. A single checked-out copy of reference data solves the data explosion. Much easier to find which reference data state matches a given historic ffc repository state since the reference commit id is stored with the ffc state. Can make it easy to get a previous reference state and check against that. Clearer for the developer in a merge situation.

[Martin Sandve Alnæs]

New system is in next, will merge into master tomorrow.

[Anders Logg (Chalmers)]

As long as it works for everyone I'm happy if you change it.

[Martin Sandve Alnæs]

Already did :) And it seems to work pretty well now. I've tried to handle all corner cases but haven't tested them all so scream out if there are any issues left.

I'll update the readme with instructions for different use cases later.

[Martin Sandve Alnæs]

All issues known (to me) should be resolved now.

[Anders Logg (Chalmers)]

Great!