Remove randomness from tests

Issue #173 resolved
andrew_peterson repo owner created an issue

Some of our tests are inherently random. Here I'm referring to how they generate the training data through molecular dynamics. This leads to sporadic failure of the tests because we're not sure how to set the tolerance. Shall we just remove this randomness? We could either just make a trajectory by "manually" moving atoms in a script, or perhaps setting a random seed for the velocity distribution.

======================================================================
FAIL: Gaussian/Neural numeric-analytic consistency.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/users/ap31/scratch/amp-tests/20170809/env/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/gpfs/scratch/ap31/amp-tests/20170809/runtests/tests/misc_test/numeric_analytic_test.py", line 131, in test
    % (i, atom_no, image_no + 1, diff)
AssertionError: The calculated 1 force of atom 1 of image 2 is wrong! (Diff = 0.000017)
-------------------- >> begin captured stdout << ---------------------
numeric_analytic_test/analytic-True-1
numeric_analytic_test/analytic-True-2
numeric_analytic_test/analytic-False-1
numeric_analytic_test/analytic-False-2
numeric_analytic_test/numeric-True-1
numeric_analytic_test/numeric-True-2
numeric_analytic_test/numeric-False-1
numeric_analytic_test/numeric-False-2
('diff =', 2.2059580273570134e-09)
('diff =', 2.8467007574979608e-09)
('diff =', 9.4266736083703506e-10)
('diff =', 3.5535526354824043e-08)
('diff =', 1.7467202333421028e-05)

--------------------- >> end captured stdout << ----------------------

Comments (3)

  1. andrew_peterson reporter

    Things haven't been failing lately, but this is worth checking to make sure there is no randomness.

  2. andrew_peterson reporter

    I think all the problematic ones are fixed now. (Only those that test against known values matter; e.g., the nodeplot test doesn't matter if it's random.)

    A random seed option was added to the NeuralNetwork class to enable repeatable "random" initialization of internal NN parameters in commit 274e4f7.

  3. Log in to comment