Fix Amp tests for tensorflow
Currently the nightly tests crash on the tensorflow module. I think this is just a lack of an import. This will need to be fixed by me (Andy).
Comments (4)
-
-
Could you comment on the warning "OptimizeWarning: Unknown solver options: factr result = scipy.optimize.minimize(minimize_args, *minimize_kwargs)"?
-
That warning has to do with the
scipy.optimize.minimize
"wrapper" not accepting thefactr
solver option. Thefactr
keyword argument is set here from the moduletflow.py
in Amp. TensorFlow uses ADAM or L-BFGS-B as optimizers. In Amp the latter is default. As I see it, the tensorflow'sScipyOptimizerInterface
module is imported fromtflow.py
. Then it receives,ExternalOptimizerInterface
as an argument, and does:import scipy.optimize # pylint: disable=g-import-not-at-top result = scipy.optimize.minimize(*minimize_args, **minimize_kwargs)
and the scipy.optimize.minimize does not accept
factr
. I think the way around this issue would be changing line 1171 intflow.py
to useftol
instead offactr
mentioned here. -
I have observed some inconsistencies in the prediction from tensorflow NN. I slightly modified the test script for this module:
diff --git a/tests/test_gaussian_tflow.py b/tests/test_gaussian_tflow.py index 69f2382..f83ec96 100644 --- a/tests/test_gaussian_tflow.py +++ b/tests/test_gaussian_tflow.py @@ -62,21 +62,28 @@ def train_test(): from amp.model.tflow import NeuralNetwork label = 'train_test/calc' - train_images = generate_data(2) + train_images = generate_data(15) + train_images = train_images[0:10] + print('Length: {}' .format(len(train_images))) convergence = { 'energy_rmse': 0.02, 'force_rmse': 0.02 } calc = Amp(descriptor=Gaussian(), - model=NeuralNetwork(hiddenlayers=(3, 3), + model=NeuralNetwork(hiddenlayers=(5, 5), convergenceCriteria=convergence), label=label, cores=1) calc.train(images=train_images,) + train_images = train_images[-5:] + print('Test Length: {}' .format(len(train_images))) + for image in train_images: + print("Energy =", image.get_potential_energy()) print("energy =", calc.get_potential_energy(image)) + print("Forces =", image.get_forces()) print("forces =", calc.get_forces(image))
Below, I am posting target values vs predictions for one of the unseen images using TensorFlow 0.11.0. Properties with a capital letter are those coming from the original calculator and the others are ML predictions:
Test Length: 5 ('Energy =', 8.310588888972827) ('energy =', array([8.664198], dtype=float32)) ('Forces =', array([[ 0. , 0. , 0. ], [-1.25839239, -0.52908486, -0.44971333], [ 0. , 0. , 0. ], [-0.3558867 , 1.70085379, -0.58707117], [ 0.63315985, -0.92845858, 0.13016976], [-0.26600996, 0.45560162, 0.01447438], [ 1.93367376, -0.06405294, 0.65480354], [ 0.68849322, -0.24157533, 1.40880371], [ 0.12725669, 0.05685044, 0.38817817], [-0.08650451, -0.25796725, -0.62260205]])) ('forces =', array([[ 3.6847105e-03, 3.5128396e-03, 9.5515354e-03], [ 8.2092658e-03, -3.8295847e-04, 5.4547829e-03], [ 2.4447001e-03, -2.2697991e-03, 8.1457123e-03], [ 5.5896752e-03, -6.0966425e-03, 7.8025507e-03], [-4.7097844e-03, 3.6573287e-03, -6.8126558e-03], [ 1.0994184e-03, -6.6993287e-04, -6.6648466e-03], [-1.1098579e-02, 1.8842700e-03, -7.8503452e-03], [-5.3612911e-03, 8.1996399e-04, -1.6735876e-02], [ 1.8541722e-04, -1.7552178e-03, 7.1176267e-03], [-4.3533884e-05, 1.3001483e-03, -8.4821713e-06]], dtype=float32))
As you can see, energy predictions seem right, but for forces, it is not the case. Using TensorFlow 1.6.0 is the same:
Test Length: 5 ('Energy =', 8.183153071890118) ('energy =', array([8.726102], dtype=float32)) ('Forces =', array([[ 0. , 0. , 0. ], [-0.09430469, -1.75691708, -0.74334414], [ 0. , 0. , 0. ], [-0.32062687, 0.82518729, -0.20991994], [-0.00854701, -0.06651756, 0.76368747], [ 0.87476395, -0.26460163, 0.45419152], [ 1.15732471, 0.3518591 , 0.50060917], [-1.00486088, 0.73141072, 0.78732044], [ 0.84537417, -0.2599461 , 0.35097638], [-0.94463245, 0.28454476, -1.08690144]])) ('forces =', array([[ 0.00204374, -0.00087876, -0.00648994], [-0.00154252, 0.00188018, -0.00022534], [ 0.00164029, 0.00079411, -0.00620799], [-0.00169598, -0.00332731, -0.00135216], [-0.00205044, -0.00030446, 0.00346738], [ 0.00194829, 0.0006923 , 0.00235331], [-0.00176497, 0.00147636, 0.00717454], [ 0.00087587, -0.00012318, 0.00458725], [-0.00537482, -0.00309596, -0.00819426], [ 0.00592053, 0.00288671, 0.00488721]], dtype=float32))
Finally, by reverting the changes above and printing each prediction for the two images in the original test we get:
('Energy =', 9.391512796710966) ('energy =', array([8.907822], dtype=float32)) ('Forces =', array([[ 0.00000000e+00, 0.00000000e+00, 0.00000000e+00], [ 1.13878891e-01, -7.27048925e-02, 1.66379366e+00], [ 0.00000000e+00, 0.00000000e+00, 0.00000000e+00], [ 1.13878891e-01, 7.27048925e-02, 1.66379366e+00], [-2.21177243e-17, -2.95553512e-16, -1.73360473e+00], [ 0.00000000e+00, 2.26635248e-15, -1.59103977e+00], [-1.75207071e-15, 1.76594850e-15, -1.72730447e+00], [-5.34294831e-16, 3.55271368e-15, -7.41800987e-01], [-4.11996826e-18, -8.32667268e-17, 8.09683455e-01], [ 0.00000000e+00, 0.00000000e+00, -1.67110814e+00]])) ('forces =', array([[ 2.7275153e-03, -6.7170954e-04, 2.5449730e-03], [-2.7275153e-03, -6.7170954e-04, 2.5449730e-03], [ 2.7275153e-03, 6.7170954e-04, 2.5449730e-03], [-2.7275153e-03, 6.7170954e-04, 2.5449730e-03], [-3.9048289e-19, -9.1108568e-19, -7.5114728e-04], [-9.1027742e-19, -4.8418743e-19, -1.1872562e-03], [-3.9402485e-19, 2.0781786e-19, -4.4262956e-04], [-7.3480563e-19, 3.4504131e-19, 1.3120837e-03], [-4.1095448e-21, 3.0218169e-19, -1.3046423e-02], [ 1.2543149e-23, -1.6664003e-19, 3.9354805e-03]], dtype=float32)) ('Energy =', 8.765803171959153) ('energy =', array([8.910725], dtype=float32)) ('Forces =', array([[ 0. , 0. , 0. ], [ 0.04438365, -0.0252446 , 1.16583466], [ 0. , 0. , 0. ], [ 0.00512817, 0.0735986 , 0.99858469], [ 0.13447516, 0.04628451, -1.25258365], [-0.16177521, 0.00476531, -1.12960904], [ 0.13352245, -0.06903842, -1.13518543], [-0.02881105, -0.11618577, -0.6047823 ], [ 0.01005221, 0.2448184 , -2.56701604], [-0.01026368, -0.23819793, 1.98938524]])) ('forces =', array([[-2.4364408e-04, -1.0109526e-03, -1.0723310e-04], [-7.4692571e-04, 1.5144423e-03, -2.9116340e-03], [-8.9384930e-04, 1.6709700e-03, -1.3705894e-03], [-8.4772491e-04, -9.0343715e-04, -4.4189831e-03], [ 3.4581765e-03, 4.7460230e-04, 2.1660191e-03], [-2.9057818e-03, 1.2856203e-03, 4.0753628e-03], [ 1.6220108e-03, -6.9638854e-04, 3.0755636e-03], [ 3.7753131e-04, -2.2243659e-03, 6.3252676e-04], [ 1.6954025e-04, -2.2936975e-04, -2.7846899e-03], [ 1.0666315e-05, 1.1887904e-04, 1.6436602e-03]], dtype=float32))
It seems to be a problem in the loss function minimization?
- Log in to comment
I was able to compile tensorflow 0.11.0 to be used in the cluster. It is located at
data/tensorflow-0.11.0-py2-none-any.whl
. It can be used withtest_gaussian_tflow.py
: