Weird behavior corresponding to parallel comuting

[Alireza Khorshidi]

If we use a single core (cores=1), then I would expect that the method calculate_loss at Line 276 of model/init.py:

https://bitbucket.org/andrewpeterson/amp/src/089fb53e893123bccff32ebdaf905449d1a30202/model/init.py?at=master&fileviewer=file-view-default#init.py-276

is run only once. However, I am seeing that in the version v0.5 it is running eight times instead(!!), which leads to incorrect calculation of loss function and derivatives during training.

This issue seems crucial; without this being fixed, I cannot do further developments of v0.5.

[andrew_peterson]

Please post an example script that shows the problem.

[Alireza Khorshidi]

If you run the test_gaussian_neural.py (also given below), I expect that it goes through calculate_loss twice for each step of optimization, but it is going several times:

#!/usr/bin/env python
"""Simple test of the Amp calculator, using Gaussian descriptors and neural
network model. Randomly generates data with the EMT potential in MD
simulations."""

from ase.calculators.emt import EMT
from ase.lattice.surface import fcc110
from ase import Atoms, Atom
from ase.md.velocitydistribution import MaxwellBoltzmannDistribution
from ase import units
from ase.md import VelocityVerlet
from ase.constraints import FixAtoms

from amp import Amp
from amp.descriptor.gaussian import Gaussian
from amp.model.neuralnetwork import NeuralNetwork


def generate_data(count):
    """Generates test or training data with a simple MD simulation."""
    atoms = fcc110('Pt', (2, 2, 1), vacuum=7.)
    adsorbate = Atoms([Atom('Cu', atoms[3].position + (0., 0., 2.5)),
                       Atom('Cu', atoms[3].position + (0., 0., 5.))])
    atoms.extend(adsorbate)
    atoms.set_constraint(FixAtoms(indices=[0, 2]))
    atoms.set_calculator(EMT())
    MaxwellBoltzmannDistribution(atoms, 300. * units.kB)
    dyn = VelocityVerlet(atoms, dt=1. * units.fs)
    newatoms = atoms.copy()
    newatoms.set_calculator(EMT())
    newatoms.get_potential_energy()
    images = [newatoms]
    for step in range(count):
        dyn.run(5)
        newatoms = atoms.copy()
        newatoms.set_calculator(EMT())
        newatoms.get_potential_energy()
        images.append(newatoms)
    return images


def train_test():
    label = 'train_test/calc'
    train_images = generate_data(2)

    calc = Amp(descriptor=Gaussian(),
               model=NeuralNetwork(),
               label=label,
               cores=1)

    calc.train(images=train_images, energy_tol=10.**10., force_tol=None)
    for image in train_images[0:2]:
        print "energy =", calc.get_potential_energy(image)
        print "forces =", calc.get_forces(image)


if __name__ == '__main__':
    train_test()

[andrew_peterson]

I can't recreate this in the current version of the code as the code crashes in parallel mode. I created an [issue on this topic[(https://bitbucket.org/andrewpeterson/amp/issues/44/loss-function-issues-in-parallel). When it is fixed let me know if you are still confused on this.

[Alireza Khorshidi]

This was my mistake. It was taking multiple steps of optimization, and not running on multiple processes.