Make energy normalization common to models, and remove scikit-learn dependency

[Zachary Ulissi]

Scikit-learn is currently used to guess a per-atom energy in tflow. This isn't really necessary and should probably be removed.

Perhaps this per-atom energy guessing (energy normalization etc) could be handled by amp outside of the specific models.

[Oscar Lundberg]

How does the energy normalization work now? We are three students using your code in a project. While working in atom-centered mode we are trying to train on a data-set with different carbon materials in it.

The structures are having quite different energies, and after training amp seems to have extracted only the "larger" features in the sense that it can classify the different structures, however, we would like to reach the accuracy that it can classify the different carbon materials internally within one of the carbon materials (i.e. for molecular dynamics where the energy differences are small). That is, a more detailed energy classification.

Compare this with the forces, as the forces are calculated per atom. As far as we see the code calcualtes the energy per image rather than atoms?

Based on this it would be nice to have a normalization per atom and training based on this and forces per atom rather than the energy per image(structure).

Furthermore, is there any channel for discussion using amp?

[andrew_peterson]

The energy is internally calculated per atom. However, electronic structure codes can not calculate energy per atom, only per image. Therefore, to make a comparison, the Amp-predicted energies per atom are summed to an energy per image. Forces are inherently calculated on a per-atom basis in electronic structure calculators, so the comparison at a per-atom level is appropriate for forces and a per-image level for energies.

Because the range of outputs of a typical activation function for a neural network is [-1,+1], we need to have a final pair of parameters that brings this up to energy scale. That is, if the node output is "x", then we just do a final "mx + b" on the output, where m and b are also fit parameters. In principle, this "b" (intercept) parameter is large and is pretty close to the value of, for example, the energy calculated in the electronic structure calculator for a lone atom. E.g., it might be -314.38 eV for a C atom, -24.15 eV for a H atom, ... . (I am making those up.) We haven't tracked this, but it's likely that Amp spends a decent amount of time finding those intercepts during the optimization, but we could in principle make much better guesses for those, which is what Zack is referring to in his top post.

We haven't set up an amp-users mailing list, but it's probably a good idea to put one together one of these days.

[andrew_peterson]

If anybody wants to start on this: As a quick way to see the impact of making this change, it would be useful to just print out the value of these parameters at each step during the optimization, to see how much time the optimizer is wasting (or even getting stuck) because of not pre-normalizing the energy. This could be accomplished easily with an observer.

[andrew_peterson]

@zachary_ulissi Is this dependency still there? I don't see anything about it in tflow.py except a variable called scikit_model. How is this handled now?

I would still be interested in looking at this with the observer at some point...

[andrew_peterson]

Looks like this is gone, but there seems to be a scikit keyword in tflow that is still there. This should be removed.