the type of weights in tflow model is not compatible with that in the normal NN model

[Yin-Jia Zhang]

In tflow model, if you set the weights to be the ones extracted from a normal NN model. It will give an error message in the following location of codes. It is caused by that the data type difference. The type of weights[element][1] from normal NN model is a list; while it requires a numpy array here in tflow model.

    if weights is not None:
        self.elementFingerprintLengths = {}
        self.elements = weights.keys()
        for element in self.elements:
            self.elementFingerprintLengths[element] =\
                weights[element][1].shape[0] - 1

Also, in the parameter descriptions, the keyword 'weights' is described as a numpy array type. It should be a dictionary. ;)

[andrew_peterson]

I think there is one other slight difference in how the network is specified that makes the two not quite identical. (By accident, I think we could make them identical.) I think if you specify hidden layers as (20, 20) for example, in one of the implementations we get a neural network like

Fingerprint -> 20 -> 20 -> 1 -> Energy

And in the other:

Fingerprint -> 20 -> 20 -> Energy

That is, I think one uses an extra parameter or two at the end to scale up the neural net output to an energy, and the other does this directly with the neural network parameters. I may not be remembering this right. @akhorshi or @zachary_ulissi would know. But my point is that we may need to reconcile how the networks are structured before we can transfer parameters between them.

[Zachary Ulissi]

Yeah, it's whether or not it's just a linear combination of the output layer nodes, or whether it goes through a final activation function. I think the tflow one is more general, but I don't think it really matters.

On that note, check the tflow test force/energy test case to show how to set weights directly to make things identical between the two modules.

[Yin-Jia Zhang]

Thanks a lot for the information! I just want to test how the same weights and same training set will behave in two models. It would be nice if we can make such a comparison in future.

[Zachary Ulissi]

Same weights and same activation functions should yield identical results between the two models (that's basically the energy/force test case, it currently passes). If they're not identical, there's a problem.

[andrew_peterson]

Yin-Jia was having convergence issues with the tf version. If I understand, with the regular neural network it was slowly converging, while with the tf version it was quickly not converging, using the same training data and same hidden layers specification. So I am guessing she wants to try both from the same guess of initial NN parameters to try to figure out what's going on.... if one is just making a better initial guess than the other.

Without modifying the current version, is there a way to make the NN architectures identical? E.g., could we specity (20, 20) in one and (20, 20, 1) in the other?

[Yin-Jia Zhang]

Thanks for Andy's explanation. That's what I am struggling about. Is there any good way of comparing?