Relative force training

[andrew_peterson]

It makes sense that we should train smaller forces to a tighter convergence than larger forces. E.g., we care about a 0.03 eV/A error if the force is 0.02 eV/A, but we don't care (much) about that error if the force is 12.23 eV/A. However, if we naively try to implement this as a relative force threshold (e.g., all forces are within 1% of their true force), we hit a problem because zero forces are allowable, and are quite common in our training sets as they represent stationary points on the PES.

As a solution, we discussed a criteria that, for example, all force residuals would be below: max(0.005, 0.01 * force), for example. Call this max(A, B * force). This would introduce an "if" statement which would make writing derivatives hard. Instead, we could make something smooth that approximates this. Then the component that goes into the cost function should be something like

( Delta F_i ^2) / (A^2 + B^2 F_i^2)

If this were used as a convergence criteria, then when the above formula is greater than one (for any force) that component is unconverged.

In practice, we can still decide convergence based on the 'max' formula, but if we put the above math in our cost function it should find this fairly efficiently with gradient descent.

[Andrew Doyle]

If we're concerned about division by zero, wouldn't the easiest way to avoid it be by adding a constant to the denominator? For instance:

|Force| / ( |Force| + A)

The magnitude of the force must be nonnegative, so for any positive value of A this is guaranteed to have a finite value. Admittedly this introduces another constant we could argue about (as it will impact how severe the difference in scaling is between small forces and large forces), but it seems like a reasonable simple way to start testing.

Edited - I spoke with Zach today and it seems I misunderstood the notation you used in the initial post. It seems you've beat me to this idea. :)