This is a good question. I interpreted that you get back the accelerations i.e. enough to make a timestep from your (input) initial values. In either case (Baseline or Lagrangian NN) it seems the accelerations are the training data - it is just how they are used. For predictions, the difference is how the accelerations for the next step are generated - the autodifferentiated L version being superior, presumably being more constrained by the Lagrangian rather than simply being some sort of ML interpolator of the training observations. While I can see this is potentially interesting for deducing underlying dynamics from observations, I am curious how it useful it is for chaotic systems. Consider the training data as samples of positions and velocities in the phase space (e.g. 4 dimensional for the double pendulum), then of course velocities (given) and accelerations indicate the tangent to a trajectory at each of these phase space points. However, given that trajectories of initially close points diverge in chaotic systems, how realistic is this for marching forwards?

In his older videos he used a PINN that was was an integrator to show that they are much better at long term predictability. I imagine it is exactly the same. These are local methods, not global. Since they are discrete methods they can't do any global derivations which would be symbolic. It is likely an impossible problem to have a global solver. Effectively you are then asking to be able to compute an exact timestep to get where you want with zero loss. This would, at the very least, require one to have the symbolic description of the system rather than just discrete sample points.

It's extremely unlikely that you happen upon an architecture that preserves symmetries by accident and then train that network so that it exactly fulfils symmetry relations. It's so improbable that you can be certain that the result will be wrong. By embedding the model in the lagrangian formalism the opposite becomes true: you're suddenly guaranteed that these symmetries are kept.