[Very simple abstract setup, describing an ensemble of physical systems through a probability distribution over phase space, and dynamics by a differential equation.] Then, the well-known Liouville equation describes the dynamics of this distribution. For this type of evolution the Kullback-Leibler information measure provides a convenient way to measure the distance between two distinct probability distributions P1 and P2 because, remarkably, it is invariant under dynamical changes prescribed by the Liouville equation.
I’m not a physicist, but I feel compelled to annotate that — in the style of chess games — with a (!) (although I don’t think this is really a physics result). The Kullback-Leibler is a well-known (if not the) information distance between distributions, but I had no idea that it was preserved by such a wide range of dynamic systems.