KL divergence
Also: Kullback-Leibler divergence, relative entropy
How much information is lost when one distribution is used in place of another, in bits or nats. Preference training uses it as a leash, penalising the tuned model for straying from the model it started as.
What people get wrong
Not a distance: it is asymmetric, so KL(P‖Q) and KL(Q‖P) differ, and which order you use changes whether the approximation spreads out or concentrates.
A definition is not an understanding. This is taught properly in Entropy, cross-entropy, KL and perplexity, part of Probability & statistics.
Learn this properly