Information Geometry, the Fisher Information Metric and Natural-Gradient Optimisation on Statistical Manifolds: chapter answers

A family of probability distributions can be treated as a space. How "far apart" are two nearby distributions? Does that distance change if the same family is described with different parameters? In which direction should an optimiser descend so that the choice of coordinates does not matter? This book answers these questions with mathematical statistics, information theory and optimisation theory, and checks, qualifies or corrects the popular claims in the author's framework one by one. It covers the Fisher information as a Riemannian metric and as the second-order term of the KL divergence…

DOI 10.5281/zenodo.23836268 · Read the monograph · 中文版

Illustrative model calculation, not a live quote