CAREER COLLOQUIUM:

The notion of the stationary equilibrium ensemble has played a central role in statistical mechanics. In machine learning as well, training serves as generalized equilibration that drives the probability distribution of model parameters toward stationarity. In this talk, I first review the basic dynamics of machine learning. I then present a fluctuation-dissipation relation that links measurable quantities and hyperparameters in the stochastic gradient descent algorithm.