On June 22nd, 2023, the paper titled “The Physical Effects of Learning” was published by PICS PhD student Menachem Stern, PICS faculty Andrea Liu, and PICS affiliated faculty Vijay Balasubramanian.
Abstract
Interacting many-body physical systems ranging from neural networks in the brain to folding
proteins to self-modifying electrical circuits can learn to perform specific tasks. This learning, both
in nature and in engineered systems, can occur through evolutionary selection or through dynamical rules that drive active learning from experience. Here, we show that learning leaves architectural imprints on the Hessian of a physical system. Compared to a generic organization of the system components, (a) the effective physical dimension of the response to inputs (the participation ratio of low-eigenvalue modes) decreases, (b) the response of physical degrees of freedom to random perturbations (or system “susceptibility”) increases, and (c) the low-eigenvalue eigenvectors of the Hessian align with the task. Overall, these effects suggest a method for discovering the task that a physical network may have been trained for.
Link
You can find the paper here.