TY - CHAP

T1 - Nerve Theorems for Fixed Points of Neural Networks

AU - Santander, Daniela Egas

AU - Ebli, Stefania

AU - Patania, Alice

AU - Sanderson, Nicole

AU - Burtscher, Felicia

AU - Morrison, Katherine

AU - Curto, Carina

N1 - Funding Information:
Acknowledgments This research is a product of one of the working groups at the Workshop for Women in Computational Topology (WinCompTop) in Canberra, Australia (1–5 July 2019). We thank the organizers of this workshop and the funding from NSF award CCF-1841455, the Mathematical Sciences Institute at ANU, the Australian Mathematical Sciences Institute (AMSI), and Association for Women in Mathematics that supported participants’ travel. We thank Caitlyn Parmelee for fruitful discussions that helped set the foundation for this work. We would also like to thank Joan Licata for valuable conversations at the WinCompTop workshop. CC and KM acknowledge funding from NIH R01 EB022862, NIH R01 NS120581, NSF DMS-1951165, and NSF DMS-1951599.
Publisher Copyright:
© 2022, The Author(s) and the Association for Women in Mathematics.

PY - 2022

Y1 - 2022

N2 - Nonlinear network dynamics are notoriously difficult to understand. Here we study a class of recurrent neural networks called combinatorial threshold-linear networks (CTLNs) whose dynamics are determined by the structure of a directed graph. They are a special case of TLNs, a popular framework for modeling neural activity in computational neuroscience. In prior work, CTLNs were found to be surprisingly tractable mathematically. For small networks, the fixed points of the network dynamics can often be completely determined via a series of graph rules that can be applied directly to the underlying graph. For larger networks, it remains a challenge to understand how the global structure of the network interacts with local properties. In this work, we propose a method of covering graphs of CTLNs with a set of smaller directional graphs that reflect the local flow of activity. While directional graphs may or may not have a feedforward architecture, their fixed point structure is indicative of feedforward dynamics. The combinatorial structure of the graph cover is captured by the nerve of the cover. The nerve is a smaller, simpler graph that is more amenable to graphical analysis. We present three nerve theorems that provide strong constraints on the fixed points of the underlying network from the structure of the nerve. We then illustrate the power of these theorems with some examples. Remarkably, we find that the nerve not only constrains the fixed points of CTLNs, but also gives insight into the transient and asymptotic dynamics. This is because the flow of activity in the network tends to follow the edges of the nerve.

AB - Nonlinear network dynamics are notoriously difficult to understand. Here we study a class of recurrent neural networks called combinatorial threshold-linear networks (CTLNs) whose dynamics are determined by the structure of a directed graph. They are a special case of TLNs, a popular framework for modeling neural activity in computational neuroscience. In prior work, CTLNs were found to be surprisingly tractable mathematically. For small networks, the fixed points of the network dynamics can often be completely determined via a series of graph rules that can be applied directly to the underlying graph. For larger networks, it remains a challenge to understand how the global structure of the network interacts with local properties. In this work, we propose a method of covering graphs of CTLNs with a set of smaller directional graphs that reflect the local flow of activity. While directional graphs may or may not have a feedforward architecture, their fixed point structure is indicative of feedforward dynamics. The combinatorial structure of the graph cover is captured by the nerve of the cover. The nerve is a smaller, simpler graph that is more amenable to graphical analysis. We present three nerve theorems that provide strong constraints on the fixed points of the underlying network from the structure of the nerve. We then illustrate the power of these theorems with some examples. Remarkably, we find that the nerve not only constrains the fixed points of CTLNs, but also gives insight into the transient and asymptotic dynamics. This is because the flow of activity in the network tends to follow the edges of the nerve.

UR - http://www.scopus.com/inward/record.url?scp=85130576982&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85130576982&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-95519-9_6

DO - 10.1007/978-3-030-95519-9_6

M3 - Chapter

AN - SCOPUS:85130576982

T3 - Association for Women in Mathematics Series

SP - 129

EP - 165

BT - Association for Women in Mathematics Series

PB - Springer Science and Business Media Deutschland GmbH

ER -