Curved neural network model proposed for explosive memory retrieval

In complex systems such as physical, biological, and social systems, higher-order interactions between components that simultaneously relate to each other have been revealed to play essential roles in generating emergent changes and diverse behaviors. These higher-order interactions are thought to be involved in information representation in biological networks like the brain and in improving the performance of artificial neural networks. However, theoretical frameworks that can handle them based on consistent principles had not been sufficiently developed to date.

An international collaborative research team consisting of Associate Professor Hideaki Shimazaki of the Graduate School of Informatics, Kyoto University (also Visiting Associate Professor at the Center for Human Nature, Artificial Intelligence, and Neuroscience at Hokkaido University), Researcher Miguel Aguilera of the Basque Center for Applied Mathematics (Spain), Chief Researcher Pablo A. Morales of Araya, and Assistant Professor Fernando E. Rosas of the University of Sussex (UK) extended the maximum entropy principle of statistical physics and proposed a new neural network model, "Curved Neural Networks" defined on curved statistical manifolds. This model is a new framework that can naturally incorporate higher-order interactions that could not be realized in conventional networks describing only pairwise interactions by using deformed exponential distributions based on Rényi entropy. The research was published in Nature Communications.

The research team first constructed a stochastic neural network model that follows deformed exponential distributions rather than conventional exponential distributions by extending the maximum entropy principle to Rényi entropy. When the deformation parameter has positive values, neural networks can be constructed on positively curved statistical manifolds (spherical type), and when the values are negative, they can be constructed on negatively curved statistical manifolds (saddle type). This enables structures that naturally encompass higher-order correlations through curvature to emerge even in low-order models that describe only pairwise interactions as before.

Based on this framework, the researchers conducted diverse theoretical analyses. Through mean-field analysis, they theoretically demonstrated that order-disorder phase transitions and spin glass phase transitions appear not as continuous changes but as abrupt "explosive phase transitions," creating regions where multiple memories are simultaneously stable. Additionally, through dynamic mean-field analysis and path integral methods, they theoretically showed that a self-regulating annealing mechanism emerges where temperature changes according to network state, enabling accelerated memory retrieval. Using the replica method, they conducted quantitative analysis of memory capacity and discovered a trade-off structure where negative deformation parameters increase memory capacity while positive deformation parameters suppress spurious memories.

It was confirmed that, through numerical simulations using visual image datasets commonly used in machine learning, the accuracy and speed of memory retrieval, as well as the degree of spurious memory mixture, change through curvature adjustment.

These results can be considered important achievements that provide a framework for a unified understanding of explosive phase transitions and multi-stability in memory models, as well as precision and capacity balance control.

Aguilera stated, "We have great expectations for the diverse possibilities that this research opens up. This new modeling method that captures higher-order statistical features found in biological neurons will lead to deeper understanding of brain mechanisms. It also brings new approaches to improving the information-encoding capabilities of artificial neural architectures used in image and language generation AI. We believe this framework will serve as a powerful clue in exploring the essence of information processing in both the natural and artificial intelligence fields."

Morales explained, "This paper presents an elegant approach to understanding the effects of higher-order interactions (HOIs) in complex systems. What is particularly impressive is that the generalized maximum entropy principle effectively captures HOIs without exponential increase in model parameters, enabling research on fascinating phenomena. This allows us to study systems showing explosive order-disorder phase transitions and multi-stability. These are not hard-coded but arise from the way memory spaces are constructed."

Rosas commented, "I am particularly excited about the discovery of self-regulating annealing as a fundamental mechanism that triggers explosive phase transitions from systems with higher-order interactions. This mechanism resembles the well-known annealing method but is characterized by unique feedback between energy and temperature, allowing the system to self-regulate. Feedback systems are prominent features of biological systems, and I find it particularly fascinating that such self-control mechanisms emerge in artificial neural systems."

Shimazaki said, "Creating neural networks in curved statistical spaces and theoretically handling the nonlinearity and higher-order interactions of neurons...! Research that began with such a simple idea developed through dialogue with colleagues who shared this vision, revealing unexpected behaviors of memory retrieval. The time spent pursuing unexpected developments with my colleagues was stimulating. We believe this research presents a framework that enables high-speed memory retrieval and adaptive control, proposing new design guidelines for AI."

Journal Information
Publication: Nature Communications
Title: Explosive neural networks via higher-order interactions in curved statistical manifolds
DOI: 10.1038/s41467-025-61475-w