Development of an artificial neural network capable of discriminating natural sound — Human-like response to changes in sound amplitude discovered by NTT

NTT announced that it has discovered that artificial neural networks (NNs), which can distinguish natural sounds that humans hear every day, exhibit human-like responses to changes in sound amplitude. These results provide a unified understanding of the human perceptual characteristics of amplitude modulation (AM), as investigated in perceptual psychology studies, and AM processing by the brain, as investigated in neuroscience studies, within a single framework. In the future, the company expects this achievement to be applied in various fields, including the development of devices that more closely resemble the mechanism of the human ear in fields such as medicine and welfare. These findings were published in the Journal of Neuroscience.

NTT Communication Science Laboratories have been working with NNs to understand auditory AM processing. The AM sound was input to NNs trained via machine learning to recognize natural sounds, and their responses were similar to those of the brains of animals that heard AM sounds. This suggests that the animal brain's response to the AM sound may also have been formed as a result of adaptation to recognize natural sounds. However, only the response characteristics of a single neuron in the brain have been studied, and the relationship between perception and sound recognition, which would consist of the actions of many neurons, is not yet understood. Until now, only the brains of non-human animals had been considered, and it was not known whether the same framework could explain human perception, which cannot be easily measured from a single neuron.

In response, NTT has conducted a new study on NNs to compare them with human perceptual characteristics and reveal their similarities. As a target perceptual characteristic, the study focused on the minimum depth limit at which humans can hear AM (AM detection threshold), which has been investigated in many auditory studies.

Simulated perceptual and neural activity recording experiments on NNs trained in natural sound recognition indicated that the NNs exhibit human-like AM detection threshold characteristics, even though the nature of human and animal hearing was not considered in the construction of the NNs. AM in natural sounds has been found to be important in maintaining this property.

The regions in the model with human-like AM detection threshold characteristics have also been found to correspond to the inferior colliculus, medial geniculate body, and auditory cortex of the brain, providing insight into the brain regions involved in human AM detection. It should be noted that the AM patterns of natural sounds used for training are also important for NNs to acquire an AM detection threshold similar to that of humans.

The results of the aforementioned studies provide a unified explanation for previous findings in perceptual psychology and neuroscience as a result of adaptation to natural sounds. Researchers in the field of perceptual psychology have also proposed excellent models of human AM detection thresholds to explain and understand perceptual properties such as detection thresholds.

NTT will continue to clarify the correspondence between the processing stages in these existing models and the model constructed in this study and will examine in detail which stages of auditory information processing can or cannot be explained by adaptation to sound recognition.

Journal Information
Publication: Journal of Neuroscience
Title: Human-Like Modulation Sensitivity Emerging through Optimization to Natural Sound Recognition
DOI: 10.1523/JNEUROSCI.2002-22.2023