Using mathematics to understand intelligence and cultural evolution: Creating science that contributes to the advancement of the arts

Q1. How did you enter the research field?

A1. I have been interested in music and physics since childhood, and aimed to explore them using informatics

I have been interested in music since childhood, starting with piano lessons at the age of three. In junior high school, I became interested in science, especially physics, which uses mathematics and logic to explain natural phenomena. These experiences sparked my desire to study music scientifically, and here I am today.

While majoring in physics through my doctoral program, I was searching for a scientific field that matched what I wanted to do. There were some music-related courses at the university where I was enrolled, but they mainly examined cultural aspects and did not offer much of an objective approach based on a scientific perspective. After completing my degree in physics, I jumped into the world of informatics seeking to combine science and musical exploration. I thought that if I treated sound and musical scores as quantitative information and used algorithms and artificial intelligence to analyze and clarify them, I would be able to conduct research on music while utilizing my expertise in physics.

Q2. What is the significance and appeal of automatic musical transcription?

A2. Making sheet music created by experts more accessible to everyone and exploring the changing times through machine learning.

One of my research themes is "musical transcription." Musical transcription is the process of notating recorded music, including melodies and accompaniment parts, into sheet music. A significant portion of commercially available sheet music is created through the transcription skills of individuals with specialized expertise. Many people understand and perform music through completed sheet music, grasping its content and techniques. I am engaged in research to automate this process of "understanding and reproducing music" using computers, aiming to make it more accessible to everyone.

Automatic musical transcription uses a method called "machine learning," in which artificial intelligence (AI) learns rules and calculation methods from training data. Collecting the training data (musical data) is an immense task, requiring an understanding of such aspects as the history and culture of the music with which to deal. Incorporating the latest knowledge from both informatics and music into research is challenging, but it is rewarding, as both are essential elements that cannot be neglected.

In my FOREST research, I aim to understand the changing times of "culture," including music, using the methods of intelligence science and evolutionary science. I would like to contribute to the development of AI that evolves along with people and culture, such as understanding people's preferences and predicting and suggesting new styles of expression.

Q3. What is your research motto?

A3. I would like to give the results I obtain back to society. Cherish your own unique theme.

In our research to date, we have presented the automatic composition system "CREEVO," arrangements by AI, and a system to create brass band scores from piano scores. By releasing the results of our research as open data, we dream not only of returning the benefits of these findings to society and further developing them but also of realizing a world in which everyone can easily enjoy the arts.

I believe that I am still a researcher today because I pursued my interest in music and science, which dates back to when I was a child. Staying true to your own interests is a challenging journey, but the joy that you acquire from it is equally significant. It is also important to value the uniqueness of the theme that you have chosen. To produce results and survive in the research competition, I hope that you will never give up and continue to refine your own unique value in your daily studies and research.

(TEXT by Manami Yokoi)