Using mathematics to prove system safety: Creating fundamental technologies for building trustworthy AI

Q1. What motivated you to choose this research theme?

A1. Realizing the importance of safety through robot contests

Actually, I didn't have a strong desire to become a researcher until I graduated from university. During my master's program, I was also job hunting, but ultimately, I decided that conducting research at a university was the best way to achieve the dreams I wanted to realize, which led me to where I am today.

The context for choosing my research field was my experience participating in robot contests when I was an undergraduate student. In these competitions, you need to build robots while pushing the limits of their performance. However, if a robot collides with a human, it can cause injury, and there's also the risk of motor damage or the compressed air used to move components exploding. Unlike software bugs, there are physical dangers. This experience made me realize the importance of "systems operating safely."

Q2. What are you currently working on?

A2. Exploring methods for clarifying evidence

I'm conducting research aimed at "making systems in the world operate safely." Compared with ordinary software, cyber-physical systems (CPS) such as autonomous vehicles and robots that operate in real environments have black-box interiors, making their behavior difficult to predict. I'm exploring methods to enable mathematical proof of whether such systems truly operate safely, have rapid detection of safety violations, or clear explanations of the grounds for safety. By using mathematics, computers can now automatically check or verify their own safety.

For example, in the case of autonomous vehicles, basic performance characteristics like "this car cannot exceed 60 kilometers per hour" or "it cannot stop within 3 seconds even with sudden braking" are known in advance. By combining this information with actual driving records, it becomes possible to prove that "no accidents occurred" even during periods when records are interrupted.

I'm also working on explaining safety. By using approximation models learned in human-interpretable formats, it becomes possible to verify system safety more efficiently and explain why it can be trusted even to non-experts. This is extremely important for trusting and using AI systems.

Q3. What message would you share with aspiring researchers?

A3. Doing research you love is the best approach

The fascinating part of this research lies in covering everything from theory to application. Since I used to consider becoming a programmer and actually enjoy implementation, I get excited when software that implements verification methods I've devised works as intended. In collaborative research with the industrial sector, I feel rewarded when another party is pleased that their work has become more efficient.

Long-term, I want to make mathematical verification technologies widely usable not only by large corporations but also by general companies. Recently, thanks to large language models (LLMs), even people unfamiliar with programming can create software. That's precisely why mechanisms for externally verifying safety will become even more important. To people aspiring to become researchers, I want to say that ultimately, doing the research you want to do is the best approach. I myself am more interested in building versatile methods applicable to various fields rather than in today's hot topics, but conversely, people who actively advance research in cutting-edge fields are also necessary. I think society achieves balance when various people conduct research that suits them.

(Article: Yasuhiro Hatabe)