Recognizing Damage from Natural Disasters Using Aerial Photographs Top Evaluation Obtained at International Conference Joint project by the NII, NICT, and Hitachi

A joint team comprising members of the National Institute of Informatics (NII), the National Institute of Information and Communications Technologies (NICT), and Hitachi, Ltd., participated in the TREC Video Retrieval Evaluation (TRECVID) conference, study group on technical evaluation in the field of video retrieval, hosted by the U.S. National Institute of Standards and Technology (NIST), and achieved the highest level of precision in the Disaster Scene Description and Indexing (DSDI) task aimed at disaster related image recognition. The results were presented during the TRECVID 2020 Workshop held online from December 8 through 11 of last year, and released to the public in detail as a Notebook Paper this March.

Quickly gaining an awareness of effected areas and the scope of damage after a large scale natural disaster occurs is growing increasingly important year by year from the standpoint of responding to disasters. The issue has been taken by the Cabinet Office Council for Science, Technology and Innovation (CSTI) as a topic of research for a national Strategic Innovation Promotion Program (SIP).

The DSDI task was added to TRECVID as a new challenge in 2020. The aim of the challenge is to take actual areal photographic data from disaster effected regions, and select and sort the top 100 images most likely to represent an example of each of the categories - 32 types of damage (disaster categories) including flooding, landslides, and rubble. The winner of the challenge is the team with the result closest to the correct order.

To complete the DSDI task, competing teams must use image recognition technology to identify the correct disaster category of each aerial photograph of the ground taken from low altitude. The team comprised of NII, NICT, and Hitachi used deep learning, specifically a combination of techniques such as label encoding, class imbalance learning, automated machine learning, and model ensemble methods, to achieve a strong evaluation for their extremely high precision recognition.

With further development, this technology is expected to support direct observation, and help greatly optimize disaster rescue operations accordingly, by automatically analyzing images collected over a large disaster area using drones or helicopters.

Accordingly, aerial photographs are expected to become a valuable source of information to prevent wasting valuable human resources as much as possible when responding to disasters.