NTT develops AI that accurately recognizes rust from MMS images for detection in social infrastructure equipment: Introduced at the Tsukuba Forum

NTT has succeeded in detecting rust on various social infrastructure facilities with high accuracy using image recognition AI. The company confirmed that this image recognition AI can identify multiple infrastructure facilities, such as road guardrails, signs, mirrors and cables and transformers on utility poles, from roadside images acquired using a mobile mapping system (MMS) and can detect rust on each facility with 97.5% accuracy. This accuracy is the ratio of the number of images in which rust is correctly detected by the image recognition AI divided by the number of images in which the occurrence of rust is visually confirmed, meaning that the image recognition AI can detect rust almost as well as a visual check by a person. NTT exhibited and introduced this technology at the Tsukuba Forum 2022, which the company held on May 18 and 19 at its Tsukuba Research and Development Center in Tsukuba City, Ibaraki prefecture.

Social infrastructure such as roads are facing issues including deterioration due to age, increasing inspection costs, and a shortage of inspection personnel. To address this situation, the Japanese government's Growth Strategy 2018 seeks to increase the percentage of infrastructure managers who deploy new technologies such as sensing and AI to 100% by 2030. The NTT Group is working to reach this goal by using a 4D digital platform, which integrates, accumulates, and analyzes a variety of sensing data obtained from society in digital space. It is conducting inspections (data acquisition and automated analysis) using MMS, drones, and other technologies.

In these developments the group developed technology to capture images of roadside infrastructure in a single run by using multiple high-resolution MMS cameras in order to capture images of roadside infrastructure, and successfully developed an image recognition AI for high-accuracy detection of rust on the facilities from the images. NTT West used MMS to capture images of installed infrastructure facilities, and a vehicle equipped with MMS photographed roadside infrastructure at regular intervals. The vehicle's side-facing digital camera took images of roadside equipment such as guardrails, signs and mirrors, while the vehicle's upward-facing digital camera took images of utility pole-mounted equipment such as hardware and cables. Then, for the captured 1,000 images of roadside equipment (587 images showing rust on equipment) and 1,000 images of equipment on utility poles (135 images showing rust on equipment), NTT used their image recognition AI to recognize the facilities and detect rust on each piece of equipment. The results of the experiment confirmed that the system correctly recognized facilities (94.3% recognition rate) in 1,885 out of 2,000 images showing roadside equipment and utility pole-mounted equipment, and correctly detected rust (97.5% detection rate) in 704 out of 722 images with rust.

NTT's newly developed recognition AI makes it possible for separate field inspections currently performed by each infrastructure manager to be consolidated into one MMS run that can acquire images in batches, thereby reducing the number of operations required. The image recognition AI is also expected to improve the uniformity of inspection quality because it can detect areas of rust with high accuracy and uses uniform standards from captured images.

This image recognition AI built by NTT was trained sufficiently and evenly on images containing a variety of facility types and shapes and having different lighting conditions and compositions to achieve correct recognition of many different types of facilities. For example, it can recognize types of roadside equipment such as guardrails, signs, and mirrors, and distinguish parts of utility pole-mounted equipment such as metal hardware and cables. The AI detects rust in each region on a pixelwise basis. The technology combines the results of multiple AIs, such as an AI that detects rust in dark images and an AI that spots minute rust regions to comprehensively determine the presence of rust. It can thus detect even small areas of rust with high accuracy in darkly lit images, such as backlit images and images taken under cloudy weather. In addition, because attributes of roadside equipment and utility pole-mounted equipment are assigned on a pixelwise basis, it is possible to determine the type or part of the facility where rust has formed. This image recognition AI was trained on several tens of thousands of actual images of infrastructure in the field and images of rust that has formed on equipment. It thus can be used by infrastructure managers to inspect equipment without being limited to specific equipment.

NTT will move forward with the practical application of rust detection using its image recognition AI for images taken by MMS and drones.