AI-based Image Analysis Technology for Efficient Maintenance Cycle of Roads for Local Public Entities
- Joint development of 3 companies of the total solution for road maintenance: road surface inspection/diagnosis/treatment using new diagnostic technology -
Nichireki Co., Ltd. (Head office: Chiyoda-ku, Tokyo; President: Manabu Obata hereinafter Nichireki), Nippon Telegraph & Telephone East Corporation (Head office: Shinjuku-ku, Tokyo; President: Fukuzo Inoue, hereinafter NTT EAST) and NTT COMWARE CORPORATION (Head office: Minato-ku, Tokyo; President: Satoshi Kurishima, hereinafter NTT COMWARE) launched a joint development project for "local damage*1)" diagnostic technology by AI as a total solution for road maintenance services including road surface inspection, diagnosis and treatment at a lower cost than existing methods. The service is scheduled to start during fiscal 2019.
Prior to the service, the solution will be showcased in a demonstration project organized by "CHUGOKU" Forum, People's Council for Infrastructure Maintenance to be held in August 2019.
*1) Areas with progressive damage and in need of urgent treatment to be taken
1. Background/IssuesAlthough roads developed intensively during the period of high economic growth are a major concern in the industry as they deteriorate as time passes, the budget for maintenance and repair has been significantly reduced. The roads managed by local governments are enormous in both the number of routes and route extensions, and it has become difficult to respond to the needs of maintenance for road repair such as overlaying*2) all damaged areas. Then, such institutions eventually have to rely on post-measures once an issue is revealed for cases such as potholes*3).
In the past, "cracking rate"*4) had been used exclusively for the evaluation of road cracks. Although this "cracking rate" is a useful evaluation parameter to select areas for repair, it is not sufficiently reliable in identifying areas that require urgent action.
Therefore, an efficient solution is necessary to detect "damaged areas that require urgent treatment" that may further seriously deteriorate and have problems such as potholes with environmentally friendly repair materials in order to secure safety and security at the maximum level within a limited budget.
*2) A method of layering asphalt mixture on existing roads
*3) Holes and dents that can be removed from the surface of roads
*4) Counting the number of cracks in each 50 cm square of paved road surface. When it is 0, the occupied area is 0 m². When it is 1, the occupied area is 0.15 m², and 2 or more for 0.25 m². The denominator of the area to be surveyed, the value obtained as the integrated value of the occupied area calculated according to the number of cracks as a numerator (%) Japan Road Association, Road Survey and Test Method Handbook [1st Volume], Maruzen Publishing Co., Ltd., March 2019,  - Page 215 – Page 224
2. PurposeFor solving the above problems, we proposed the concept of "local damage" based on the "technology for automatically finding the area for repair that really needs urgent action"*5) based on the inspection and diagnosis results. Based on the knowledge gained from crack detection AI, we will develop a new AI specialized in "local damage". Then, the "local damage" will be quantitatively evaluated by inspection and diagnosis, and a preventive maintenance system will be designed to repair areas with potential for damage, treating them with a surface temperature treatment method before damage reaches a serious level. This technology can also be utilized as an effective technology for maintenance of roads without potholes in the coming autonomous driving age.
*5) Patent applied for "Road maintenance/repair planning support device and support method" Patent Applied (Japanese Patent Application No. 2017-51148)
3. About this solution(1) Diagnosis of local damage by AI
In evaluation of "local damage", the number of intersection points (node) of cracks in a 50 cm x 50 cm mesh are noted to rank the level. The higher the number of ranked points (with more nodes) is considered to be a state in which the soundness of the roadbed is lost*6), and this may progress to serious damage such as potholes.
As evaluation of "local damage" is too complicated for manual analysis (see the photo below), AI technologies allows it to be used at a practical level for the first time. Conventional road diagnosis AI was a technology to calculate the "crack rate" from the area of the detected cracks. Based on this, the new AI is a technology that detects crack nodes and ranks damage by using NTT COMWARE's image recognition AI "Deeptector(R)" to optimize evaluation of "local damage". The combination of "local damage" and the conventional "cracking rate" also allows the users to select various measures to solve various issues.
*6) "A Study of Total Solutions for Road Inspection, Diagnosis and Treatment", Japan Society of Civil Engineers (2018)
(2) Measures based on diagnosis results
Based on the results of the diagnosis by AI, we provide consultation services to find the appropriate measures based on the budget of customers, utilizing the know-how on road maintenance materials and methods accumulated by Nichireki. In addition to the repair work based on the conventional crack rate diagnosis results, a feasible road management plan that does not deviate from the current budget level is designed by combining the preventive maintenance of local damage based on the new "local damage" diagnosis results for safe and secure roads. In particular, Nichireki's environment-friendly "normal temperature surface treatment method" is actively used. This method is more cost-effective than conventional repair materials and reduces road management costs.
4. Role of each company
- Provision of inspection/diagnosis services, provision of maintenance repair know-how
- Development of road materials, provision of materials and construction
- Road management plan development (2) NTT EAST
- Provision of network services for image transmission and security
- Provision of image storage and data center for AI analysis
- Provision of AI environment in Smart Innovation Lab*7)
- Sales agency of AI image analysis services (3) NTT COMWARE
- Development of AI image analysis software using Deeptector(R)
- Provision of AI image analysis service
*7) Demonstration environment of AI and IoT technologies provided by NTT East. A new collaboration opportunity to find social problem solutions by industry, government and academia
5. Future scenariosNichireki:
Through inspections and diagnoses, we will adopt not only conventional repair work but also preventative maintenance for "local damage" to provide a feasible road management plan that does not deviate from the current budget, and respond to aging measures for road infrastructure. In addition, we will further proceed with verification to achieve a highly reliable technology aiming for road management without potholes that utilizes "local damage" in the era of automated driving,
As a community-based ICT company, we use the various assets owned by the NTT EAST Group, as well as cutting-edge technologies such as AI and IoT, starting with solving road inspection-related problems and contribute to solving social issues in infrastructure maintenance.
Based on our experience in providing road diagnostic services that utilize AI and ICT technologies, we are advancing efforts to upgrade road infrastructure maintenance, including road surface inspection, diagnosis and integrated solutions using this "local damage" diagnosis. In addition, we will contribute widely to the maintenance of social infrastructure, such as responding to advanced maintenance of communication infrastructure, which is the NTT Group's key enterprise assets, by applying AI and ICT technologies, and using drones for bridge and tower inspection.
6. Contact us[Inquiries about road surface inspection, maintenance, and the total solutions for inspection, diagnosis and treatment]
Pavement Consulting Department: Naka, Hazama
[Inquiries about AI/IoT/Smart Innovation Lab]
Nippon Telegraph and Telephone East Corporation
Business Innovation Headquarters
BBX Marketing Department: Miyata, Okubo, Niimura
[Inquiries about image detection AI solution]
NTT COMWARE CORPORATION
Public Relations Office: Hishida, Moriyama
* "Deeptector(R)" is a registered trademark of NTT COMWARE CORPORATION.
* Company names and product names may be trademarks or registered trademarks of their respective owners.