The West Nippon Expressway Engineering Shikoku Company Limited ensures safe road traffic in Japan’s winter: With MVTec HALCON, the company automates the inspection of tires in ongoing traffic. Thanks to deep learning technologies, tires can be classified into summer and winter wheels, for example, which has made it possible to automate the inspection process.
West Nippon Expressway Engineering Shikoku Company Limited (NEXCO) has specialized in the construction and management of expressways and highways in the Shikoku region of Japan. The company is also responsible for the complete maintenance of the road network, including inspections, maintenance and repair work. The aim is to ensure safety, speed, punctuality and reliability in road traffic while complying with the latest environmental standards. This supports the entire infrastructure in the region and ensures maximum mobility for the benefit of the population.
In order to guarantee maximum safety even in the winter months and especially in mountainous regions, the vehicles must be continuously checked for their roadworthiness. This also includes regular inspection of the tires. NEXCO carries this out in ongoing traffic on the side of highways and mountain roads. This involves checking whether the cars have winter tires with sufficient tread depth in case of snowfall or icy roads. Previously, employees took over this task and checked each tire manually, which involved an enormous amount of work.
Reduce false classifications to zero
To reduce the number of employees, shorten waiting times and increase the overall efficiency of the process, tire inspection should be automated using machine vision. “Our goal was to increase the precision in distinguishing between winter and normal tires while reducing incorrect classifications to zero. In addition, machine vision would significantly reduce the number of inspectors required to perform tire checks in inclement weather,” explains Shogo Hayashi, a member of NEXCO’s Engineering Division. “If normal tires were misclassified as winter tires, this would lead to a significant increase in hazardous situations and accidents on snow-covered roads. Only by reliably distinguishing between the two types of tires the highest possible level of safety in winter road traffic can be ensured,” Hayashi adds.
In order to implement these high requirements in automated tire inspection, the experts at NEXCO work with a technical setup that the vehicles to be inspected pass by at a speed of about 30 km/h. The setup is based on a high resolution sensor. The setup consists of high-resolution cameras, LED lighting systems, tablet PCs and monitors for visualizing the test results. All hardware components must be able to withstand the weather conditions directly at the roadside and ensure stable operation even under harsh winter conditions. The MVTec HALCON image processing software integrated into the system ensures precise identification of the respective tire type. HALCON is the comprehensive standard software for industrial image processing developed by the German based MVTec Software.
Rule-based algorithm reached its limits
However, the system design has to meet some challenges: For example, depending on the type of vehicle – whether a car, truck, van or bus – there are many different types of tires that need to be detected and inspected. In addition, the light and visibility conditions in the winter environment are subject to strong fluctuations. Developing a rule-based algorithm that can cope with these demanding conditions proved to be extremely difficult. Another challenge: To improve accuracy in detection, false classifications must be avoided. This means normal tires must not be classified as winter tires. Therefore, technicians at NEXCO defined and added classes to also identify tires that cannot be clearly identified as winter or summer tires due to splash or snow buildup.
In test series based on logistic regression, a high rate of conventional tires was incorrectly identified as winter tires. This result was not satisfactory and showed that rule-based algorithms are not target-oriented under the given conditions. In order to achieve more robust detection results in tire inspection and to reduce false classifications to a minimum, NEXCO relies on the machine vision standard software MVTec HALCON. The deep-learning algorithms integrated in it were trained with about 13,000 tire images, which significantly improved the precision in distinguishing tire types. As a result, the rate of normal tires being falsely identified as winter tires (false positives) could be reduced to zero.
Robust recognition results thanks to Deep Learning
The system based on MVTec HALCON was rolled out in the Fukuchiyama, Oita, and Chiyoda regions in winter 2019 and has produced very robust detection results in practical use. For example, no summer tires were classified as winter tires, which raises road safety to a new level. In addition, tires classified as non-winter tires are subject to manual visual inspection according to the legal requirements in Japan. Thanks to the power of the deep learning technologies integrated in HALCON, the number of vehicles that need to be visually inspected has been reduced to one third. This saves time and costs to a large extent. In addition, the automation of the inspection process allowed long traffic jams and waiting times for motorists to be reduced, which in the end significantly eases the burden on road traffic. Due to the excellent results and numerous benefits, the system was put into operation at a further 18 locations in winter 2020.
Various tools and operators ensure high performance
“MVTec HALCON optimally addresses our requirements for an machine vision solution. The software has a variety of sophisticated machine vision tools and powerful operators. On top of that, it is very easy to use – our users do not need any special prior knowledge,” confirms Shogo Hayashi. Based on the positive experience and the high level of satisfaction, NEXCO is already planning further applications for MVTec HALCON: In the future, applications for evaluating the integrity of road structures will also be developed with the help of machine vision software.
Find out more about MVTec Software.
