North American Editor
Machine vision technology is now part of our everyday lives. We might not realize it, but it plays a role in our daily routines.
To show the wide-ranging reach of machine vision, we’re publishing a three-part series about how this technology is used on a day-to-day basis. The first article in this series is about machine vision in vehicles. We’ll explore the history of machine vision in this space, how it’s being used today, and discuss future developments.
Machine Vision in the Automotive Sector: A Crucial Part of Advanced Driver Assistance Systems
Machine vision in the automotive sector plays a vital role in advanced driver assistance systems (ADAS). ADAS are features that make driving a vehicle safer. They include adaptive cruise control, anti-lock brakes, high beam safety systems, forward collision warnings, lane departure alerts, and traffic signals recognition. Many accidents on the road are due to human error. ADAS aims to reduce those errors to the greatest extent possible. Car and Driver’s online magazine noted that ADAS functionality can be split into two broad categories: features that automate driving, such as automatic emergency braking systems, and features that improve drivers’ awareness, such as lane departure warning systems.
The History of Machine Vision Systems in ADAS
The concept of ADAS is not new. Neither is machine vision. However, it took some time for machine vision to mature to the point where it could be integrated into ADAS. One of the earliest uses of machine vision in ADAS is lane departure warning systems. Mitsubishi designed and launched a basic camera system that tracked road markings in 1992. When a driver crossed over those markings, an alert would sound. It took a few years for other manufacturers to begin offering lane departure warning systems on a broader scale. More advanced lane departure warning systems became available on commercial trucks in Europe in 2000. In 2001, some passenger vehicles in Japan came equipped with that technology. Three years later, Infiniti began offering it on its North American models.
Within a few years, another application of machine vision in ADAS became common: rear-view cameras. The popularity of larger cars meant drivers had larger blind spots. Sadly, those blind spots could be fatal for pedestrians, especially small children. After two-year-old Cameron Gulbransen of Oyster Bay, NY was run over as a car was backing up, his parents fought tirelessly for enhanced safety legislation. The Cameron Gulbransen Kids Transportation Safety Act of 2007 directed the U.S. Department of Transportation to issue new safety standards within specific time periods leading to the installation of safety technologies as standard equipment in all vehicles to prevent deaths and injuries in and around motor vehicles. One of the results of that act was the 2014 announcement by the United States Department of Transportation’s National Highway Traffic Safety Administration (NHTSA) that all new vehicles under 10,000 pounds (4,500 kg) are required to have rear-view cameras by 2018.
Machine Vision in ADAS Today
Machine vision technology is vital to improving drivers’ awareness, one of the two key functions of ADAS.
Here’s how machine vision works under the umbrella of ADAS:
- A camera detects other vehicles, pedestrians, road signs, lane markings, etc.
- Supporting software analyzes the images quickly to determine if there’s a problem.
- If there’s an issue, the software will trigger a vehicle response, such as automatic braking, a warning that the vehicle is departing its lane, or an alert that a collision is imminent.
All of this happens very quickly—within moments. However, that rapid response is enabled by a complex system of cameras, sensors, and software. There are two types of cameras:
- Monocular cameras: these cameras use one “eye” to magnify images.
- Stereo cameras: these cameras simultaneously photograph objects from several directions using a camera with two “eyes.”
Monocular cameras identify blind spots, identify lanes, and recognize crosswalks, among other things. Stereo cameras can measure depth. They’re used for detecting objects at a distance. The cameras work with image sensors that convert the pictures to electronic signals. Those image sensors require complex algorithms to analyse electronic signals so ADAS can decide on the best step to take.
Images go through a process before ADAS can decide on the best action:
- Pre-processing: the image sensors determine the position of the vehicle on the road, the location of the sky relative to the road, and important features such as road markers.
- An algorithm matches objects in the image to similar images in its library.
- The system also detects “noise”—random variations in colors or brightness and corrects it.
- Moreover, the system identifies “edges,” sudden changes or breaks in continuity and figures out what the image is supposed to look like.
The Future of Machine Vision in ADAS
ADAS will evolve from being a set of systems to keep drivers safe to a set of systems that enable self-driving vehicles. Machine vision will remain essential in ADAS. For cars to be completely autonomous, they must be able to reach the fifth level of automation. In the fifth level, the car can control itself without any need for human interference. That means that the car can sense obstacles, including pedestrians and other cars, as well as “read” road signs. The fifth level of automation will rely heavily on machine vision systems to safely and effectively navigate roads.
Machine Vision Systems: An Integral Part of Our Lives
Over the years, machine vision technology has matured and improved to enable us to drive safely. They work in the background so we can get to where we need to be without expensive, potentially fatal accidents. These systems will also be a linchpin in tomorrow’s self-driving cars, allowing vehicles to function autonomously.
The automotive industry offers one example of how machine vision can become an essential part of our everyday lives. In the next issue, we’ll look at how machine vision is used in the home security industry to protect people from threats.
