Machine vision is baked into so many aspects of our lives, even if we don’t realize it.
In our last two issues, we discussed the role machine vision plays in our lives, in areas such as cars and home security cameras. In the final instalment of the series, we’ll talk about machine vision in augmented reality.
Augmented Reality 101
Augmented reality (AR) involves integrating digital information with a user’s environment. It stands in contrast to virtual reality (VR), which involves the creation of a completely virtual environment. AR involves the users being in a real environment, with digital information overlaid on it.
Here’s how it works: AR delivers sensory information such as visual data and sound through a device such as a smartphone, glasses, or headset. The effect is an immersive experience in which the digital information alters the way the user interacts with the physical world. AR can be used to enhance an environment or mask features of it.
Most Common AR Use Cases
Here are some of the most common ways we use AR today:
- Retail: Many stores have online apps that allow you to see what furniture looks like in your space before you buy it.
- Gaming: You can use AR to overlay a game in the real world, like Pokemon Go.
- Entertainment: AR can animate users’ faces on social media.
- Navigation: You can overlay a route on a live view of a road and see information about local businesses in your immediate surroundings.
- Measurement: AR tools can measure 3D points in the user’s environment to determine distance.
A Brief History of AR
The field of AR was built on the foundations of VR. Computer scientist Ivan Sutherland invented the first VR system in 1968; it was a head-mounted display into a virtual world.
Other scientists and researchers made strides with VR and AR technologies, although neither VR nor AR had official names until 1989 and 1990, respectively. Two years after Boeing engineer Thomas Caudell coined the term “augmented reality,” US Air Force researcher Louis Rosenberg created the first properly functioning AR system. It was known as Virtual Fixtures, and it was a complex robotic system that enabled an overlay of sensory information on a workspace.
The beginning of the 21st century saw more developments in AR, including Adobe releasing an AR design tool kit in 2009. A few years later, Google announced it was launching the open beta phase of Google Glass, AR eyeglasses. Two years after that, Microsoft released its AR headset HoloLens.
Machine Vision in AR
AR relies heavily on machine vision. Machine vision has three roles in AR:
- Object detection
- Object tracking
- Simultaneous localization and mapping (SLAM)
Object Detection
Machine vision uses algorithms to identify objects within an image and then locate them. For example, let’s say you’ve got pictures of cats. It’s not enough for the computer to say, “This is an image of a cat.” The computer must also be able to locate where the cat is within the image. Is it towards the lower right corner, or to the left?
The computer creates a box around the cat and specifies the box’s x and y coordinates. This issue becomes more complicated when the number of objects within the image increases. For example, you could have a cat in the picture that’s sitting two feet in front of the couch.
That’s object detection in a nutshell—figuring out what something is within an image and where it is so that the user has a sense of its location in time and space.
Object Tracking
It’s not enough to detect where the object is—you also must be able to track the object from the previous frame. Object tracking involves understanding the object’s location, speed, and motion so you can predict where it will be in the next frame.
Tracking algorithms are important when it comes to the use of machine vision in AR. Let’s say you’re looking at a cat, but the cat moves behind a box. A good tracking algorithm handles occlusion (the technical term for blockage). Object tracking preserves the object’s identity.
SLAM
Let’s say you’re using AR eyeglasses on a walking tour of a new city. The AR eyeglasses would use SLAM to create a map of the environment and track the position of the map creator. This technology allows AR to adapt to a rapidly changing environment.
What’s Next for Machine Vision in AR
AR adoption will only grow in the coming years.
Adoption has grown in part from the use cases for entertainment and for retail. The popularity of games such as Pokemon Go and Target and Ikea’s virtual showrooms appeal to consumers. These apps add fun and convenience to their daily lives.
Companies have taken note of consumer appetites for AR and will continue to improve their offerings. Apple and Google will continue to invest in AR by refining their AR development toolkits. As such, we can expect to see improved image quality and better mapping.
Some of the developments we can expect to see are:
- Lighter, more powerful devices
- Faster network speeds, which will enhance streaming and cloud processing
- Greater industrial and enterprise adoption—organizations see the value of AR and how it can make workers more efficient
Developments within the field of machine vision will also contribute to increased AR adoption. These developments include:
- Advanced real-time processing and edge computing for speedier data processing
- Synthetic data and data augmentation to enhance training model sets
- 3D vision and spatial intelligence, which enable machines to see, understand, and interact with the world in three dimensions
Machine Vision: The Hidden Hero in Our Everyday Routines
AR is one of the many applications in which machine vision is a hidden hero. Machine vision enables in AR enables us to make smarter decisions while shopping, experience immersive entertainment, and simplify everyday tasks such as measurement. In the future, expect AR to become an even more integral part of our lives, powered by machine vision.
