How does Augmented Reality Work?

Augmented reality (AR) is a technology that enriches or augments what you see. It does so by adding digital content (images or information) to real-life objects captured by a camera in real-time. That silly smartphone app that gives your face a handlebar mustache or makes you look like a bunny is an example of an AR-enabled app.

Check out the video below of people at an art exhibit and how AR makes the artwork come to life.

How does Augmented Reality Work and What are the technologies behind It?

Augmented reality overlays digital information on top of a camera-captured natural environment. To work, it needs the following components:

• Depth-sensing camera: You need a camera to record visual information to add to an existing object or place. The said camera should be capable of figuring out the subject’s distance and angle from it.

• Registration tools: These are tools, such as motion sensors and accelerometers, that allow a computer to define the space where it would superimpose or place the sensory information or the objects that should be around the AR user.

• Computer vision: As you use the camera, it takes images from the outside world for interpretation and referencing using a machine learning (ML) algorithm. For example, when you train a camera’s focus on a box, it uses the pixels from that image as a reference to recognize similar-looking objects. The next time you take a photo of another box, the algorithm would try to recall this information to guess if the object is indeed a box. The ML algorithm also combines all pieces of information and adds a creative touch to them to give the user an immersive and believable AR experience.

• Output device: This refers to the display device where users can view the resulting image or video, such as a phone or computer monitor.

What’s the Difference between AR and Virtual Reality?

Many people mistakenly think that AR and virtual reality (VR) are the same. Sure, they’re similar, but they are also quite different from each other. VR creates an entirely new reality using computer technology. It makes you believe you are in a different world by shutting the real world out.

AR, on the other hand, enhances the world you are in. You remain in the same world, except that you see additional objects on display that you can even interact with.

What are the Different Types of AR?

There are different ways to execute AR, which include:

• Marker-based AR: This type needs a visual marker or a distinct image that the camera recognizes and processes to reveal information about an object. The marker is usually a QR code or a custom logo or artwork. Marker-type AR is very popular because it is easy and inexpensive to implement. However, it also requires the download of a custom app that recognizes specific patterns. The video below demonstrates this. The camera recognizes the QR codes on the coasters and this triggers the corresponding image to be displayed.

• Markerless AR: As the term indicates, it requires no visual markers for a camera to augment an image into a physical space. Instead, you move virtual objects. An example of this is an interior design app that lets you put pieces of furniture in various arrangements to see how they would look. Markerless AR is best for apps that do not require an “anchor” to the real world. In the video below, no marker is used. The camera simply tracks the fingertips and displays the image.

• Projector-based AR: This type uses advanced projection technology to simplify complex manual tasks that are part of a company’s manufacturing, assembly, sequencing, and training operations. It can be applied to light guide systems. In a factory, for instance, instead of asking employees to follow step-by-step instructions to create a product using a manual, lights indicating each step are used instead. Check out the projector-based AR demo in the video below.

What are the Uses of Augmented Reality?

While AR has been around since the late 1960s, businesses and industries have yet to exploit its full potential. But that may be changing as more and more AR apps are created and launched. Here are just a few actual and potential uses of AR.

• Public safety and security: Smartphone apps that help people find critical information during emergencies now use AR. These public safety apps guide people on where to go and what to do in particular situations.

• Gaming and entertainment: Pokemon Go was such a massive hit that it essentially became a proof of concept (PoC) of how valuable AR is when applied to mobile games. We expect more AR games to emerge in the mobile space.

• Education: AR is a potential game-changer in education. Just imagine a classroom enhanced by AR. The medical and healthcare fields are also developing training apps that cover a wide range of skillsets, from operating sophisticated medical equipment to performing surgery with AR’s help.

• Travel and tourism: Tourism can use a hand from AR. Imagine walking around town in a foreign country, and when you see an interesting landmark, all you need to do is point your phone camera at it, and you instantly know more about it.

• Equipment maintenance: If AR can help you learn how to perform surgery on a human patient, then it can teach you how to fix a tractor. Or a toaster. Or practically any machine or appliance that needs to be repaired or maintained.