Augmented Reality

The Ultimate Guide to Augmented Reality Technology

Introduction to Augmented Reality (AR)

Augmented reality may not be as exciting as a virtual reality rollercoaster ride, however, it may prove to be a very useful tool in our everyday lives. It holds this potential because it brings elements of the virtual world, into the real world, enhancing the things we see, hear, and feel. Among the other reality technologies, augmented reality lies in the middle of the mixed reality spectrum, being between the real and virtual world.

Augmented Reality Explained

Augmented Reality Definition What is Augmented Reality (AR)?


An enhanced version of reality where live direct or indirect views of physical real-world environments are augmented with superimposed computer-generated images over a user's view of the real-world, thus enhancing one’s current perception of reality.

Augmented Reality Explained Simple Explanation of AR

The origin of the word augmented is augment, which means to add something. In the case of augmented reality (also called AR), graphics, sounds, and touch feedback are added into our natural world. Unlike virtual reality, which requires you to inhabit an entirely virtual environment, augmented reality uses your existing natural environment and simply overlays virtual information on top of it. As both virtual and real worlds harmoniously coexist, users of augmented reality experience a new and improved world where virtual information is used as a tool to provide assistance in everyday activities.

Applications of augmented reality can be as simple as a text-notification or as complicated as an instruction on how to perform a life-threatening surgical procedure. They can highlight certain features, enhance understandings, and provide accessible and timely data. Cell phones apps and business applications are a few of the many applications driving augmented reality application development. The key point is that the information provided is highly topical and relevant to what you want you are doing.

Types of Augmented Reality Augmented Reality (AR) Categories

Several categories of augmented reality technology exist, each with varying differences in their objectives and applicational use cases. Below, we explore the various types of technologies that make up augmented reality:

Marker Based Augmented Reality

Marker Based Augmented Reality
Marker-based augmented reality (also called Image Recognition) uses a camera and some type of visual marker, such as a QR/2D code, to produce a result only when the marker is sensed by a reader. Marker based applications use a camera on the device to distinguish a marker from any other real world object. Distinct, but simple patterns (such as a QR code) are used as the markers, because they can be easily recognized and do not require a lot of processing power to read. The position and orientation is also calculated, in which some type of content and/or information is then overlaied the marker.

Markerless Augmented Reality

Markerless Augmented Reality
As one of the most widely implemented applications of augmented reality, markerless (also called location-based, position-based, or GPS) augmented reality, uses a GPS, digital compass, velocity meter, or accelerometer which is embedded in the device to provide data based on your location. A strong force behind markerless augmented reality technology is the wide availability of smartphones and location detection features they provide. It is most commonly used for mapping directions, finding nearby businesses, and other location-centric mobile applications.

Projection Based Augmented Reality

Projection Based Augmented Reality
Projection based augmented reality works by projecting artificial light onto real world surfaces. Projection based augmented reality applications allow for human interaction by sending light onto a real world surface and then sensing the human interaction (i.e. touch) of that projected light. Detecting the user’s interaction is done by differentiating between an expected (or known) projection and the altered projection (caused by the user's interaction). Another interesting application of projection based augmented reality utilizes laser plasma technology to project a three-dimensional (3D) interactive hologram into mid-air.

Superimposition Based Augmented Reality

Superimposition Based Augmented Reality
Superimposition based augmented reality either partially or fully replaces the original view of an object with a newly augmented view of that same object. In superimposition based augmented reality, object recognition plays a vital role because the application cannot replace the original view with an augmented one if it cannot determine what the object is. A strong consumer-facing example of superimposition based augmented reality could be found in the Ikea augmented reality furniture catalogue. By downloading an app and scanning selected pages in their printed or digital catalogue, users can place virtual ikea furniture in their own home with the help of augmented reality.

How Does Augmented Reality Work? How Does AR work?

In order to understand how augmented reality technology works, one must first understand its objective: to bring computer generated objects into the real world, which only the user can see.

In most augmented reality applications, a user will see both synthetic and natural light. This is done by overlaying projected images on top of a pair of see-through goggles or glasses, which allow the images and interactive virtual objects to layer on top of the user's view of the real world. Augmented Reality devices are often self-contained, meaning that unlike the Oculus Rift or HTC Vive VR headsets, they are completely untethered and do not need a cable or desktop computer to function.

How Do Augmented Reality Devices Work (Inside)

Augmented realities can be displayed on a wide variety of displays, from screens and monitors, to handheld devices or glasses. Google Glass and other head-up displays (HUD) put augmented reality directly onto your face, usually in the form of glasses. Handheld devices employ small displays that fit in users hands, including smartphones and tablets. As reality technologies continue to advance, augmented reality devices will gradually require less hardware and start being applied to things like contact lenses and virtual retinal displays.

Key Components to Augmented Reality Devices

  • Sensors and Cameras

    Sensors are usually on the outside of the augmented reality device, and gather a user's real world interactions and communicate them to be processed and interpreted. Cameras are also located on the outside of the device, and visually scan to collect data about the surrounding area. The devices take this information, which often determines where surrounding physical objects are located, and then formulates a digital model to determine appropriate output. In the case of Microsoft Hololens, specific cameras perform specific duties, such as depth sensing. Depth sensing cameras work in tandem with two "environment understanding cameras" on each side of the device. Another common type of camera is a standard several megapixel camera (similar to the ones used in smartphones) to record pictures, videos, and sometimes information to assist with augmentation.

  • Projection

    While “Projection Based Augmented Reality” is a category in-itself, we are specifically referring to a miniature projector often found in a forward and outward-facing position on wearable augmented reality headsets. The projector can essentially turn any surface into an interactive environment. As mentioned above, the information taken in by the cameras used to examine the surrounding world, is processed and then projected onto a surface in front of the user; which could be a wrist, a wall, or even another person. The use of projection in augmented reality devices means that screen real estate will eventually become a lesser important component. In the future, you may not need an iPad to play an online game of chess because you will be able to play it on the tabletop in front of you.

  • Processing

    Augmented reality devices are basically mini-supercomputers packed into tiny wearable devices. These devices require significant computer processing power and utilize many of the same components that our smartphones do. These components include a CPU, a GPU, flash memory, RAM, Bluetooth/Wifi microchip, global positioning system (GPS) microchip, and more. Advanced augmented reality devices, such as the Microsoft Hololens utilize an accelerometer (to measure the speed in which your head is moving), a gyroscope (to measure the tilt and orientation of your head), and a magnetometer (to function as a compass and figure out which direction your head is pointing) to provide for truly immersive experience.

  • Reflection

    Mirrors are used in augmented reality devices to assist with the way your eye views the virtual image. Some augmented reality devices may have “an array of many small curved mirrors” (as with the Magic Leap augmented reality device) and others may have a simple double-sided mirror with one surface reflecting incoming light to a side-mounted camera and the other surface reflecting light from a side-mounted display to the user's eye. In the Microsoft Hololens, the use of “mirrors” involves see-through holographic lenses (Microsoft refers to them as waveguides) that use an optical projection system to beam holograms into your eyes. A so-called light engine, emits the light towards two separate lenses (one for each eye), which consists of three layers of glass of three different primary colors (blue, green, red). The light hits those layers and then enters the eye at specific angles, intensities and colors, producing a final holistic image on the eye's retina. Regardless of method, all of these reflection paths have the same objective, which is to assist with image alignment to the user's eye.

How Augmented Reality is Controlled

Augmented reality devices are often controlled either by touch a pad or voice commands. The touch pads are often somewhere on the device that is easily reachable. They work by sensing the pressure changes that occur when a user taps or swipes a specific spot. Voice commands work very similar to the way they do on our smartphones. A tiny microphone on the device will pick up your voice and then a microprocessor will interpret the commands. Voice commands, such as those on the Google Glass augmented reality device, are preprogrammed from a list of commands that you can use. On the Google Glass, nearly all of them start with "OK, Glass," which alerts your glasses that a command is soon to follow. For example, "OK, Glass, take a picture" will send a command to the microprocessor to snap a photo of whatever you're looking at.

Augmented Reality Use Case Example: Healthcare

A strong example of augmented reality in use is in the field of healthcare. From a routine checkup, to a complex surgical procedure, augmented reality can provide immense benefits and efficiencies to both patient and healthcare professional.

Physical Exams

Imagine that you walk into your scheduled doctor (or dentist) appointment, only to find your doctor (or dentist) wearing an augmented reality headset (e.g. Google Glass). Although it may look strange, this technology allows him (or her) to access past records, pictures, and other historical data in real-time to discuss with you. Instantly accessing this digital information without have to log into a computer or check a records room, proves to be a major benefit to healthcare professionals. Augmented reality healthcare example and surgery use case Integration of augmented reality assisted systems with patient record management technologies is already highly desirable utility. Data integrity and accessibility is a major benefit to this type of system, where record access becomes instantaneous and consistent across all professionals to the most current records, instructions, and policies.

Surgical Procedures

Let’s take this example one step further and imagine that we are going in for a surgical procedure. Before the anesthesia takes effect, we notice that the doctor is wearing an augmented reality headset. The doctor will use this throughout the procedure for things such as display of surgical checklists and display of patient vital signs in a dashboard fashion. Augmented reality assisted surgical technologies assist professionals by providing things such as interfaces to operating room medical devices, graphical overlay-based guidance, recording & archiving of procedures, live feeds to remote users, and instant access to patient records. They can also allow for computer generated images to be projected onto any part of the body for treatment or can be combined with scanned real time images. The benefits of using augmented reality include a reduced risk of delays in surgery due to lack of familiarity with new or old conditions, reduced risk of errors in performing surgical procedures, and reduced risk for contamination if the device allows surgeons to access information without having to remove gloves (i.e. hands-free) to check instruments and data.