Augmented reality may be a bit more promising than virtual reality for industrial engineering applications given its fundamental difference—it allows you to layer digital information right on top of the physical “data,” or reality. It’s important to keep in mind that the nascent augmented reality market has not proven itself to be a reliable commodity for engineers. For media and entertainment, it's impossible not to notice the success and popularity of Pokémon Go, the augmented reality game from Nintendo. Engineering applications are in short supply but they do exist.
In this post, we'll cover a cross-section of augmented reality headsets and focus on the ones that have the most promise for engineering applications, such as training, maintenance, visualization and collaboration.
Differentiating Augmented Reality Products
Augmented reality can be experienced on mobile devices like a tablet and smartphone. There are also augmented reality headsets known as head-mounted displays (HMDs), eyeglasses, visors, helmets and even a pair of augmented reality contact lenses.
Truly Immersive Augmented Reality Requires a Big Headset
One of the most interesting issues with making immersive augmented reality is the amount of physical real estate it requires from the user. There is a direct ratio that requires the amount of optics to increase as the desired display size and field of view increases. With a compact wearable like Google Glass, for example, the widest field of view (FoV) you'll achieve is around 20 to 30 degrees. Google Glass is 13 degrees and something like the Epson MoverioBT-2000 gets up to 23 degrees.
This is basically why headsets yield a more immersive experience.
Augmented Reality Terminology
Many of the terms such as FoV, latency, frame rate and refresh rate are similar to those you need to familiarize yourself with in order to understand virtual reality, which you can see here, in a previous post I wrote called “Understanding Virtual Reality Headsets.”
Virtual retinal display (VRD), which is specific to AR, beams a raster projection directly onto a user's irises. The effect is similar to seeing a display directly in front of your eyes, much like a computer or television screen. The effectiveness of VRDs has greatly increased with the advancement of LED technology, allowing users to see them even during daylight hours.
Overview of Augmented Reality Applications
Augmented reality has been used in a variety of novel ways, across many different fields and disciplines, including archaeology, construction, medicine, emergency management, industrial design and the military.
The first three headsets featured here have the most potential uses for engineers. Afterwards, I'll briefly explore a cross-section of augmented reality headsets and glasses with industrial and enterprise applications and potential.
1) DAQRI: The Smart and Safe Helmet
The DAQRI Smart Helmet (DSH) is a combination safety helmet and augmented reality headset that overlays virtual instructions, safety information, training and visual mapping over specific reality data. Workers in oil and gas, automation and manufacturing sectors who need to understand or follow complex instructions to perform complex processes can look through the DSH and see digital information overlaid on a variety of different contexts—whether it is a Siemens controller, scanning device or quality control equipment for metrology purposes.
The helmet comes with its own battery and docking station and weighs as much as any typical industrial hardhat. The DSH varies widely in price, fetching anywhere from $5,000 to $15,000, because the features need to be custom built. Autodesk, GE and Hyperloop are some of the corporations testing and using the DSH in situ.
Powered by the sixth-generation Intel Core m7 processor and RealSense scanning technology, the DSH is the first functional and practical HMD that uses augmented reality to help human workers perform difficult tasks.
The DSH's face shield and injection-molded plastic helmet component are ANSI-compliant. The inner part of the helmet's shell is a combination of cast aluminum and carbon fiber composite.
DAQRI's multiple cameras work together to make this the first fully industrial augmented reality headset. If features a13-megapixel HD camera to capture videos and photos, track objects and recognize 2D targets and colors. Intel's RealSense technology has two infrared cameras built in, and DAQRI integrates them with an infrared laser projector that can sense depth by measuring deflected infrared light. A low-resolution camera is integrated with an industrial-grade inertial measurement unit (IMU), which allows the helmet to compute its relative position in space in real time via a combination of gyroscopes and accelerometers. Another high-quality IMU is available for additional applications. For sound, there are four microphones, volume and power buttons and an output jack for headphones.
Workers wearing the DSH can see augmented instructions that change relative to their actual environment. The worker will be able to look at a machine with 100 readouts, and the DSH will draw their attention to a pressure gauge, for example, that is reading too high or too low. The DSH's infrared cameras can constantly monitor equipment by overlaying normal thermal data and current thermal data to make distinctions and judgements on the fly. Workers equipped with the DSH can visually scan for out-of-tolerance thermal anomalies that may put an operation in danger.
The DSH was used in a case study with Hyperloop in a way that illustrates the collaborative power when used between workers of a large and widely dispersed manufacturing unit. A novice operator was using a robotic welder for precise spot welding in construction. A more experienced operator could tune into the networked DSH of their less-experienced counterpart, assess what they were doing and relay correct instructions immediately.
This means that a company could buy a custom-built series of DSHs, scale up operations with less-experienced (less-expensive) workers and have a few experts remotely monitor and guide them all the way through to production.
According to DAQRI, the DSH is available for purchase as of Q1 2016 to its top-tier customers.
The Meta 2 is an augmented reality headset from Metavision with a few features that are promising for potential industrial uses, such as a wide FoV. Compared with virtual reality, less FoV is not desirable but not prone to the same distraction as a small FoV in an augmented reality headset. In virtual reality, whatever isn’t in the FoV (which contains 3D models of various polygonal sizes) is surrounded by pitch-black darkness. In augmented reality, a low FoV equals a tiny translucent digital window with 3D content surrounded by the real world of physical data that one would see without a headset.
The FoV on the original Meta was 25 to 35 degrees, which is small compared to the average virtual reality FoV. The Meta 2 has a 90-degree FoV, which is a tremendous breakthrough, especially when considering industrial applications like training, maintenance or manufacturing. There is a trade off that allows this wide FoV. Like its predecessor, the Meta 2 is tethered. Connection to a workstation limits all kinds of training applications and limits use on a factory floor for assembly or maintenance. If you compare the Meta 2 to an augmented reality headset like the Microsoft HoloLens, which is untethered, you understand immediately that the Meta 2 is at a disadvantage for practical uses. But this has to be interpreted as a long-term design strategy on the part of both Metavision and Microsoft. Microsoft believes it can advance its untethered AR headset through developers planning for a wireless hardware device, and Metavision is planning to develop the technology and untether before a consumer version is in demand. It is important to keep in mind that both the HoloLens and Meta 2 are basically developer kits and not full-fledged consumer products.
Meta takes full advantage of the continuing miniaturization and democratization of inexpensive sensors paired with a high-definition camera to compute your hands in the context of the digital and physical environment they exist in through the headset. The hand-tracking controls of the Meta 2 are not as sophisticated as the Leap Motion Orion controllers, but the notion of separate hardware for hand-tracking may be going the way of the dodo in favor of eye-tracking technology, though this is debatable. Preorders of the Meta 2 developer kit are available now for $949, and Metavision says the devices will ship in Q3 2016.
It’s understood right now that the potential killer engineering or industrial app for augmented reality headsets like the powerful Meta 2 are still to come.
Microsoft HoloLens is an augmented reality headset that was developed under the code name ProjectBarrio. It is also known as a “mixed reality” headset, or holographic computer. “Mixed reality” is a term that is gaining momentum in the media and is sometimes used to describe headsets that can switch from virtual reality mode to augmented reality mode. Magic Leap, the mysterious startup with no products but major investments led by Google and Alibaba, has pushed for this linguistic distinction in particular.
Besides semantics, the HoloLens descends from the motion detection and scanning technology hardware known as the Microsoft Kinect, which was released in 2010. Microsoft uses the term hologram to describe the digital information that is overlaid on the physical world (which you can see through the visor). The hope is that headset holographic computing will eventually replace the screens (laptop, PC, mobile devices) we use around the clock today.
The HoloLens features an accelerometer, magnetometer, gyroscope, four depth-sensing cameras, a light sensor, four microphones and a 2-megapixel camera. Besides the typical CPU and GPU found in the majority of computing devices, the HoloLens also has something called a Holographic Processing Unit, or HPU. The HPU is just a sort of “grand central terminal” for all of the input from the various sensors.
Microsoft is also building the Windows 95 of augmented reality operating systems, called Windows Holographic, enabling manufacturers to focus on developing the hardware and not worry about the software, which, in theory, will help the development of augmented reality devices reach a tipping point with consumers and help augmented reality go mainstream.
4) A cross section comparison of alternate augmented reality headsets.There are dozens of augmented reality headsets available today, and this random cross-section is meant to highlight a few similarities and differences.
Google Glass: First we have Google Glass, which was discontinued after barely a year of being available on the market. Google Glass 2.0 is currently in development, and Google is now showcasing enterprise and industrial applications for the headset. It has a heads-up display, a microphone, a CPU, a battery, a GPS, speakers, a microphone and a projector that overlays digital information onto a user’s view by beaming it through a visual prism that focuses the digital information right onto the retina.
Google is focusing on enterprise use cases, like Boeing using them for wire harness assembly. The headsets use voice commands and a side panel a la GeordiLa Forge from Star Trek, but they won't help you with your vision, unfortunately.
R-7 Smart Glasses: The form factor of these glasses from OsterhoutDesign Group separates them from the pack of giant and boxy augmented reality headsets like Microsoft HoloLens and Meta 2. They just kind of look like awkward, oversized sunglasses.
They run a custom version of Android KitKat called ReticleOS, which means you can run Android apps and load movies.
The R-7s are very light as well, weighing about 2.5 lbs, which is about a pound less than the HoloLens.
Vuzix M300 Smart Glasses: This headset seems like a carbon copy of Google Glass, except it has slightly better resolution and also runs IOS. The 64 GB of internal storage isn't all that interesting, but it's partnership with Ubimax and use in the logistics industry is worth mentioning. DHL uses xPick on the earlier version of these smart glasses (Vuzix M100 Smart Glasses).
Ubimax produces the Enterprise Wearable Computing Suite, which is a group of industrial augmented reality applications similar to xPick, including xMake for manufacturing, xAssist for remote assistance and xInspect for maintenance.
Moverio Pro BT-2000: Epson's first version of this augmented reality headset, the BT-100, was released before Google Glass. This latest version is definitely targeting enterprise customers specifically for remote viewing with its 5-megapixel camera, 3D mapping and gesture recognition capabilities.
5) Magic Leap: News of this unicorn startup comes wrapped in mysterious claims of “light-field displays” and “photonic chips” that are threatening to upend everything we know about consumer-oriented headset kits like the Meta 2. This startup is called Magic Leap, and it's raised about $1.5 billion in funding. The funding was led by Google (which many speculate was a response to Facebook's $2 billion purchase of Oculus) on the strength of supposedly ground-breaking technology in which a special light apparatus beams holographic images right onto your eyes.
Magic Leap comes last in this overview because it represents the promise, potential and global interest in the future of augmented reality as a new computing platform. This company has not released a product as of yet but promises to revolutionize the field of “mixed reality” or augmented reality, or whatever you prefer to call it.
The potential uses for engineers are there, particularly in the DSH, but a standardized platform that is the equivalent of the iPhone for augmented reality still remains elusive.