Challenges of AR Technology

Update: Headers, Visuals, and Resources

The core concept of Augmented Reality is that this technology is interjecting digital content into the already-existing physical reality. This has a completely different level of challenges than VR, as we all know that the real world is very messy, and more importantly - unpredictable. Digital content is now battling with the physical environment for user’s attention, and there is a higher necessity towards object integrity (an underlying idea that a digital object actually exists in the world) that both competes and merges with the physical environment. The biggest challenge in designing for AR is designing for a 3D space that you as a designer have no knowledge of, and have absolutely no control over. Because of this, your interfaces and systems have to be designed well enough to do that heavy lifting of avoiding those edge cases for you.

Helping Your Users Understand the Augmented World

Obviously one of the biggest challenges of Augmented Reality is the complications of bringing games to the physical world. This is a spectrum of design in which you now have to consider the user's space. But not only a specific space, any potential space that a user could occupy. We have to relinquish control over the confines of our digital content, and this can be challenging.

Considerations for Digital Content

Digital content will always take a secondary role in a user’s physical world. Think of content as a guest or a stranger that they have allowed to inhabit their physical space. If an object is not necessary for environmental contextualization or object interaction then do not put it into your users space. Avoid large or fast-moving objects, and be considerate of the fact that more digital content means more cognitive load.Write a caption

The Construction of the Game

One of the first considerations to tackle is that of the construction of said game. Currently due to the limitations of the technology, there are two main approaches to this. The first is an environmentalist approach through changing the landscape of the room itself by providing windows or vistas into alternate realities. The second is more of an additive object approach where objects are either placed strategically by you as the designer or through the user themselves. Good gaming experiences use a mixture of both approaches, as we want to expand upon physical reality in new and engaging ways through the environment and distinct objects.

https://www.youtube.com/watch?v=xZtSxqWsRB0www.youtube.com

The Best Piece of Advice

Regardless of the approach that one takes in constructing the experience, in order to make believeable AR content there are three easy rules:

Content should acknowledge the world
Content should acknowledge the player
Content should acknowledge other digital content

Approach to Environment

In AR, how you integrate the physical environment with your application is paramount. You must decide what the ideal space scenario is for your game, be it a living room or a coffee shop table. One thing you are always guaranteed in AR is a surface and volumetric space above said surface; however, you need to take into consideration the potential for stationary and moving objects in this environment. Think of varying amounts of real-world objects in that environment as well as your user’s preferences in said space. As a designer, you need to think of your environment as another stakeholder and design against different requirements that it might need as well.

Making hard requirements based on space, (i.e. a user needs a wall and a table to play) is also not advisable, but if you do make hard requirements either allow the user to adjust them or allow flexibility in your system to accept smaller or larger spaces.

Digital Curtain

A great way of stabilitizing content within the environment is creating what I dub the "digital curtain" around environmental content. When creating content in augmented reality, you'll notice very quickly that objects start to look very stark if placed by themselves against physical reality. Because of this harsh juxtaposition, it has a tendency to look off or misplaced. When trying to establish a sense of environment and purpose within the location, you can create content that encircles, fans out, and then fades from the edges of your object. This slow gradiation from physical reality to established digital content makes it more believable as an object.

Obviously since our objects are 3D, a great way of doing this volumetrically is by adding particle affects and breadcrumb trails that eminate from the objects as well. This will break up the dead space around your environment.

Keep in mind, however, if you have a multitude of objects or fast-moving characters you need to heavily avoid the idea of continued "digital space garbage" that can clutter up the environment quickly. It should have a very short lifespan and dewspawn pretty quickly.

Use translucent UI elements

UI elements should be see-through whenever possible. If you use a transparent UI element, the text or icons that you place on them will not be visible enough when the background is too dark or too light. Due to many reasons (most probably additive light constraints), the general standard is lighter text on a darker-colored background.

Respecting (and Avoiding the Limitations of) Real-World Physics

While digital objects don’t have to adhere to the exact same rules real-world objects do, they do have to respect real-world objects in order to maintain immersion and presence. Defining and getting the user to quickly understand how this dynamic between the real world and digital world is key for positive user learning and reinforcement. Integrating real-world object and digital object interactions is the best way to maintain this.

https://c.tenor.com/PvsmswBxi48AAAAd/wile-e-coyote-and-the-road-runner-wile-e-coyote-runs-into-tunnel.gifc.tenor.com

More than anything else, acknowledging the physical and social realities of your user’s environment is important. Your users are more engaged and aware of their physical space and there is a higher rate of interruption. Focus their attention on what you want them to see in front of them, and if you want to draw their attention to another location, do so with visual and auditory affordances.

Understanding the Physical World

The way in which AR devices have the ability to place content in the real world is through detection and estimation of the physical environment. This is often called “meshing” or “mapping” the physical world. This could occur on a person’s face, body, or the room around them. Because the idea of tracking a user's environment is still new to many consumers, it's important to show them exactly the content that the computer is collecting or detecting as a way for them to understand the abilities and limitations of the system.

There are certain hindrances to the accurate detection of surfaces including surfaces without texture, transparent surfaces, reflective surfaces, low light environments, and extremely bright environments. This can be difficult for accurate detection and placement of content in said physical environment.

Be clear to your user about which surfaces have been detected and which have not by showcasing to the user the device’s estimation through applying a grid or semi-transparent pattern on the successfully mapped surfaces. Make sure to also educate users on how to move around the environments, as the accuracy of their room map is important for all types of important world orientation like object placement, occlusion, physics, and navigation.

Spatial Mapping can be guided or unguided, as demonstrated below. It's important to make sure that as your user is collecting data for the system you provide them with the knowledge of how much or how little is necessary for the experience.

There are way too many people to credit here for influence in my entire knowledge base on designing for AR. I want to specifically call out Javier Busto and Jonathan Brodsky for their influence on my understanding of AR Design, without whom I would know far less and my designs would be much uglier. I would also like to give a shoutout to Savannah Niles, Karen Stolzenberg, and Lorena Pazmino for helping me better understand systems-level design while at Magic Leap.

PreviousChallenges of VR Technology NextAR Mobile Design

Last updated 3 years ago