Controllers & Other Peripherals

For specifically how to design with controllers and other peripherals, please also look in the Interaction Design section. For how to align input with interfaces, follow up with the Interface Elements and Behaviors as well as the Button States and Object Manipulation sections.

Hardware Accessories

I'm definitely not going to go into all the different types of hardware accessories, because practically anything that's bluetooth compatible can be considered a controller. Now, in the times before the "tracked / 6DoF" controller we know and love today, there were aplenty of "3DoF" controllers and there still are to this day a couple of physical keyboards being integrated into VR/AR pipelines.

The Degrees of Freedom Debate

The tracked motion controllers that often exist today as a part of VR hardware are those with 6 degrees of freedom (6DoF) with both rotational and positional tracking. 6DoF allows us to reach forward, behind our backs, move our hands across our body or close to our face.

Click for fun aside

One of my absolute favorite parts about when you read old (not even that old, just a few of years!) documents on 6dof controllers you get a sense of just HOW BELOVED they were. As of July 2022, Oculus' developer portal (in comparison to 3Dof) still says, "Touch controllers give you access to hands in VR, not just implements that you can hold, but actual hands. When done properly, virtual hands let you interact with the virtual world intuitively; after all, you already know how to use your hands."

...I don't necessarily agree. But they also have an entire other section about hand tracking, more on that later.

Controllers with 3 degrees of freedom (3DoF) are limited to rotational tracking. These controllers were integral to hardware like that of Google Daydream and Samsung Gear VR. 3DoF controllers have no positional tracking, but worked well as a pointing device for when 6DoF wasn't wildly available. Any rotation input is better than none! (Or is it?)

Non-tracked controllers have no positional or rotational tracking. One would think that by process of elimination, non-tracked controllers would be the worst option for VR/AR content. However, I believe for the time being it is the current best solution for efficiently typing a decent WPM (words per minute) in headsets.

Link: https://developer.magicleap.com/en-us/learn/guides/design-essentials

One of my favorite parts about controllers as an input is that they're a solid peice of hardware. This may seem like a step backwards (as most of VR is trying to outgrow games), but hardware provides an anchor point of knowledge for the everyday user. An advantage of motion controllers over hand tracking or head pose is that the controllers have a precise position in space, allowing for fine grained interaction with digital objects. This type of fine "microinteraction" control can't be seen yet in other inputs because of tracking issues, hardware limitations, and a lack of user familiarity on embodied interactions.