How Virtual Reality Headsets Work (Inside)
Inside of each virtual reality head-mounted display (HMD) is a series of sensors, individual eye displays, lenses, and display screen(s), among other various components. The Ifixit Oculus Rift teardown offers an excellent step-by-step teardown and look inside of one of the most popular virtual reality headsets. Below we explore some of the key components inside of a virtual reality headset.
Image Source: MIT Technology Review
Key Components Inside of a Virtual Reality Headset
The three most common sensors in a virtual reality headset are magnetometers, accelerometers and gyroscopes. These sensors work together by measuring the user’s motions and direction in space. Their ultimate goal is to achieve true six-degrees-of-freedom (6DoF), which covers all the degrees of motion for an object in space.
- Magnetometer – The magnetometer tells your device which direction it is facing on the surface of the earth. The magnetometer acts as a sort of compass for the device. As such, it is able to do this by measuring magnetic fields.
- Accelerometers – The accelerometer tells your device which way up it is. To do this, your device will have several accelerometers to work together measuring things like gravitational pull in relation the accelerometer measuring the device’s orientation.
- Gyroscopes – A gyroscope calculates the orientation of your device. It does this to either help your device maintain a particular orientation or make sure it properly changes orientation when it should.
Lenses lie between your eyes and pixels on the display screen(s). They focus and reshape the picture for each eye by angling two 2D images to mimic how each of our eyes take in views of the world (also called stereoscopic). This creates an impression of depth and solidity, which we perceive to be a three-dimensional image. Lenses on each virtual reality device are not one-size-fits all and have to be adjusted for initial use as all devices have different lens properties.
Display screens show the images that user view through the lenses. They are typically LCD and receive video feed from the computer or smartphone. Depending on the headset, the video feed is either sent to one display or two displays (one per eye). This happens via wireless connection, smartphone connection, or HDMI. The most common types of virtual reality display technology is a Liquid Crystal Display (LCD) screen, similar to the kinds used in smartphones and computer monitors. An alternative display technology is an Organic Light-Emitting Diode (OLED) screen.
Virtual reality systems demand a substantial amount of power, even in comparison to notoriously power hungry gaming systems. The processing power required by virtual reality systems can be broken down into several categories:
- Input Processor – Controls the devices used to input information to the computer. They retrieve and distribute data to the rest of the system with minimal lag time. Examples include keyboards, mouses, 3D position trackers, and voice recognition systems.
- Simulation Processor – Takes the user inputs along with any other tasks that are programmed from the natural world and determines the actions that will take place in the virtual world. This is a core component of the VR system.
- Rendering Processor – Creates the sensations that are output to the user. These include visual, auditory, haptic and other sensory systems. Separate rendering processes are used for each sensory system.
Other Concepts to Understand How Virtual Reality Headsets Work
Field of View
Field of view (also called Field of Vision or FOV) is an important component used in virtual reality to provide users with a realistic perception of their environmental landscape. Simply put, field of view refers to how wide the picture is. Field of view is measured based on the degree of display (e.g. 360°). Most high-end headsets make do with 100° or 110° field of view which is sufficient for most virtual reality content.
Frame rate refers to the frequency (rate) at which the display screen shows consecutive images, which are also called frames. Television shows run at 30 frames per second (fps) and some game consoles run at 60 frames per second (fps). In virtual reality, a minimum frame rate of approximately 60 frames per second is needed to avoid content stuttering or cause of simulation sickness. The Oculus Rift runs at 90 fps, providing Oculus Rift users with a very lifelike experience. Future Frame rates for virtual reality headsets are set to inevitably continue getting faster, providing for a more realistic experience.
Latency refers to the amount of time it takes for an image displayed in a user’s headset to catch up to their changing head position. Latency can also the thought of as a delay, and is measured in milliseconds (ms). In order for an experience to feel real, latency usually needs to be in the range of 20 milliseconds (ms) or less. Low latency, or very little delay, is needed to make the human brain accept the virtual environment as real. The lower the latency, the better. The higher the latency, a noticeable and unnatural lag may set in, consequently causing simulation sickness for the user.
Virtual reality audio may not be as technically-complex as the visual components, however, it is an equally important component to stimulate a user’s senses and achieve immersion. Most virtual reality headsets provide users with the option to use their own headphones in conjunction with a headset. Other headsets may include their own integrated headphones. Virtual reality audio works via positional, multi-speaker audio (often called Positional Audio) that gives the illusion of a 3-dimensional world. Positional audio is a way of seeing with your ears and is used in virtual reality because it can provide cues to gain a user’s attention, or give them information that may not be presented visually. This technology is already quite common and often found in home theater surround sound systems.
Tracking handles the vital task of understanding a user’s movements and then acting upon them accordingly to maintain full immersion in virtual reality. Below, we explore the three of the main types of virtual reality tracking: