Microsoft researchers are exploring how digital holography can be used to build novel near-eye displays for virtual and mixed (or augmented) reality.

The goal of this research is to put on devices that look and feel like ordinary eyeglasses but can bring us to new places. In addition, the full fidelity of human vision while experiencing VR/MR should be maintained.

Of course, we are still in the early days on the journey toward this vision, and there isn't a clear route to solving all the optical challenges. For example, today's near-eye displays feature a trade-off between bulkiness and field of view. They provide only part of the information the brain uses to judge depth. It's not clear how to make a display that looks like eyeglasses when many must wear actual eyeglasses to see clearly. Typically, when the researchers try to solve one of these challenges, they take a step back in another area.

In a new publication to be presented at SIGGRAPH 2017, Microsoft researchers propose a new solution to VR/MR displays that has the potential to address all these problems at once: holography. When light waves from a 3D scene meet on a flat surface, they interact to form a sophisticated pattern, known as a hologram - much like a series of waves combining to form a detailed set of ripples in a pool. The hologram can then be "played back" with a beam of laser light to re-create all the intricacies of the 3D scene, including the depths of all the objects.

One of the greatest challenges to building near-eye displays is balancing the trade-off between the field of view and device form factor. A wide field of view is desirable so that we can see the whole scene at once; at the same time, we desire a lightweight, compact and aesthetically pleasing design that is both comfortable to wear all day and socially acceptable. Today we find these qualities only in isolation: sunglasses-like displays with very narrow fields of view (<20 degrees horizontal), or goggle- or helmet-sized displays with wide fields of view (>= 80 degrees horizontal). In their work, the researchers created a simple prototype display with a sunglasses-like form factor, but a wide field of view (80 degrees horizontal), shown below. The key insight from the prototype is that the researchers can create an optical design with visual defects, that is very compact, and then correct the defects in the hologram calculation.

Another key challenge in building near-eye displays is reproducing the focal depth cues. When viewing real objects, our eyes can focus on different depths, and the blurriness of out-of-focus objects is used to judge depth. However, most near-eye displays present all objects at the same focus. This limitation causes visual discomfort, reduces the realism of the scene and can cause focus conflicts between real and virtual objects. Many solutions have been proposed to address the problem - varifocal displays, multifocal displays and light-field displays - however, these solutions provide only a coarse approximation of the focal cues of the real scene. The researchers claim that holographic displays can provide highly accurate focal control that is individually addressable on a pixel by pixel basis, like a real scene.

Another important challenge to solve in near-eye displays is handling problems with the user's vision. A display the size of eyeglasses should have the functionality of eyeglasses into it. A current solution is to provide adjustable image focus control by mechanically moving the distance between the display panel and the lens. However, this solution does not scale to providing a display the size of eyeglasses, and does not correct for more advanced vision problems such as astigmatism. Microsoft's researchers demonstrate that holographic displays can correct for advanced vision problems entirely in software by pre-distorting the emitted light waves so that they appear correct when viewed by the eye.