Published as chapter 32 of the book VR Gems (William Sherman, editor), VR_Concepts_Illustrated_Using_OSVR describes the technology required to develop effective immersive environment systems.

The immersive nature of virtual and augmented reality systems engages the human visual system in ways that require wider field of view and lower latency than other 3D computer graphics systems to provide artifact-free rendering that avoids inducing fatigue and discomfort in viewers. The need to construct consistent transformations between multiple objects in the system (head, hands, and/or screens) requires a common space. The need to precisely match the viewing direction requires off-axis projection matrices that are carefully matched to the relative positions of the viewer’s eyes and screens. System lens designs often induce chromatic aberration and nonlinear distortions of the screen images that depend on the location of the viewer’s eyes with respect to the lenses and the location of the lenses with respect to the screens. The temporal sampling apertures of tracking systems and the finite times required to render and scan out the images, together with operating-system-
induced delays, introduce latency between the viewer’s head pose and the images being displayed at a given moment in time.

This chapter introduces each of these issues, presenting both theoretical descriptions and example implementations in the Open-Source VR system (OSVR.org).