Fig. 12.1
Illustration of the reality-virtuality continuum
12.2 History of Virtual Reality
12.2.1 Early Milestones
While the concept of VR dates back to early science fiction writers, its history is rooted in the idea of an “experience theater,” described by Morton Heilig around 1950 (Burdea and Coiffet 2003). The focus of Heilig’s idea was a cinematic experience for users involving all the senses rather than just the usual 2D display with sound. Twelve years later, in 1962, Heilig introduced the Sensorama Simulator (US Patent # 3,050,870): an arcade-style device for a single user that featured displays of 3D video feedback (obtained by a pair of side-by-side 35 mm cameras), stereo sound, a moving chair, wind effects via small fans near the user’s head, and even odor producers. The Sensorama is considered the earliest archetype of immersive, multisensory technologies.
Heilig may also be the first to propose head-worn displays with his concept of a simulation mask. He was granted a patent for his concept in 1960 (US Patent # 2,955,156), which featured 3D analog displays encompassing the user’s periphery, optical controls, stereophonic sound, and smells. In 1961, Philco Corporation introduced their version of a headset device tethered to a closed-circuit television system that could be used by the wearer to transmit findings while navigating dangerous environments. However, it was Ivan Sutherland who is credited with producing the first example of a fully immersive head-mounted display (HMD; sometimes called the head-mounted audio-visual display). Released in 1966, and called the Sword of Damocles, Sutherland’s HMD used two cathode ray tubes to produce a stereoscopic display with a 40° field of view. The device was suspended from a ceiling-mounted cantilever—being too heavy to be supported by the wearer—which also tracked the wearer’s viewing direction via potentiometers. Sutherland later incorporated computer-generated scenes to take the place of analog images with his groundbreaking development of a scene generator that produced primitive 3D wireframe graphics. Introduced in 1973, Sutherland’s scene generator was capable of displaying 200–400 polygons per scene (frame) at a rate of 20 frames per second. These scene generators are the precursors to modern graphics accelerators—a key component of VR computer hardware.
Other important elements of immersive experiences followed shortly after the emergence of HMDs. In 1971, the first example of haptic feedback was demonstrated by Frederick Brooks Jr. and his colleagues. This development, as well as others, was incorporated into several iterations of military flight simulations in the 1970s and 1980s which was classified at the time. Other government agencies were also pursuing their own interests in simulators. In 1981, the National Aeronautics and Space Agency (NASA) created an HMD that used liquid crystal displays with optical controls to focus the images they produced close to the eyes. The initial NASA device was called the Virtual Visual Environment Display, or VIVED. Their successor system, called the VIEW for Virtual Interface Environment Workstation, was introduced in the late 1980s and boasted upgraded computer hardware as well as an interactive glove for manipulating wireframe objects that were spatially and mechanically tracked.
By the late 1980s and early 1990s, commercial VR systems began to emerge. The DataGlove, the same glove used by NASA’s VIEW system, was introduced in 1987 by VPL Inc. and was the first break from the standard keyboard and mouse computer interface tools. VPL Inc. was also the first company to release an immersive VR solution consisting of an HMD (called, interestingly, the EyePhone) that featured two LCD displays to produce stereoscopic images, each with a resolution of just 360 × 240 pixels. The HMD was used together with their previously released DataGlove, and their system was called the RB2 system (Reality Built for 2). It retailed for over $11,000.00, and the HMD weighed over 5 lbs. Nintendo later released an answer to the DataGlove in 1993, called the Power Glove.
While hand-worn and head-mounted devices were under development, other companies focused on improving VR hardware and software platforms. In 1991, Division Ltd. in the UK produced a scalable and integrated VR workstation to support their line of VR products. On the software side, the US company Sense8 in 1992 developed a library of VR-specific programming functions, called the WorldToolKit. This was followed by the Virtual Reality Toolkit (VRT3) software framework by Dimensions International in the UK.
12.2.2 Alternative Technological Approaches
While head-worn displays are currently considered the de facto standard for fully immersive VR and are the most practical technological solution for consumers, previous limitations associated with HMDs (e.g., weight) motivated the exploration of other VR system concepts. One popular example is the cave automatic virtual environment (CAVE) or its variations. A CAVE is a small room enclosed by whole-wall displays of virtual images produced by a series of video projectors. A stereoscopic 3D effect can be achieved through the use of positionally tracked active shutter glasses worn by the occupants and synced with the projectors. In active shuttering, the projected image alternates between the views for the left and right eye, while a shutter blocks the eye for which the view does not apply, producing a 3D perspective. CAVEs are commonly used in engineering, manufacturing, and construction industries to prototype designs.
12.2.3 Historical Applications in Medicine
The earliest applications of VR in medicine were centered around visualizing medical images and performing surgical planning (Chinnock 1994). Since then, medical applications of VR have expanded into the realm of medical education and training, facilitated communication (between clinicians or between clinicians and patients), and in a variety of therapies, including the treatment of phobias, PTSD, anxiety disorders, rehabilitation, and pain management. Interest in medical applications of VR has also been steadily accumulating. A recent search by Pensieri and Pennacchini (2014), for VR-related articles in the medical literature, uncovered nearly 12,000 publications as of 2012 using the most common search terms representative of VR applications in healthcare (but excluding “virtual environment,” “augmented reality,” etc.). Rather than focusing on the traditional applications of VR in medicine, the rest of this chapter will focus on the current landscape of VR technologies and how these technologies may be used to enhance the domain of 3D printing and the domain of 3D visualization in general.
12.2.4 A Technology Outpaced by Vision
Despite the pace of early development, as well as considerable amounts of media attention, VR companies in the 1990s failed to secure a widespread consumer base. Early systems were prohibitively expensive, with the fastest available graphics workstation by Silicon Graphics Inc. costing over $100,000, and were plagued with performance and reliability issues. As such, the VR industry remained small and largely contained to corporations, government institutions, and universities despite several attempts by the video game industry to generate interest in VR systems. Eventually, the rise of the internet claimed the public’s attention and, subsequently, interest in VR technologies waned when the few remaining companies failed to deliver on media hype (Stone 2006).
12.3 Modern Commercial Virtual Reality Technologies
12.3.1 Renewed Interest in VR
A new era of affordable virtual reality technology has recently emerged—driven primarily by the video game industry and enabled by breakthroughs in smartphone display technology, graphic processing units (GPUs), and tracking technology. VR recaptured significant public attention in 2012 largely due to the successful crowd-funding campaign for the Oculus Rift (Oculus VR, Menlo Park, CA) (Largent 2011; Kickstarter 2012). The campaign presented a prototype of a rotationally-tracked HMD using IMUs and smartphone displays. Following two developer kits and acquisition of Oculus by Facebook (Largent 2011), the Oculus Rift consumer version was released in March of 2016—consisting of a high-resolution, low latency head-mounted display. Six degrees of freedom positional tracking of the HMD is facilitated by a proprietary tracking system called Constellation which uses IMUs and optical cameras that track infrared (IR), patterned LED markers. Tracked handheld controllers were later released for the Rift in December of 2016.
While Oculus received the bulk of public attention throughout its development of the Rift, the emergence of modern VR technology resulted from the work of a number of players. One notable example is Valve Corporation (Bellevue, WA) who are credited with the development or discovery of a number of key components that facilitate immersive VR (e.g., the necessity of low-persistence displays) (James 2015). Following an early collaborative relationship with Oculus, Valve partnered with HTC Corporation (New Taipei City, China) to produce the HTC Vive—released 1 month after the Oculus Rift. The Vive was released with tracked controllers and uses a full room-scale, 360° tracking system called SteamVR® Tracking. SteamVR Tracking uses IMUs in conjunction with two “base stations” that regularly sweep the room with IR lasers (which are detected by photodiodes on the tracked objects) and boasts high-frequency submillimeter tracking accuracy within a 5 m corner-to-corner volume (SteamVR® Tracking 2017).
Together, the Oculus Rift and HTC Vive represent the first widely available, modern, PC-based, consumer VR platforms. However, the new landscape of VR devices is rapidly evolving with other offerings such as Razer OSVR, FOVE, MindMaze MindLeap, and Vrvana Totem which all present interesting technological variations (Largent 2011). With many choices available, and certainly more to come, early adopters of modern VR will likely be concerned with compatibility both now and in the future. To this end, Valve has made their SteamVR® software platform open to all hardware manufacturers through the OpenVR software development kit and application programming interface and have even gone so far as to freely license the use of SteamVR® Tracking so that any hardware manufacturer can make use of their tracking system (SteamVR® Tracking 2017; Lee 2017). The future of VR technology compatibility will also likely be greatly facilitated by the development of OpenXR: a cross-platform open standard for virtual reality and augmented reality applications and devices created in collaboration with a group of companies under the direction of the Khronos Group (Khronos Group 2017).
12.3.2 Mobile VR
Beyond advances in PC-based or “tethered” virtual reality technology, modern developments have also introduced a new domain of mobile VR driven primarily by smartphones. These devices take the form of custom lenses mounted in cases of various designs that hold compatible smartphones. Software is run on the smartphones themselves, and tracking—accomplished by relying on the phone’s internal IMUs or mounted IMUs—is generally limited to rotational (three degrees of freedom) only. Current examples of mobile VR at the time of writing are the Samsung Gear VR (Samsung, Seoul, South Korea), Google Cardboard (Google, Mountain View, CA) (simply a handheld cardboard shell with lenses), and Google Daydream (Wiederhold 2016).
Considering that the computational ability of smartphones is significantly less than that of high-end PCs and that mobile VR is generally limited to rotational-only tracking, the experiences available with mobile VR have been comparatively limited in capability to date. Despite this, mobile VR has already been used in medical roles such as anatomical education (Moro et al. 2017), ophthalmic image display (Zheng et al. 2015), surgical training (Gallagher et al. 2016), and patient education (Forani 2017).
Related posts:
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
