Virtual Reality in Neuroscience: History, Applications, and Future

As imaginative as human beings are, it is no mystery how easy of a task it is to think up different worlds and live different lives, so to speak, all inside the comforts of one’s own mind. Literature, cinema, music–all of these noble and artistic pursuits illustrate just how creative humankind has been these past few centuries. The act of creating, however, is just as much science as it is art, and this fact is best demonstrated by one of the latest, most popular technologies elucidated by science-minded creatives of today–virtual reality.

Reality isn’t simply something we objectively experience, but one which we subjectively perceive. Our differing perceptions of reality involve the complex process of recognizing, organizing, and interpreting sensory information (Galotti, 2017). Simply stated, when we perceive, we infuse meaning into the things that we see, hear, touch, feel, and taste. Engineering different versions of the physical world can, therefore, result in different perceptions, and through this principle, our realities can easily be manipulated. Any story can be turned into some form of reality, but the extent of that reality is determined by the limitations that our environments set. Virtual Reality (VR) technology operates on this assumption, and it proves that with the right tools and applications, the human brain can be convinced that it is somewhere it is really not, experiencing something that it really isn’t.

What is Virtual Reality?

Virtual Reality, a term coined in the 80s by Jaron Lanier, founder of VPL Research, is a form of technology that “recreates our relationship with the physical world in a new plane, no more, no less”. VR technology is the most immersive type of reality technology, because it puts the user in a state of total immersion: when sensory experiences feel so real, that one forgets it is a virtual-artificial environment and interact with it as one would in a real situation (Biocca & Levy, 1995). This state is achieved by making use of realistic, three-dimensional images or environments that are presented with interactive hardware and software. Specifically, the key elements of a virtual reality experience include (1) a synchronized sensory stimulation to multiple sensory channels, which usually includes a stereoscopic visual display, auditory stimulation, and tactile feedback; and (2) interactivity (Bohil et al., 2011). It is through these key aspects that users of VR technology can not only observe carefully-crafted virtual worlds but fully become part of them as well. For a more generalized information about Virtual Reality, check out this guide to virtual reality article.

How is Virtual Reality Achieved?

Simply enough, VR technology simulates the senses and integrates sensory inputs to maintain the illusion of realistic experience. This is achieved through a combination of hardware, software, and sensory synchronicity that ensures the user is maintained in a state of total immersion. Currently, the use of HMD proves to be the most popular piece of VR technology in the entertainment and gaming industry.

A head-mounted display, or HMD, is a headset or a pair of goggles that contains a mounted display in front of a user’s eyes, covering the whole field of view. Aside from the visual display, HMDs also include auditory output. Beyond these, the HMD also includes sensors for motion and direction in space, allowing the display to respond accordingly to the user’s movement through complex processing systems. Aside from virtual emulated content, VR systems also make use of input devices that allow the user to interact freely with components of the virtual world. These devices include trackpads, joysticks, and control buttons, and more complex inventions such as data gloves, treadmills, motion platforms, and motion trackers on bodysuits. All of these gadgets need to be synchronized perfectly with the user’s movement in real time to experience virtual reality as technology offers it today.

The study of the human brain is a complex, tricky thing. Apart from the intricacies that continue to shroud the human mind, many other research-related concerns constantly arise.

Testing theory in the laboratory, for instance, presents the issue of low ecological validity–the extent to which experimental results can be generalized to real-world scenarios. Laboratory conditions aren’t always as realistic as researchers can hope them to be, due to a number of reasons. For one, presentations of experimental variables are often limited to simplistic text-, graphic-, or computer-based abstractions of real-world objects and situations (Bohil et al., 2011) because these are the forms over which researchers can practice the most control that technology and resources can allow. Another concern is the ethical constraints of human participation in experiments. For instance, studies on driving must deal with the challenge of simulating the complex activity of operating a car and navigating along urban city roads, as well as the moral implications of allowing participants to literally go out driving in the streets and putting themselves and civilians in danger. Also, beyond the behavioral aspect of neuroscientific research, one must also consider its biological foundations. The widely-applied tests for brain activity such as fMRI and EEG require participants to be stationary, and due to this, the possible activities that the participant can undergo during testing are considerably narrowed. This makes it hard for researchers to explore neuronal foundations of certain actions and situations in particularly naturalistic environments (Doucet et al., 2016).

Because of these hurdles, it has easily become evident how useful a tool VR technology can be in the study of the human brain and behavior. Through virtual environments, a compromise between naturalistic laboratory conditions and a high degree of experimental control is achieved (Bohil, 2011). So far, the advantages that VR technology has brought upon the field of neuroscientific research are abundant, not just among humans but as well across multiple species. One advantage of VR technology in neuroscientific research is that participants can undergo elaborate tests on brain activity while kept sufficiently stationary, as they experience virtual scenarios that elicit very real responses. Also, the high degree of control that researchers have over the creation and manipulation of virtual environments is incomparable to real-world testing environments. VR-based experiments provide a more engaging alternative to passive experimental content such as video or written recordings of sample scenarios. Finally, VR environments circumvent many ethical constraints (Doucet et al., 2016).

How our brains’ realities can be fooled is one thing, but how we can use that fact to gain much-needed knowledge and insight into the anatomy, physiology, and psychology of the human mind is a different thing altogether.

One of the ways through which researchers attempt to unfurl the mysteries of the human mind is through comparative psychology and neuroscience, a field of study which deals with in-depth research on animal behavior. The human brain and the brain of animals such as rodents and monkeys share many similarities, and observing them in different scenarios has contributed greatly to what we know of how the nervous system functions today. However, VR technology now presents different applications of curating human versions of classical animal research of different fields such as spatial navigation and social interaction, as well as the exploration of uniquely human activities such as driving.

A huge domain of animal research pertains to spatial cognition and navigation, which usually involve putting rats or mice in mazes. It’s easy to see how difficult and cost-ineffective it would be to hold human trials in elaborate, life-size mazes. But, with the advent of VR technology and its compatibility with functional imaging techniques (Bohil et al., 2011), virtual mazes have become a possibility. In a study by Bohbot et al., 2007, virtual mazes for spatial cognition helped identify two distinct strategies in navigation, namely, spatial learners and response learners. Weniger et al., 2011 used computer-simulated, first-person environments in looking at the spatial abilities of patients with amnestic MCI (mild cognitive impairments) versus healthy adults in terms of allocentric (location relative to other objects) or egocentric (location relative to the self) learning. To examine allocentric learning, the participants were made to explore a virtual park, and to examine egocentric learning; they were made to solve a virtual maze.  A similar exploration of allocentric and egocentric spatial ability was also conducted among patients with early Alzheimer’s disease (Organti et al., 2013). In this study, the VR-Maze spatial task (VR-MT) and VR-Road Map task (VR-RMT) were used. Other studies of spatial cognition examined other patient populations through VR navigation, such as those with Huntington’s disease (Voermans et al., 2004)  and epilepsy (Frings et al., 2008).

The conventional rodent paradigms of social interaction are in use for ages. VR technology also allows for the observation of brain activity during naturalistic scenarios of face-to-face social interaction between two or more individuals (Bohil et al., 2011). Our virtual body is a representation of the self; a factor in VR that increases presence and immersion; a foundation of a model for general and body-centered interaction; a medium of communication with others in shared environments, and may lead to a theory of virtual reality in understanding the relationship between the physical and the virtual, and proprioception and presence  (Slater and Usoh, 1994). Therefore, having a virtual self can be a great tool for social neuroscience–a field of study that involves looking at which specific brain regions are activated in certain social scenarios, such as in the interpretation of face and eye movements. Hyperscanning, meanwhile, is a VR-based methodology for observing reactions of multiple participants in shared social situations in the virtual world (Montague et al., 2002). This method links magnetic resonance scanners and brain imaging technology in synchrony with the behavioral interaction of participants put in social situations, such as sharing a game of deception. Kozlov et al., 2010 looked at the possibility of exploring real behavior during sophisticated social interactions in video-game-based virtual environments, such as the bystander effect. VR technology has also been used to replicate classic studies in social psychology, such as Milgram’s (1963) controversial study on obedience to authority. The reason why this ground-breaking but disputable study on behavior cannot be replicated is its morally-questionable nature, wherein participants were asked to deliver electric shocks to strangers at the command of an authority figure. Slater et al., 2006 study built upon these findings by examining the extent to which participants would respond to extreme social situations with the knowledge that the whole set-up was only virtual, and none of it were real.

Advancements in social neuroscience have also contributed greatly to both understanding and treating social deficits that occur in autism spectrum disorders, through evidence-based interventions. The Virtual Reality Social Cognition Training intervention (Kandalaft et al., 2012) is a virtual reality program that’s designed to enhance social skills, social cognition, and social functioning. In Kandalaft et al., 2012 study, the program was proven effective among young adults diagnosed with high-functioning autism. In addition, virtual rehabilitation also has applications in the diagnosis and assessment of social avoidance in the context of social anxiety disorders, specifically through eye-tracking systems during public speaking exercises in front of virtual scenes (Grillon et al., 2007).

Galotti, K. M. (2017). Cognitive psychology in and out of the laboratory. Sage Publications.

Biocca, F. & Levy, M. Communication in the Age of Virtual Reality (Lawrence Erlbaum Associates, Hillsdale, 1995).

Bohil, C. J., Alicea, B., & Biocca, F. A. (2011). Virtual reality in neuroscience research and therapy. Nature reviews neuroscience, 12(12), 752.

The Ultimate Guide to VR Technology. Retrieved February 15, 2018 from http://www.realitytechnologies.com/virtual-reality

Doucet, G., Gulli, R. A., & Martinez-Trujillo, J. C. (2016). Cross-species 3D virtual reality toolbox for visual and cognitive experiments. Journal of neuroscience methods, 266, 84-93.

Bohbot, V., Lerch, J., Thorndycraft, B., Iaria, G. & Zijdenbos, A. Gray matter differences correlate with spontaneous strategies in a human virtual navigation task. J. Neurosci. 27, 10078–10083 (2007).

Weniger, G., Ruhleder, M., Lange, C., Wolf, S., & Irle, E. (2011). Egocentric and allocentric memory as assessed by virtual reality in individuals with amnestic mild cognitive impairment. Neuropsychologia, 49(3), 518-527.

Morganti, F., Stefanini, S., & Riva, G. (2013). From allo-to egocentric spatial ability in early Alzheimer’s disease: a study with virtual reality spatial tasks. Cognitive neuroscience, 4(3-4), 171-180.

Voermans, N. et al. Interaction between the human hippocampus and the caudate nucleus during route recognition. Neuron 43, 427–435 (2004).

Frings, L. et al. Lateralization of hippocampal activation differs between left and right temporal lobe epilepsy patients and correlates with postsurgical verbal learning decrement. Epilepsy Res. 78, 161–170 (2008).

Slater, M., & Usoh, M. (1994). Body centred interaction in immersive virtual environments. Artificial life and virtual reality, 1(1994), 125-148.

Montague, P. R., Berns, G. S., Cohen, J. D., McClure, S. M., Pagnoni, G., Dhamala, M., … & Fisher, R. E. (2002). Hyperscanning: simultaneous fMRI during linked social interactions.

Kozlov, M. D., & Johansen, M. K. (2010). Real behavior in virtual environments: Psychology experiments in a simple virtual-reality paradigm using video games. Cyberpsychology, behavior, and social networking, 13(6), 711-714.

Milgram, S. (1963). Behavioral study of obedience. The Journal of abnormal and social psychology, 67(4), 371.

Slater, M., Antley, A., Davison, A., Swapp, D., Guger, C., Barker, C., … & Sanchez-Vives, M. V. (2006). A virtual reprise of the Stanley Milgram obedience experiments. PloS one, 1(1), e39.

Kandalaft, M. R., Didehbani, N., Krawczyk, D. C., Allen, T. T., & Chapman, S. B. (2013). Virtual reality social cognition training for young adults with high-functioning autism. Journal of autism and developmental disorders, 43(1), 34-44.

Grillon, H., Riquier, F., & Thalmann, D. (2007, September). Eye-tracking as diagnosis and assessment tool for social phobia. In Virtual Rehabilitation, 2007 (pp. 138-145). IEEE.