Review: Movies and Narratives as Naturalistic Stimuli in Neuroimaging
Key findings
- This review discusses the use of movies and narratives as stimuli during neuroimaging research into memory, attention, language, emotions and social cognition
- In the realm of memory, naturalistic stimuli enabled the identification of an entirely new short-term memory mechanism, temporal receptive windows, and have proved to be powerful tools for research into event segmentation
- Movies and narratives are helping researchers elucidate the neural basis of selective attention
- By eliciting genuine and robust emotions, movies and narratives have helped uncover the neural basis of basic and social emotions and have even facilitated the creation of machine-learning algorithms that classify emotions using functional MRI data
- Movies and stories make the study subject a passive viewer/listener, so the next step may be to expand naturalistic stimulation to virtual reality and computer-game/simulated environments
Subscribe to the latest updates from Radiology Advances in Motion
Naturalistic stimuli, such as movies and stories, are growing in popularity for human neuroimaging experiments. They are more dynamic than stimuli such as photographs, and they are more engaging and true-to-life than traditional tasks such as presenting subjects with lists of items to be memorized.
In Neuroimage, Jyrki Ahveninen, PhD, an investigator in the Martinos Center for Biomedical Imaging at Massachusetts General Hospital, Martinos Center alumnus Iiro Jääskeläinen, PhD, and colleagues, recently reviewed the use of movies and stories as stimuli during neuroimaging research into memory, attention, language, emotions and social cognition. This summary focuses on research opportunities that naturalistic stimuli have made newly possible or easier.
Memory
Temporal receptive windows—Naturalistic stimuli helped researchers discover an entirely new short-term memory mechanism: a hierarchy of temporal receptive windows (TRWs), the length of time before a response during which sensory information may affect that response. For example, understanding the sentence "Jill kicked the ball" requires its subject "Jill" to be kept in memory. The longer the narrative, the longer the TRW needs to be for grasping that "Jill" is the same person mentioned earlier.
The longest TRWs have been observed in default mode network structures. These seem to support the processing of the narrative's evolving plot. Intriguingly, TRWs emerge spontaneously in deep neural networks during machine learning, a form of artificial intelligence.
Event segmentation, perhaps one of the most fundamental short-term memory functions, refers to the need to process points of discontinuity. For example, a discussion with a colleague is encoded as an event separate from a subsequent staff meeting. Movies and stories are naturally composed of changes in elements such as time, location and protagonists, so they are powerful tools to uncover what takes place in a subject's brain when one event ends and another begins.
Attention
Naturalistic stimuli have helped elucidate the neural basis of selective attention. For example, a study published in Cerebral Cortex showed that changes in visual salience (the perceptual quality that makes some items stand out from others) and unexpected turning points in a movie activated the cerebellum, something not documented in neuroimaging studies using traditional tasks.
Another example is that commercial movies are very powerful in building suspense, which makes it possible to study the engagement of attention under more genuine conditions than those in traditional experiment paradigms.
A fun fact cited in the review comes from a study of people who watched movie trailers while their brain activity was monitored on electroencephalography, as reported in Frontiers in Neuroinformatics. Intersubject correlation of EEG activity predicted box-office performance more than 20 times better than self-reporting-based methods used in the movie industry. This approach has obvious applications in marketing and perhaps also in education, the sciences and other fields.
Emotions
By eliciting genuine and robust emotions, naturalistic stimuli have helped uncover the neural basis of basic emotions (e.g., anger, fear, sadness and happiness) and social emotions (emotions that depend on, or are influenced by, other people's actual or imagined emotions, such as shame and pride). Movies can even induce anger, which is difficult to do in a laboratory.
In particular, naturalistic stimuli have allowed the creation of machine-learning algorithms that classify basic and social emotions on the basis of functional MRI data. Researchers can now determine with fair accuracy which emotion the subject was experiencing during neuroimaging. The ability to classify emotions opens new possibilities for investigating brain activity during emotional experiences and how people regulate their emotions.
"Catching," or being influenced by other people's emotions, plays a central role in the phenomenon of herding, in which an individual's decisions are influenced by the behavior of a group. Methods that permit studying underlying neural mechanisms create new possibilities for research into these experiences as well.
Toward the Future
As a stimulus becomes more naturalistic, researchers lose some control. For example, brain responses to a movie clip could be attributed to emotions when in fact the brain responded to the fact that the scene was a close-up shot.
In addition, movies and stories differ from real life in many ways, notably in making the subject a passive viewer/listener instead of an active agent with goals to pursue while making decisions. Expanding naturalistic stimulation from movie clips and stories to virtual reality and computer-game/simulated environments would circumvent that limitation and might be the next step in this area of research.
view original journal article Subscription may be required
Learn about the Martinos Center for Biomedical Imaging
Refer a patient to the Department of Radiology