Decoding the Brain's Pathways for Visual Recall

After nearly ten years of intensive research, a group from the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) has made significant progress in understanding the brain dynamics that determine why some images linger in our memory while others quickly fade. This groundbreaking study combined the techniques of magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI) to map the exact moments and locations in the brain that process memorable images. Their findings were recently published in the open-access journal PLOS Biology.

The research utilized 78 pairs of images that represented the same concept but varied greatly in their memorability. Each pair included one image that was easy to recall and another that was not. These images covered a wide range of subjects, from action scenes of skateboarding to diverse animal habitats, mundane objects like cups and chairs, natural settings like forests and beaches, bustling urban environments, and various facial expressions. Fifteen participants viewed these images during the study.

Benjamin Lahner, an MIT PhD student in electrical engineering and computer science and a CSAIL affiliate, and the study's first author, highlighted the findings: “People tend to remember some images better than others, even when they are conceptually similar, like different scenes of a person skateboarding. We've identified a brain signature of visual memorability that emerges around 300 milliseconds after seeing an image, involving areas across the ventral occipital cortex and temporal cortex, which processes information like color perception and object recognition. This signature indicates that highly memorable images prompt stronger and more sustained brain responses, especially in regions like the early visual cortex, which we previously underestimated in memory processing.”

The study reveals that while memorable images evoke a prolonged and intense response in the brain, the reaction to less memorable images diminishes swiftly. This new understanding could significantly alter our comprehension of memory formation and persistence. The research team is optimistic about the potential clinical applications of these findings, particularly for the early diagnosis and treatment of memory-related disorders.

The innovative MEG/fMRI fusion method developed in CSAIL Senior Research Scientist Aude Oliva’s lab played a crucial role in this research. It enhanced the precision in capturing the brain’s spatial and temporal dynamics, traditionally limited by the individual constraints of each method. A machine-learning algorithm also aided in analyzing the brain's activity when viewing different images by creating a "representational matrix" - a detailed chart mapping the similarities in neural responses across different brain regions.

Lahner further explained the methodology behind selecting the image pairs based on their memorability scores, emphasizing the importance of a balanced representation across various visual categories in their selection process.

While the research has provided valuable insights, the team acknowledges certain limitations, particularly in identifying the exact roles of different brain regions in memory processing.

Aude Oliva, who is also the director of strategic industry engagement at the MIT Schwarzman College of Computing, MIT director of the MIT-IBM Watson AI Lab, and a CSAIL principal investigator, sees great promise in this line of research: “Understanding the neural underpinnings of memorability opens up exciting avenues for clinical advancements, particularly in diagnosing and treating memory-related disorders early on. The specific brain signatures we've identified for memorability could lead to early biomarkers for Alzheimer's disease and other dementias. This research paves the way for novel intervention strategies that are finely tuned to the individual's neural profile, potentially transforming the therapeutic landscape for memory impairments and significantly improving patient outcomes.”

Wilma Bainbridge, an assistant professor of psychology at the University of Chicago who was not involved in the study, commented on the significance of these findings: “These findings are exciting because they give us insight into what is happening in the brain between seeing something and saving it into memory. The researchers here are picking up on a cortical signal that reflects what's important to remember, and what can be forgotten early on.”

The research team, which also includes Western University Assistant Professor Yalda Mohsenzadeh and York University researcher Caitlin Mullin, received support from a shared instrument grant from the National Institutes of Health and was funded by a variety of prestigious awards including the Vannevar Bush Faculty Fellowship, a National Science Foundation award, and several others. Their paper is available in PLOS Biology.