AI tool reconstructs viewed images from brain scans

Michal Irani and colleagues at the Weizmann Institute of Science have developed an AI system that can reconstruct images a person has seen by analyzing high-resolution fMRI scans. The tool can also work in reverse, predicting brain activity from an image, a capability the team hopes will help reveal more about how the brain processes visual information.

The system uses a brain decoder with separate branches for image structure and content, feeding those predictions into a diffusion model to produce more accurate visual reconstructions. It was trained on scans from eight people who each viewed around 9,000 images, then improved with an encoder that generated predicted brain scans from additional images. Around 70% of the training data came from images that were not originally paired with fMRI scans.

The resulting universal brain encoder can work with minimal calibration, requiring one hour of data for a new person rather than about 40 hours in other approaches. Irani is exploring extensions to video, audio, imagined images, and dreams, with potential uses for helping locked-in people communicate and studying conditions such as PTSD.

Neuroscientists and ethicists praised the technical progress while warning about privacy risks, especially if similar methods move from fMRI scanners to EEG devices. Researchers said easier collection of brain activity could make consent and potential misuse more urgent concerns.