A team of researchers from Shanghai Jiao Tong University and Microsoft Research have developed a new method to reconstruct visual images directly from brain signals. Published in a paper titled “Seeing through the Brain: Image Reconstruction of Visual Perception from Human Brain Signals“, this breakthrough demonstrates the possibility of linking human visual perception with complicated EEG brain signals.
The method, called NEUROIMAGEN, is able to take EEG signals recorded while a person views images and reconstruct approximations of those original images. This is achieved through a pipeline that first extracts multi-level semantic information from the EEG data, including both fine details and high-level features. These are then fed into a pretrained generative model that synthesizes a new image matching the original visual stimulus.
In experiments using a public dataset of EEG responses to ImageNet photos, NEUROIMAGEN achieved significantly higher accuracy and realism compared to previous approaches. On average it correctly matched the category of the original image 86% of the time, versus just 5% for earlier methods. The quality of reconstructed images was also greatly improved, with higher inception scores demonstrating more coherent and realistic outputs.
This research provides an important proof of concept for decoding visual information from portable brain imaging. While image quality remains limited compared to the complexity of human vision, the techniques show promise for brain-computer interfaces and neuroscience applications. Potential use cases include reconstructing blurred or occluded images when normal vision is impaired, and studying how perception arises within cognitive systems.
As EEG technology and neural networks continue advancing, the fidelity of reconstructed visuals is likely to improve. The authors suggest that integrating other modalities like fMRI* could provide additional signals to further enhance image decoding from the brain. By revealing more of the “black box” of vision, projects like NEUROIMAGEN inch us closer to replicating and understanding intelligence.
*) fMRI: functional Magnetic Resonance Imaging is a neuroimaging technique that measures brain activity by detecting changes in blood flow. It works by using MRI scanners to take a series of brain images over time that detect small changes in the magnetic properties of blood oxygenation and flow.