Scientists from University College London have trained artificial intelligence to reconstruct videos based on the brain activity of mice. By utilizing data from individual cells in the visual cortex, the researchers were able to recreate what the animals saw with high precision. This approach opens up new possibilities for studying the mechanisms of visual perception.
Artificial intelligence reconstructed video from mouse brain signals
Mind-reading technologies are advancing rapidly beyond fundamental science. In similar studies on humans, scientists use algorithms to decode not only videos but also internal speech. This enables paralyzed patients to communicate with the outside world. In the future, combining brain activity data with neural networks could help create ideal visual prosthetics for the blind, transmitting video from cameras directly to the cortex.
In recent years, researchers have been actively exploring how the brain deciphers signals from the visual organs. Previously, such experiments were primarily conducted on humans using functional magnetic resonance imaging, which allowed for the decoding of visual information.
However, the new work by British specialists has taken research to a qualitatively new level by capturing the activity of individual cells. Instead of relying on general changes in brain blood flow, as with fMRI, the scientists employed a microscopic imaging method. They tracked calcium levels in the cells of the mouse visual cortex, which enabled precise identification of when each specific neuron was activated.
Frames from the videos shown to the mice (top row) compared with frames from the reconstructed videos (bottom row). University College London
To analyze the data, a dynamic neural network model was adapted. The algorithm considered not only the video footage displayed to the rodents but also their movements and changes in pupil diameter. By comparing actual neuron activity with predicted activity, the AI started from a completely blank screen and gradually, pixel by pixel, adjusted the image. As a result, the system learned to generate high-quality video based solely on brain signals. The study was published in the journal eLife.
The successful training of the model allowed the experiment’s authors to recreate ten-second clips that the mice were seeing for the first time. The accuracy of the reconstruction directly depended on the number of neurons included in the analysis, confirming the importance of collecting the most comprehensive biological data sets possible.
A comparison of the original frames and the resulting output showed minimal time delays and a high degree of correspondence. The researchers now plan to use this new technology to explore fundamental differences between objective reality and how it is perceived by a living organism.
The developers aim to understand why the internal representation of visual information sometimes becomes distorted and which specific factors influence these changes. Lead author Dr. Joel Bauer noted: “There is no perfect representation of the world inside our heads. The process of visual information processing distorts our perception, altering the data. This discrepancy between reality and the brain’s representation is not necessarily an error; it is more of a feature reflecting how our mind interprets and supplements sensory information. We want to understand how this happens.”
About the Author
