Well this just might be the coolest thing you’ll see all month.
UC Berkeley scientists have found a way to use brain activity to recreate moving images, i.e. movies. Basically, they were able to record signals in people’s brains, and then they used that data to reconstruct visual images.
The researchers used functional Magnetic Resonance Imaging to see how the subject’s brains processed visual stimuli, and applied that knowledge to the recreation of images using existing YouTube clips.
Here’s how they did it:
[The subjects] watched two separate sets of Hollywood movie trailers, while fMRI was used to measure blood flow through the visual cortex, the part of the brain that processes visual information. On the computer, the brain was divided into small, three-dimensional cubes known as volumetric pixels, or “voxels.”
“We built a model for each voxel that describes how shape and motion information in the movie is mapped into brain activity,” lead author of the study Shinji Nishimoto said.
The brain activity recorded while subjects viewed the first set of clips was fed into a computer program that learned, second by second, to associate visual patterns in the movie with the corresponding brain activity.
Brain activity evoked by the second set of clips was used to test the movie reconstruction algorithm. This was done by feeding 18 million seconds of random YouTube videos into the computer program so that it could predict the brain activity that each film clip would most likely evoke in each subject.
Finally, the 100 clips that the computer program decided were most similar to the clip that the subject had probably seen were merged to produce a blurry yet continuous reconstruction of the original movie.
Here’s the amazing video showing what those recreations looked like as compared with the original film clip –
As of now, the science can only reconstruct images from movie clips that people have already viewed. It can’t simply pull images right out of the brain and create a video of what someone is thinking.
But imagine the possibilities – dream mapping, reconstructing memories, and helping people with neurodegenerative diseases or people in a coma who can’t communicate verbally. The prospect of being able to record and then see a visual manifestation of your mind’s eye is terrifyingly awesome.
The researchers say that they are decades for the scientific breakthroughs to do this, however. It will be quite awhile before you’ll be able to read someone’s thoughts by monitoring their visual cortex.
But damn, isn’t this fascinating?
