“Attention is known to modify the neural representations of perceived images,” says study lead author Ryohei Fukuma. “However, we did not know that even imagining a different image could change these representations.”
To test this, the researchers developed a new technique by working with epilepsy patients who already had electrodes placed in their brains to record and display electrocorticogram readouts of the images they were visualizing. . Patients were shown an image of the real-time readout and instructed to mentally paint a different image representing a “landscape,” “human face,” or “word” (for example, different types of Looking at the thinking images of human faces) to control the readout.
“The results clarified the relationship between brain activity when people see images when they imagine them,” explains senior author Takufumi Yanagisawa. “The electrocorticogram readouts of the imagined images were different from those stimulated by the real images seen by the patients. They could be modified to be even more specific when the patients received real-time feedback.”
The time required to produce a very clear distinction between the imagined image and the observed image was different to that of imagining a “word” and a “scenario”, which have different parts of the brain involved in visualizing these two concepts. could have something to do with it. ,
“Our findings suggest that a readout image controlled by the subject’s imagery can be inferred by the observer using this technique,” says Fukuma.
Similar approaches could be used to develop a communication tool for severely paralyzed patients, such as those with amyotrophic lateral sclerosis, with the accuracy with which this new technology displays images in the subject’s brain. Similar tools already used by some patients with this condition rely on motor control, which deteriorates more quickly than visual cortical activity, so an imagery-based tool can be highly valuable.
Electrocorticogram (ECOG) recordings were taken from 17 epileptic patients who had implanted subdural cortical electrodes related to visual perception. A decoder was trained to estimate the semantic meaning of the images the patient was viewing from intracranial ECOG recordings using the visual-semantic space. Based on the real-time estimated semantic information with the decoder, an image was displayed on a monitor placed in front of the patient. The patient then visualized and attempted to display an image with a directed meaning.
During the experiment, subjects were instructed to imagine an image of a target meaning (human face, landscape, or word). Subjects attempted to display the target meaning image by visually visualizing it on a monitor.
A total of 32 images shown from left to right for one of each of the two images were displayed for 250 ms during each attempt to display the image of the target sense. The images underlined in red are those that correctly match the target meaning.
The article, “Voluntary control of semantic neural representations by imagery with paradoxical visual stimuli,” was published in communication biology Doi at: https://doi.org/10.1038/s42003-022-03137-x
