Japanese Scientist Develops ‘Mind-Captioning’ Technology to Translate Mental Images into Text
In a groundbreaking advancement at the intersection of neuroscience and artificial intelligence, a Japanese researcher has unveiled a technique that can convert a person’s mental images into detailed descriptive sentences. This novel method, termed “mind-captioning,” leverages brain scanning technology combined with AI to transform complex visual thoughts into accurate textual descriptions.
The study, published on November 5, 2025, in the journal Science Advances, was led by Tomoyasu Horikawa, a scientist at NTT’s Communication Science Laboratories near Tokyo. Horikawa and his team focused on decoding brain activity to translate mental images, a task that has remained challenging compared to previous efforts to convert thought-based words into text.
How Mind-Captioning Works
The technique starts with functional magnetic resonance imaging (fMRI), a non-invasive brain scanning method that captures brain activity patterns while participants view visual stimuli. In this study, six native Japanese speakers—four men and two women aged 22 to 37—watched 2,180 silent video clips depicting a wide range of objects, scenes, and actions.
Using large language models, the captions describing these videos were converted into sequences of numerical data. Horikawa then trained simpler AI decoders to correlate the measured brain activity with these numerical representations. When participants later watched or recalled previously unseen videos, the decoders interpreted their brain signals, enabling a separate AI algorithm to generate coherent, descriptive text that matched the mental visual content.
The resulting AI-generated captions effectively described objects, places, actions, events, and their interrelations as represented in the brain. Remarkably, the AI produced these texts in English, despite the participants being non-native English speakers.
Implications and Potential Applications
Experts hail this technology as a significant step toward true brain- or mind-reading capabilities. Marcello Ienca, a professor of AI and neuroscience ethics at the Technical University of Munich, called it “one additional step forward” in that direction. He and others warn, however, of potential ethical concerns related to mental privacy and consent.
Beyond technological innovation, the approach holds promise for transformative health applications. Because it does not rely on areas of the brain responsible for language, the method could assist individuals with speech impairments or language-related neurological damage. Conditions such as aphasia—a disorder impairing language expression—and amyotrophic lateral sclerosis (ALS), which progressively affects speech, could particularly benefit. Psychologist Scott Barry Kaufman, who was not part of the study, suggested this could also help non-verbal autistic individuals communicate more effectively.
Ethical Challenges and Privacy Considerations
The ability to decode inner visual thoughts raises profound ethical questions. As Horikawa’s study notes, such technology might one day decode unspoken thoughts, dreams, or the mental perceptions of infants and animals. Ienca emphasized that if neural implants or similar technologies become broadly accessible, “very, very strict rules” will be essential to safeguard sensitive neural data, including early indicators of dementia and psychiatric conditions.
Privacy advocates underscore the urgency of protecting mental freedom. Łukasz Szoszkiewicz, a social scientist and neurorights expert, stresses treating neural data as highly sensitive by default and advocates for strict consent protocols, user-controlled access, and regulation tailored to AI-powered brain decoding technologies.
Limitations and Current Status
Horikawa cautions that the current approach requires extensive data collection from active participants and is limited to more predictable visual stimuli, such as everyday scenes presented in study videos. For instance, unusual events like “a man biting a dog” were not included, making it unclear if the system can reliably interpret less common mental images.
While concerns over privacy risks exist, Horikawa maintains that the present technology cannot easily access or decode a person’s private thoughts without their cooperation. It remains primarily a tool for neuroscience research rather than practical mind-reading applications.
The Future of Mind-Decoding
This study paves the way for further research into brain-based communication aids and cognitive understanding. As AI and brain imaging technologies progress, mind-captioning could evolve into powerful assistive tools for people facing communication challenges, while simultaneously prompting society to consider the ethical frameworks necessary to protect individual mental privacy.
For those interested in the latest scientific breakthroughs, CNN’s Wonder Theory newsletter offers ongoing updates about fascinating discoveries shaping our understanding of the universe and ourselves.
