Aug 24, 2026 06:45 PM
https://www.eurekalert.org/news-releases/1140973
EXCERPTS: For people with blindness or who have low vision, moving around independently is a daily challenge. Harvard researchers have developed a new tool that could help such people navigate more quickly and safely, using nothing but a device they likely already have – a smartphone.
Engineers at the John A. Paulson School of Engineering and Applied Sciences (SEAS) have invented a new smartphone app called Mobilio that combines machine learning; algorithms that streamline input from multiple sensors; and personalized audio cues to provide a blind or low-vision person turn-by-turn directions, path guidance, and obstacle avoidance. The research was led by Ph.D. student Raymond Liu in collaboration with Patrick Slade, assistant professor of bioengineering.
The research is published in Nature Biomedical Engineering. [...] “Our system uses the sensors in a smartphone, GPS information, and the actual motion of the person to create what’s essentially a small autonomous vehicle plan on how to navigate them from where they are to where they want to go,” Slade said.
Since most publicly available street-scene datasets are captured from cars, many standard models misclassify pedestrian-relevant infrastructure, explained first author Liu. To address this issue, Liu trained a semantic segmentation model specifically on pedestrian-view images so it could reliably recognize sidewalks, crosswalks, roads, and other key surfaces during walking.
“There’s a need for more data that comes from the perspective of humans walking along sidewalks, or walking outdoors,” Liu added. Liu said a major design challenge was ensuring all the algorithms and models could run in real time on a standard smartphone, which has far less computing power than a laptop or specialized hardware.
Instead of just spoken prompts, Mobilio provides continuous audio cues in the form of beeps as the user walks. The beeps originate from a direction, such as the user’s left or right, to indicate steering, and the guidance is personalized in real time using human-in-the-loop optimization. As a person walks, the system measures how accurately they follow the audio cues and automatically adjusts the cues and pitch patterns to work best for that individual.
[...] Compared to using Google Maps and a cane, participants using Mobilio and a cane completed the outdoor route about 13% faster, and reduced contact with obstacles in the indoor route by about 41%. In terms of reliability, or how consistently the system guided users to the destination without major errors, Mobilio performed comparably to a human guide... (MORE - missing details, no ads)
EXCERPTS: For people with blindness or who have low vision, moving around independently is a daily challenge. Harvard researchers have developed a new tool that could help such people navigate more quickly and safely, using nothing but a device they likely already have – a smartphone.
Engineers at the John A. Paulson School of Engineering and Applied Sciences (SEAS) have invented a new smartphone app called Mobilio that combines machine learning; algorithms that streamline input from multiple sensors; and personalized audio cues to provide a blind or low-vision person turn-by-turn directions, path guidance, and obstacle avoidance. The research was led by Ph.D. student Raymond Liu in collaboration with Patrick Slade, assistant professor of bioengineering.
The research is published in Nature Biomedical Engineering. [...] “Our system uses the sensors in a smartphone, GPS information, and the actual motion of the person to create what’s essentially a small autonomous vehicle plan on how to navigate them from where they are to where they want to go,” Slade said.
Since most publicly available street-scene datasets are captured from cars, many standard models misclassify pedestrian-relevant infrastructure, explained first author Liu. To address this issue, Liu trained a semantic segmentation model specifically on pedestrian-view images so it could reliably recognize sidewalks, crosswalks, roads, and other key surfaces during walking.
“There’s a need for more data that comes from the perspective of humans walking along sidewalks, or walking outdoors,” Liu added. Liu said a major design challenge was ensuring all the algorithms and models could run in real time on a standard smartphone, which has far less computing power than a laptop or specialized hardware.
Instead of just spoken prompts, Mobilio provides continuous audio cues in the form of beeps as the user walks. The beeps originate from a direction, such as the user’s left or right, to indicate steering, and the guidance is personalized in real time using human-in-the-loop optimization. As a person walks, the system measures how accurately they follow the audio cues and automatically adjusts the cues and pitch patterns to work best for that individual.
[...] Compared to using Google Maps and a cane, participants using Mobilio and a cane completed the outdoor route about 13% faster, and reduced contact with obstacles in the indoor route by about 41%. In terms of reliability, or how consistently the system guided users to the destination without major errors, Mobilio performed comparably to a human guide... (MORE - missing details, no ads)