Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Sep;15(3):20.
doi: 10.1145/3522757. Epub 2022 Aug 19.

Beyond the Cane: Describing Urban Scenes to Blind People for Mobility Tasks

Karst M P Hoogsteen  1 , Sarit Szpiro  2 , Gabriel Kreiman  3 , Eli Peli  1
Affiliations

Beyond the Cane: Describing Urban Scenes to Blind People for Mobility Tasks

Karst M P Hoogsteen et al. ACM Trans Access Comput. 2022 Sep.

Abstract

Blind people face difficulties with independent mobility, impacting employment prospects, social inclusion, and quality of life. Given the advancements in computer vision, with more efficient and effective automated information extraction from visual scenes, it is important to determine what information is worth conveying to blind travelers, especially since people have a limited capacity to receive and process sensory information. We aimed to investigate which objects in a street scene are useful to describe and how those objects should be described. Thirteen cane-using participants, five of whom were early blind, took part in two urban walking experiments. In the first experiment, participants were asked to voice their information needs in the form of questions to the experimenter. In the second experiment, participants were asked to score scene descriptions and navigation instructions, provided by the experimenter, in terms of their usefulness. The descriptions included a variety of objects with various annotations per object. Additionally, we asked participants to rank order the objects and the different descriptions per object in terms of priority and explain why the provided information is or is not useful to them. The results reveal differences between early and late blind participants. Late blind participants requested information more frequently and prioritized information about objects' locations. Our results illustrate how different factors, such as the level of detail, relative position, and what type of information is provided when describing an object, affected the usefulness of scene descriptions. Participants explained how they (indirectly) used information, but they were frequently unable to explain their ratings. The results distinguish between various types of travel information, underscore the importance of featuring these types at multiple levels of abstraction, and highlight gaps in current understanding of travel information needs. Elucidating the information needs of blind travelers is critical for the development of more useful assistive technologies.

Keywords: Blindness; assistive technologies; impaired vision; independence; mobility; navigation; outdoor; scene description.

PubMed Disclaimer

Figures

Figure 1:
Figure 1:
One of the 8 stops, with three example sentences given per object (Experiment 2). The objects described here are highlighted in the image using a color corresponding to the colors of the text-balloons.
Figure 2:
Figure 2:
Illustration of the various types of relative position of objects (hexagon) from the participant (circle). These include: difference of relative distance with a similar angular direction (a), difference of relative angle with a similar distance (b), difference of both relative distance and angular directions (c).
Figure 3:
Figure 3:
Procedure used in experiment 2. The procedure was executed as follows: for each object, descriptions were read to the participant, the participant was asked to score each description based on usefulness and rank order based on priority. After all sentences were rated and rank ordered the participant was asked if in general, it is useful to describe a given object, and to explain why the information is or is not useful to them. When this process was completed for all 5 objects within the stop, participants were asked to rank order the subjects in terms of priority.
Figure 4:
Figure 4:
Mean self-reported travel frequency in times per week (a) and travel confidence (b), for independent travel in unfamiliar areas. The sample is split into early blind participants (N = 5) and late blind participants (N = 8). Early blind participants reported travelling more frequently and more confidently than late blind participants. The asterisk (*) denotes a significant difference in travel frequency and travel confidence between the two groups, and the error bars provide the standard errors.
Figure 5:
Figure 5:
Content of 302 questions categorized based on subject (a) and aspect (b).
Figure 6:
Figure 6:
Content of 302 questions categorized according to the type and abstraction level of travel information.
Figure 7:
Figure 7:
Mean usefulness score per object for the detect descriptions ordered from high to low usefulness. The error bars are the standard errors. Due to a high number of significant differences a pairwise comparison table with significant indications is provided in Appendix A.
Figure 8:
Figure 8:
Mean usefulness score for each group of object instances at a closer and farther position in relation to the participant. The figure shows that the description of objects which are: closer but at the same relative angle (distance only), at the same distance but closer to the prospective path of the participant (angle only), both closer and closer to the prospective path of the participant (combined), are evaluated as more useful, with the asterisk (*) indicating a significant difference between the two positions in the distance only and combined groups. The error bars are the standard errors.
Figure 9:
Figure 9:
Mean priority rank per subject across participants ordered from high to low priority. All 20 subjects are included here. These scores are based on a total of 415 rank orders across all participants. The error bars are the standard errors.
Figure 10:
Figure 10:
Mean usefulness scores for the two levels of description precision: precise and imprecise for the location aspect and object route. Compared to the imprecise descriptions, the precise descriptions were rated as more useful for the location aspect. The asterisk (*) indicates a significant difference. The error bars are the standard errors.
Figure 11:
Figure 11:
Mean usefulness score per aspect as provided by the early blind and late blind participants. The asterisk (*) indicates a significant difference between the two groups for the identity aspect. The error bars are the standard errors.
Figure 12:
Figure 12:
Mean usefulness score per description aspects aggregated across participants and the objects: building entrance, bus stop, car, curb, grass, park, pedestrian, pole and street. The asterisk (*) indicates a significant difference between aspects and the error bars are the standard errors.
Figure 13:
Figure 13:
Mean priority rank per aspect as provided by the early blind and late blind participants. A higher rank is an indication of higher priority and the asterisk (*) indicates a significant difference between the two groups for the aspects: location and identity. Detect sentences were not ranked against the other aspects, so they are not included in this graph. The error bars are the standard errors.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

References

    1. Kosior PR, “Foundations of Orientation and Mobility,” Int. J. Orientat. Mobil, 2010, doi: 10.21307/ijom-2010-008. - DOI
    1. McClimens A, Partridge N, and Sexton E, “How do people with learning disability experience the city centre? A Sheffield case study,” Heal. Place, 2014, doi: 10.1016/j.healthplace.2014.02.014. - DOI - PubMed
    1. Lubin A and Deka D, “Role of Public Transportation as Job Access Mode: Lessons from Survey of People with Disabilities in New Jersey,” Transp. Res. Rec. J. Transp. Res. Board, 2012.
    1. Townley G, Kloos B, and Wright PA, “Understanding the experience of place: Expanding methods to conceptualize and measure community integration of persons with serious mental illness,” Heal. Place, 2009, doi: 10.1016/j.healthplace.2008.08.011. - DOI - PMC - PubMed
    1. Foulke E, “The perceptual basis for mobility.,” Am. Found. Blind. Res. Bull, 1971.

LinkOut - more resources

-