Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2018 Apr 18:2018:3493826.
doi: 10.1155/2018/3493826. eCollection 2018.

Extraspectral Imaging for Improving the Perceived Information Presented in Retinal Prosthesis

Walid Al-Atabany  1 , Musa Al Yaman  2 , Patrick Degenaar  2
Affiliations

Extraspectral Imaging for Improving the Perceived Information Presented in Retinal Prosthesis

Walid Al-Atabany et al. J Healthc Eng. .

Abstract

Retinal prosthesis is steadily improving as a clinical treatment for blindness caused by retinitis pigmentosa. However, despite the continued exciting progress, the level of visual return is still very poor. It is also unlikely that those utilising these devices will stop being legally blind in the near future. Therefore, it is important to develop methods to maximise the transfer of useful information extracted from the visual scene. Such an approach can be achieved by digitally suppressing less important visual features and textures within the scene. The result can be interpreted as a cartoon-like image of the scene. Furthermore, utilising extravisual wavelengths such as infrared can be useful in the decision process to determine the optimal information to present. In this paper, we, therefore, present a processing methodology that utilises information extracted from the infrared spectrum to assist in the preprocessing of the visual image prior to conversion to retinal information. We demonstrate how this allows for enhanced recognition and how it could be implemented for optogenetic forms of retinal prosthesis. The new approach has been quantitatively evaluated on volunteers showing 112% enhancement in recognizing objects over normal approaches.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Concept of optogenetic retinal prosthesis, enhanced with extraspectral wavelengths. (a) A concept wearable headset which would project light from (b) a high-density LED array. (c) Cameras which could acquire the infrared, visible, and ultraviolet. (d) Image acquisition form visible, (e) Infrared image, and (f) combined enhanced image prior to retinal processing. (g) A control unit.
Figure 2
Figure 2
The visible/IR pathways from the two cameras. This includes scene acquisition, contrast enhancement, retargeting, and simulator encoding.
Figure 3
Figure 3
The optical setup of the system. Visible/IR cameras are aligned together through specially designed beamsplitter that reflects the same scene into IR and visible pathways.
Figure 4
Figure 4
Compressing and stretching the dynamic scale for the IR image and its inverse (a). (b) Exponentially scaling the combined images in (a).
Figure 5
Figure 5
Pathway for the infrared image. (a) Captured IR image. (b, c) Segmented hot and cold objects. (d) Segmentation map.
Figure 6
Figure 6
The output of different stages of the flow chart shown in Figure 2. IR path: IR image (a) and its segmentation map (b). Visible path: visible (c), anisotropic diffusion (d), and gradient images (e). Enhanced: the edge-weighted (f), cartoon-like (g), segmented edge-weighted (h), and cartoon-like enhanced images (i).
Figure 7
Figure 7
The effect of segmentation process on image retargeting. The left column shows the original scene (a) and the importance map for the retargeting process (b). The middle column shows the linearly scaled image (c), the nonlinear retargeted image of the segmented cartoon (d), and edge-weighted images (e). (f) shows a close-up of the individual demonstrating the effect of nonlinear retargeting on the size of important features.
Figure 8
Figure 8
Simulating the vision at different sizes of stimulating arrays. A simulation for what a subject with different stimulating retinal prosthesis arrays (16 × 16, 32 × 32, 64 × 64 and 128 × 128) would perceive is shown from the top row to the bottom row. The left column is a simulation for the original scene and the middle and right columns are for the segmented cartoon and edge-weighted images, respectively, after nonlinearly retargeted by 30% in both directions.
Figure 9
Figure 9
The efficiency of the optimization algorithm in recognizing actions in dynamic videos. Simulated video perception for different stimulating array sizes (16 × 16 and 32 × 32) have been presented to the candidates and they asked to recognize the objects. Greater improvement in number of recognized objects has been achieved using our scene-optimization algorithm. The error bars represent the standard error of the data.
Figure 10
Figure 10
The efficiency of the optimization algorithm in identifying and counting persons in dynamic videos. Simulated video perception for different stimulating array sizes (16 × 16 and 32 × 32) have been presented to the candidates and asked to count the number of persons in each video. Greater improvement has been achieved using our scene optimization algorithm. The error bars represent the standard error of the data.
Figure 11
Figure 11
The efficiency of the scene optimization algorithm in recognizing actions for each video when simulating the perception of using stimulating array size of 16 × 16.
Figure 12
Figure 12
The efficiency of the scene optimization algorithm in identifying and counting subjects for each video when simulating the perception of using a stimulating array size of 16 × 16.
Figure 13
Figure 13
Visually enhanced IR segmentation. The base IR image (Figure 5(a)) above is smoothed with the anisotropic method (a) and then cartoonised via edge overlay from the visual scene (b).
Figure 14
Figure 14
Bipolar cell output and reconstructed image from RGC. A simulation for what a subject with different stimulating retinal prosthesis arrays (128 × 128, 64 × 64, 32 × 32, and 16 × 16) would perceive the full bipolar image and the approximate (positive values only), columns one and two. Columns three and four show the reconstructed images from the RGC for full bipolar and approximate bipolar.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

References

    1. Foster A., Resnikoff S. The impact of vision 2020 on global blindness. Eye. 2005;19(10):1133–1135. doi: 10.1038/sj.eye.6701973. - DOI - PubMed
    1. Foerster O. Beitrage zur Pathophysiologie der Sehbahn und der Sehsphare. Journal für Psychologie und Neurologie. 1929;39:463–485.
    1. Brindley G. S., Lewin W. S. The sensations produced by electrical stimulation of the visual cortex. The Journal of Physiology. 1968;196(2):479–493. doi: 10.1113/jphysiol.1968.sp008519. - DOI - PMC - PubMed
    1. Stone J. L., Barlow W. E., Humayun M. S., Juan E., Milam A. H. Morphometric analysis of macular photoreceptors and ganglion-cells in retinas with retinitis-pigmentosa. Archives of Ophthalmology. 1992;110(11):1634–1639. doi: 10.1001/archopht.1992.01080230134038. - DOI - PubMed
    1. Zrenner E., Greger B. Chapter 1 - restoring vision to the blind: the new age of implanted visual prostheses. Translational Vision Science & Technology. 2014;3(7):p. 3. doi: 10.1167/tvst.3.7.3. - DOI - PMC - PubMed

Publication types

LinkOut - more resources

-