Learn more: PMC Disclaimer | PMC Copyright Notice
Interactions of Prosthetic and Natural Vision in Animals With Local Retinal Degeneration
Abstract
Purpose
Prosthetic restoration of partial sensory loss leads to interactions between artificial and natural inputs. Ideally, the rehabilitation should allow perceptual fusion of the two modalities. Here we studied the interactions between normal and prosthetic vision in a rodent model of local retinal degeneration.
Methods
Implantation of a photovoltaic array in the subretinal space of normally sighted rats induced local degeneration of the photoreceptors above the chip, and the inner retinal neurons in this area were electrically stimulated by the photovoltaic implant powered by near-infrared (NIR) light. We studied prosthetic and natural visually evoked potentials (VEP) in response to simultaneous stimulation by NIR and visible light patterns.
Results
We demonstrate that electrical and natural VEPs summed linearly in the visual cortex, and both responses decreased under brighter ambient light. Responses to visible light flashes increased over 3 orders of magnitude of contrast (flash/background), while for electrical stimulation the contrast range was limited to 1 order of magnitude. The maximum amplitude of the prosthetic VEP was three times lower than the maximum response to a visible flash over the same area on the retina.
Conclusions
Ambient light affects prosthetic responses, albeit much less than responses to visible stimuli. Prosthetic representation of contrast in the visual scene can be encoded, to a limited extent, by the appropriately calibrated stimulus intensity, which also depends on the ambient light conditions. Such calibration will be important for patients combining central prosthetic vision with natural peripheral sight, such as in age-related macular degeneration.
Sensory disorders such as loss of vision or hearing are among the most debilitating medical conditions, with devastating impact on physical and social interactions. Even a partial sensory loss, as in the case of age-related macular degeneration (AMD), can have dramatic consequences on patients' well-being, with high social cost. In this disease, the central part of the visual field, which mediates high-acuity vision, is lost due to either a local invasion of choroidal blood vessels into the retina (wet form) or atrophy of the retinal pigment epithelium and a subsequent loss of photoreceptors (dry form). Although patients suffering from the wet form benefit from anti-VEGF (vascular endothelial growth factor) treatments, the dry form accounting for the majority of patients remains untreatable.
One of the potential strategies for vision rehabilitation in these patients is the implantation of retinal prostheses. Epiretinal implants aim at stimulating the ganglion cells,1 and subretinal and suprachoroidal implants stimulate primarily bipolar cells2,3 to restore some level of visual perception. Several of these prosthetic approaches are being actively tested in patients blinded by retinitis pigmentosa (RP).4–9 With both types of retinal implants, patients recover some light perception, shape recognition, and orientation capabilities, providing an important proof of principle that degenerated retina is capable of transmitting patterns of electrical activation to the brain, which is capable of interpreting these signals as patterned visual percepts. Due to their limited spatial resolution and functional benefits, these systems have been implanted so far only in patients blinded by RP. However, the technological advances in electrode density and stimulation efficiency open the door to high-resolution restoration of sight, which could match and even exceed the acuity of the remaining peripheral vision in AMD patients.10 In that case, understanding of the interactions between prosthetic signals and normal peripheral vision becomes important. Encouraging findings with cochlear prostheses demonstrated patients' ability to simultaneously utilize their remaining natural hearing at low frequencies and prosthetic stimulation at high frequencies, increasing their acoustic bandwidth and improving speech recognition.11,12
Here, we describe the interaction of prosthetic and normal visual signals in rats with local retinal degeneration mimicking the central scotoma in patients with AMD. Cortical potentials in response to simultaneous visual and electrical stimulation of the retina reveal similarities, differences, and interactions between prosthetic and natural vision.
Materials and Methods
Implant Fabrication
Photovoltaic arrays were manufactured on silicon-on-insulator wafers using an eight-mask lithographic process, as described previously.13 To produce anodic-first pulses of electric current, the n-doped and p-doped regions in the diodes were reversed compared to the previous description. Indeed, anodic-first pulses elicit network-mediated responses of ganglion cells with thresholds three to four times lower than cathodic-first pulses.14 Photovoltaic arrays consisted of 1-mm-diameter and 30-μm-thick structures (Fig. 1A) composed of 140-μm pixels, separated by 5-μm-wide trenches (Fig. 1B). Each pixel contained two photodiodes (Figs. 1B, B,3)3) connected in series between the active (Fig. 1B, B,1)1) and return (Figs. 1B, B,2)2) electrodes.
Scanning electron microscopy of the subretinal photovoltaic device. (A) A 1-mm-diameter chip composed of a hexagonal array of 140-μm pixels. (B) Each pixel is composed of a 40-μm-diameter active electrode (1) and a return electrode ring 5 μm in width (2). The electrical current is generated by two photodiodes connected in series between these two electrodes (3). The return electrodes of all pixels are connected through metal bridges.
Subretinal implantation results in local photoreceptor degeneration. (A) Optical coherence tomography (OCT) image of an implanted retina 1 year after surgery. The outer segments and outer nuclear layer are completely absent above the device. The retina outside of the implanted area retained its normal structure. (B, C) Histology of the control and implanted areas confirms the preservation of the inner nuclear layer (INL) and the ganglion cell layer (GCL) above the implant and complete degeneration of the outer nuclear layer (ONL) and outer segments (OS) 1 year after implantation.
Multifocal stimulation paradigm. (A) Stimulation of the surrounding healthy retina and the degenerate retina above the implant performed by simultaneous projection of NIR and visible light. A random pattern of 1-mm squares was applied at 2 Hz while recording from the corneal and the cortical electrodes. (B, C) Multifocal analysis of the evoked signals allowed extracting the contribution from each 1-mm square of the pattern independently for both ERG (B) and VEP (C) waveforms. The corneal signal corresponding to the area of the implant reveals the stimulation artifact, which is absent in the surrounding signals. (D) VEP signals shown over the retinal locations they originate from. Number 3 corresponds to the position of the implant and 4 to the adjacent healthy retina. (E) The VEP signals coming from the implanted area are extracted and can be directly compared to the visual signals elicited in the neighboring area.
Implantation Procedure
A total of nine Long Evans rats (Charles Rivers Farm, Wilmington, MA, USA) were used in this study. Animals were operated at 40 days. The subretinal implantation technique was similar to the one previously reported by our group.10,15 Animals were anesthetized with a mixture of ketamine (75 mg/kg) and xylazine (5 mg/kg) injected intramuscularly. A 1.5-mm incision was made through the sclera and choroid 1.5 mm posterior to the limbus; the retina was lifted with an injection of saline solution, and the implant was inserted into the subretinal space. The sclera and conjunctiva were sutured with nylon 10-0, and topical antibiotic (bacitracin/polymyxin B) was applied on the eye postoperatively. The anatomical integration of the device in the subretinal space was evaluated by OCT (HRA2-Spectralis; Heidelberg Engineering, Heidelberg, Germany) in periodic examinations beginning 1 week after surgery. All animals were used in accordance with the Association for Research in Vision and Ophthalmology Statement Regarding the Use of Animals in Ophthalmic and Vision Research after approval from the Stanford University Animal Institutional Review Board.
Histology
One year after implantation, eyes (n = 3) were enucleated and fixed in 1.25% or 2.5% glutaraldehyde, 1% paraformaldehyde fixative prepared in 0.1 M sodium cacodylate buffer with 5 mM calcium chloride and 5% sucrose for 24 hours at room temperature. Lenses were removed and eyes were trimmed to a block size and postfixed in 2% aqueous osmium tetroxide for 2 hours at room temperature. Tissue was then dehydrated in graded alcohol, infiltrated with propylene oxide and epoxy (Araldite/Embed; EMS, Hatfield, PA, USA), embedded in pure epoxy, and polymerized at 60°C for 24 hours. Thin sections (1 μm) were stained with 0.5% toluidine blue, and slides were examined under a light microscope.
Implantation of the Cortical Electrodes
Three transcranial screw electrodes (00 × 1/4 stainless steel, part FF00CE250; Morris, Southbridge, MA, USA) were implanted similarly to a previously published technique16 and secured in place with cyanoacrylate glue and dental acrylic. These electrodes penetrate the skull but do not enter the brain tissue. Two electrodes were placed over the visual cortex, one in each hemisphere, 4 mm lateral from the midline, 6 mm caudal to the bregma. One reference electrode was implanted 2 mm right of the midline and 2 mm anterior to the bregma. Nose and tail needle electrodes served as reference and ground, respectively. Recordings started 2 months after the subretinal implantation to ensure the complete loss of photoreceptor light-mediated signals above the chip.
Anesthesia During Recordings
Rats were anesthetized with a mixture of ketamine (37.5 mg/kg) and xylazine (2.5 mg/kg) injected intramuscularly. The following steps were taken to ensure steady anesthesia. Spontaneous eye movements and respiratory patterns were checked periodically; supplementary injection of half the initial dose was administered every 40 minutes, or as needed, and recording sessions were limited to 120 minutes per session. A heating pad was used to maintain the body temperature at 37.5 ± 0.5°C.
Retinal Stimulation
The stimulation system included a single-mode pigtailed near-infrared (NIR) (915 nm) laser and a visible light (532 nm) laser coupled into a 1-mm-diameter optical fiber. The collimated output beam illuminated a Digital Micro-mirror Device (DMD; DLP Light Commander; LOGIC PD, Carlsbad, CA, USA) to form the patterns. The optical system was mounted on a slit lamp (Zeiss SL-120; Carl Zeiss, Thornwood, NY, USA) to allow direct observation of the patterns on the retina with a charge-coupled device camera (acA1300-60gmNIR; Basler, Ahrensburg, Germany). Following the pupil dilation, the cornea was covered with a viscoelastic gel and a coverslip to cancel the optical power of the corneal curvature and ensure stimulus focalization. The coverslip was taped to the animal's head to maintain a stable contact with the cornea during the entire recording session. Ocular retraction was required in some cases to help align the implant with the beam. The position of the light pattern on the implant was continuously monitored and adjusted if necessary by the experimenter. For each animal, two different recording paradigms were performed. In one of these, NIR alone was presented over the entire implant, and the irradiance was varied from 0.06 to 4 mW/mm2 with 10-ms pulse duration. In another session, visible and NIR were simultaneously presented in a multifocal paradigm with a constant NIR irradiance of 4 mW/mm2, while visible light irradiance was varied from 15 nW/mm2 to 3 μW/mm2. Every recording was performed with two different ambient white light conditions: dim (0.8–1.1 nW/mm2) and bright (90–115 nW/mm2), corresponding to 2.4 and 250 nW, respectively, transmitted through a 3.5-mm iris, matching the pupil size of a dilated rat eye.
Visually Evoked Potential Recording and Analysis
Visually evoked potential (VEP) signals were recorded with an Espion E2 system (Diagnosys, Inc., Lowell, MA, USA) at 1-kHz sampling rate using 0.5- to 500-Hz bandpass filter, and averaged over 250 trials for each experiment. Cortical thresholds were determined for the stimuli covering the whole implant (1 mm in diameter) using 10-ms pulses, and defined as the minimum light intensity for which the VEP amplitude during the first 100 ms after the pulse exceeded six times the noise level. This noise level was defined as the standard deviation of the signal during the 50 ms preceding the stimulus. Modulation of the VEP amplitude by light intensity was measured using 10-ms pulses, and normalized to the response at 1 mW/mm2.
Multifocal Stimulation
The multifocal stimulation paradigm was implemented in a manner similar to that reported previously17,18 but using a binary random noise instead of the m-sequence. Light patterns (random checkerboards, 1-mm-square size) were generated by custom software (Psycholtoolbox, Matlab; The Mathworks, Natick, MA, USA). For each visible light intensity, the stimulus consisted of 1000 random checkerboards containing 1-mm squares, alternating every 500 ms and illuminated by a single 10-ms light pulse during each phase (4 mW/mm2 for NIR and variable irradiance for visible light). After acquisition, the multifocal analysis was performed offline by a custom routine (Matlab; The Mathworks). The stimulation artifact measured on the cornea was used to synchronize the stimulation pattern with the recording. For each square of the checkerboard, the first order of the VEP signal was obtained by adding the trials in which this square was ON and subtracting the trials in which it was OFF (i.e., correlating the recording with the stimulus). For two neighboring squares of the checkerboard, the second-order signal was obtained by adding trials in which the two squares were in the same state (either ON or OFF) and subtracting the trials in which the two squares were in opposition. The second-order amplitude reveals the deviation from linear interaction between the two contributions.
Fitting and Prosthetic Contrast Mapping
Dependence of the VEP amplitude on contrast of the visible light flash and on NIR irradiance was fitted by sigmoidal curves 
. For normalization of the prosthetic stimulation to the ambient light background, the scaling factor λ between the two conditions (dim and bright backgrounds) was obtained by minimizing the deviation of the combined data [Xdim;Xbright / λ] [Ydim;Ybright] from a single sigmoidal fit.
For each visible light contrast value (x), the corresponding NIR contrast (y) producing the same VEP amplitude was defined, up to the maximum of prosthetic response (ymax). Mathematically, if f(x) and g(y) are the normalized VEP responses for visible and prosthetic stimulation, the matching curve is defined by y = h(x) = g−1 ∘ f(x).
For (f) and (g) being sigmoidal curves in the form 
and 
, the 
is defined up to: 
.
Results
Subretinal Implantations and Local Degeneration of Photoreceptors
Long Evans (wild-type) rats were implanted subretinally with 1-mm-diameter silicon arrays (see Methods; Fig. 1A and previously published study13). Each pixel contained two photodiodes connected in series between an active, 40-μm-diameter electrode and a 5-μm-wide ring return electrode surrounding the pixel (Fig. 1B). These pixels convert light into electrical current flowing through the tissue between the stimulating and return electrodes. Near-infrared (915 nm) illumination was used to activate the photovoltaic pixels while avoiding any visual response in rats.15,19
The subretinal implantation triggered the loss of photoreceptor outer segments above the implant within a month and a subsequent loss of the outer nuclear layer after 3 months20 (Fig. 2A). The inner retina, however, remained preserved even 1 year after implantation (Figs. 2B, B,2C).2C). Therefore, the subretinal implantation itself created a local model of retinal degeneration with a normal retina outside of the implanted area and a scotoma above the prosthesis.
Equivalent Brightness of Prosthetic Percept and Dependence on Background Illumination
In this animal model of local retinal degeneration, we assessed prosthetic and natural vision by recording from the primary visual cortex via transcranial screw electrodes (see Methods). Prosthetic or visually evoked potentials in response to invisible NIR (915 nm) or visible (532 nm) light were recorded separately or simultaneously.
To assess the relative amplitude of prosthetic and natural visual responses, we used a multifocal protocol for probing the relative contributions and linearity of summation of the two cortical signals (previously published study17 and Fig. 3A). Both NIR and visible light patterns are simultaneously applied over the entire retinal area. However, due to the local loss of photoreceptors, the implanted area receives only electrical stimulation (see Methods). Although the square area of the checkerboard exceeds the circular implant by approximately 20%, degeneration of the photoreceptors up to 100 μm away from the edge of the implant reduces the overlap with photosensitive area to a few percent. We demonstrated previously that projection of a square pattern of similar size with visible light over the implant did not elicit detectable cortical responses.10 Based on this observation, we disregarded the mismatch between the shapes of the implant and the checkerboard. The multifocal analysis allows extracting the VEP originating from each square of the checkerboard and calculating the relative contributions of the prosthetic and normal visual inputs to the cortical signal (Fig. 3C). Simultaneous measurements of the electrical signal on the cornea using ERG electrode reveal the stimulation artifact from the implant (Fig. 3B, location 3). Such multifocal protocol is essential to avoid scattering effects in the retina and extract the cortical contribution of each 1-mm square in the pattern for comparison with the implant-mediated responses (Figs. 3D, D,33E).
To evaluate the strength of the prosthetic percepts relative to normal visual response, we modulated the visible light intensity from 0 to 3 μW/mm2 and the NIR irradiance from 60 μW/mm2 to 4 mW/mm2 in both dim and bright room light conditions (see Methods). In both cases, the VEP responses increased with increasing light intensities, and both response curves shifted to higher irradiances at brighter background conditions (Figs. 4A, A,4B).4B). This indicates that adaptation of the surrounding retina to background illumination affects the prosthetic cortical response originating in the scotoma.
Responses to visible light and prosthetic stimulation as a function of stimulus intensity and background light. (A, B) Prosthetic and visible light response curves in dim and bright room lighting. The noise level was subtracted. (C, D) Normalized representation of the data. The VEP amplitude for prosthetic and visible stimulation plotted as a function of the stimulus contrast, defined as the ratio of the flash intensity on the retina to the background intensity of room lighting. In this representation, responses at two background levels merged into a single sigmoidal curve, demonstrating that VEP amplitude depends primarily on the stimulus contrast over the background. Unlike the visible stimulation, where the two curves corresponding to 104-fold difference in background merged when the flash intensity was normalized by the ambient irradiance, with prosthetic stimulation, they merged into a single sigmoidal fit when NIR stimulus intensity was divided by 3.1 for the 104 times brighter background. (E) Definition of the strength of the prosthetic stimulus (y1) that elicits the same VEP amplitude as the visible light stimulus with normalized contrast x1. Visible light contrasts corresponding to the VEP exceeding the maximum prosthetic response (ymax) cannot be faithfully represented by prosthetic stimulation. (F) The matching curve relating the visible contrast to the corresponding NIR amplitude for a given background.
This adaptation to background illumination can also be interpreted as a modulation of the contrast of the stimulus. Indeed, expressing the flash brightness in units of contrast by normalizing the stimulus irradiance to the background level converged the two curves from Figures 4A and and4B4B into a single continuous VEP response curve fitted by a sigmoidal function (Figs. 4C, C,4D;4D; Methods). The 104-fold increase in background illumination required a 104-fold increase in intensity of the visible light stimulus to produce the same cortical response. However, with the prosthetic response the ratio was very different: Only a 3.1-fold increase in the NIR irradiance produced the same cortical response at 104-fold brighter background (see Methods). Multiplication of the NIR irradiance by this factor resulted in fusion of the sigmoidal curves corresponding to the dim and bright background (Fig. 4A).
Measuring these contrast sensitivity curves for both natural and prosthetic signals provides guidance for adjusting the contrast in the NIR image to achieve perceptual coherence in case of partial photoreceptor degeneration. To elicit VEP of the same amplitude with normal and prosthetic stimulation, we established the correspondence between the contrasts of the normal and prosthetic stimuli (Fig. 4E). In this procedure, the values of the visible light contrast (x1) are converted into the corresponding NIR irradiance (y1) producing the same VEP amplitude for a given background (Fig. 4F). However, since prosthetic response saturates at a lower VEP level than natural response, contrast exceeding a certain value (xmax) cannot be faithfully represented by prosthetic stimulation, and the contrast transformation curve plateaus above that value. This curve defines the NIR irradiance, which elicits the same VEP amplitude as the visible light flash at the same ambient light background.
Linearity of Summation Between Normal and Prosthetic Vision
To check the degree of linearity in the summation of the normal and prosthetic vision we analyzed the second moment of the multifocal signal (see Methods). Subtracting the cortical signals recorded in trials when prosthetic and normal stimuli are not simultaneous from the trials when they are simultaneous yields the first-order deviation from linear prediction (Fig. 5A and Methods). This deviation was not significantly different from the noise (Figs. 5B, B,5C),5C), indicating that normal and prosthetic contributions to the retinal signals sum linearly in the visual cortex.
Nonlinearity analysis. (A) The second order of the multifocal signal (deviation from linear behavior) is obtained by adding signals where the implant and the neighboring area are either both ON or both OFF (left), and subtracting the signal when they are in opposition (right). (B) Sample signal of the second order (black), compared to the multifocal signal (blue). (C) No significant difference was found between the second-order amplitude and the noise level for comparable levels of prosthetic and visible light amplitudes.
Discussion
This study demonstrates that prosthetic and natural visual signals are transmitted to the brain simultaneously without inhibiting each other. Coexistence of the normal and prosthetic vision is critical for restoration of sight in patients with partial loss of vision. We found that the natural visually evoked signals sum linearly with prosthetic response, without detrimental interactions. Interestingly, ambient background lighting affects the prosthetic response, although to a much lesser extent than with visible light stimulation, and therefore it should be considered for proper encoding of the prosthetic image.
Decreasing the checkerboard pitch below the implant size in the multifocal stimulation paradigm should allow characterization of the summation properties and integration of prosthetic and normal vision on a finer spatial scale. In rats, arrays with 70-μm pixels provided grating visual acuity matching the pixel pitch,10 so resolution of a single pixel might be possible. However, since signal-to-noise ratio decreases with the pitch of the checkerboard pattern, these measurements will likely require much longer integration.
In terms of the amplitude, the maximum strength of the cortical signals elicited by the subretinal prosthesis was approximately three times lower than the saturation level elicited by the visible flash applied over the same area on the retina. Several mechanisms might be responsible for this difference. First, unlike what occurs with optical stimulation, only a fraction (approximately half) of the retinal ganglion cells (RGCs) respond to subretinal electrical stimulation, as measured in vitro.21 Second, the maximum number of spikes elicited in RGCs by electrical stimulation is approximately half that elicited by visible flashes.10 Finally, indiscriminate stimulation of ON and OFF pathways could lead to partial cancellation of these signals in the brain.
One of the important questions for proper encoding of prosthetic stimulation is the relationship between the cortical potentials and the actual perceptual brightness. In human patients the VEP signal scales linearly with the logarithm of the contrast,22 and extrapolation of the VEP amplitude to the noise level closely matches the psychophysical perceptual threshold. Therefore, comparable amplitudes of the prosthetic and natural visual responses and their linear summation indicate that the contrast correspondence between the two modalities can be established.
The photovoltaic retinal prosthesis can operate in patients with remaining peripheral vision as an augmented reality system: Video goggles transparent to visible light overlie the NIR images projected onto the subretinal implant.23 Since patients will use them indoors and outside, ambient light levels will vary over several orders of magnitude. Our results demonstrate that prosthetic responses are affected by the ambient light similarly to normal vision (although we were not able to measure the absolute retinal irradiance from diffuse ambient light) and therefore prosthetic representation of the objects should depend not only on their contrast in the original scene but also on the ambient light the patient is exposed to. Since the perceptual brightness (judged by the amplitude of VEP) of prosthetic stimulation is lower than the maximum brightness of the visible light by a factor of 3, prosthetic stimulation can represent only a part of the visible light response range.
Clinical trials of this system will enable comparing perception of prosthetic and natural stimulation and thereby refine the algorithms of image processing for prosthetic vision. Such calibration of the contrast adjustment algorithm according to the background lighting and contrast of the visual scene might need to be performed for every patient at different levels of background light. In addition to lenses correcting the refractive errors in patients, the goggles could include an adjustable neutral density filter for visible light to tune the ambient illumination of the retina to match prosthetic percepts.
Coexistence of prosthetic and natural vision, combined with high resolution of photovoltaic implants10 and their ease of implantation, opens the door to application of this technology for restoration of central vision in AMD patients.
Acknowledgments
We thank G. Goetz for technical help with the image projection system, M.F. Marmor and S. Picaud for stimulating discussions and encouragement, and J. Liao for access to the VEP recording setup.
Supported by the National Institutes of Health (NIH) (Grant R01-EY-018608), the Department of Defense (Grant W/81XWH-15-1-0009), NIH Clinical and Translational Science Awards (Award UL1 RR025744), and the Stanford Spectrum fund. KM was supported by an SU2P fellowship as part of an Research Councils UK Science Bridges award. HL was supported by the Foundation Voir et Entendre (Paris) and Pixium Vision.
Disclosure: H. Lorach, Pixium Vision (F); X. Lei, None; L. Galambos, None; T. Kamins, None; K. Mathieson, None; R. Dalal, None; P. Huie, None; J. Harris, None; D. Palanker, Pixium Vision (C), P