Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2014 Oct;17(10):1410-7.
doi: 10.1038/nn.3807. Epub 2014 Sep 7.

Information-limiting correlations

Rubén Moreno-Bote  1 , Jeffrey Beck  2 , Ingmar Kanitscheider  3 , Xaq Pitkow  4 , Peter Latham  5 , Alexandre Pouget  6
Affiliations

Information-limiting correlations

Rubén Moreno-Bote et al. Nat Neurosci. 2014 Oct.

Abstract

Computational strategies used by the brain strongly depend on the amount of information that can be stored in population activity, which in turn strongly depends on the pattern of noise correlations. In vivo, noise correlations tend to be positive and proportional to the similarity in tuning properties. Such correlations are thought to limit information, which has led to the suggestion that decorrelation increases information. In contrast, we found, analytically and numerically, that decorrelation does not imply an increase in information. Instead, the only information-limiting correlations are what we refer to as differential correlations: correlations proportional to the product of the derivatives of the tuning curves. Unfortunately, differential correlations are likely to be very small and buried under correlations that do not limit information, making them particularly difficult to detect. We found, however, that the effect of differential correlations on information can be detected with relatively simple decoders.

PubMed Disclaimer

Figures

Figure 1
Figure 1
The effect of correlations on a population code with translation invariant tuning curves. (a) A neuronal population with translation invariant tuning curves to the stimulus s (arbitrary units). (b) Correlations in vivo often decrease as a function of the difference in preferred stimuli, δs. This decrease is often reasonably well described by a circular Gaussian (as shown here) or an exponential function of the stimulus. (c) Information in a population of neurons with the tuning curves and correlations shown in a and b. The different curves correspond to different maximum correlation values (the correlation coefficient, ρ, at δs = 0). When neurons are independent (ρ = 0), the information scales linearly with the number of neurons. For ρ > 0, the information saturates as N increases. This plot is often used to argue that correlations such as those in b limit information in population codes.
Figure 2
Figure 2
Decorrelation does not necessarily increase information. (a) Network architecture. Each neuron receives input from recurrent connections, and, in addition, external input with mean proportional to s, but corrupted by shared and independent noise (Online Methods, equation (9)). (b) Mean correlation coefficient as a function of the number of neurons. The three colors correspond to three networks that differ only in their connection strengths, but chosen so that, in all cases, the mean firing rate was close to 40 Hz. For a fixed number of neurons, the mean correlation coefficients can vary by more than a factor of 10 across the three networks. Solid lines show the analytical predictions for non-leaky integrate-and-fire neurons, and the open and closed circles show the results of the simulations with non-leaky and leaky integrate-and-fire neurons, respectively. Some of the open circles fall behind the closed ones and are therefore not visible. The observation time window was 10 s for the non-leaky integrate-and-fire neurons and 2 s for the leaky integrate-and-fire neurons. (c) Histogram of correlation coefficients (N = 500); same color code as in b. The red distribution has a mean very close to zero (0.013), whereas the green distribution has a mean of 0.108. (d) Information as a function of the number of output neurons (same color code as in b). The black solid line shows the input information (equation (1)) and the red solid line corresponds to the information in the input for infinite networks (equation (1) with N taken to be infinity). As in b, open and closed circles show the results of the simulations with non-leaky and leaky integrate-and-fire neurons, respectively. Inset, information as a function of the correlations for each network size. Unlike in Figure 1c, the information at which the network saturates (open dots, N = 500) has a very weak dependence on the mean correlations; instead, all three networks show nearly the same asymptotic value of information. Even for small networks away from the saturation, information is independent of overall correlations for non-leaky integrate-and-fire neurons, and only weakly dependent on correlations for leaky integrate-and-fire neurons. Thus, smaller correlations do not necessarily imply more information. Error bars correspond to s.e.m.
Figure 3
Figure 3
Differential correlations induced by a shifting hill. (a) Population activity for neurons with translation invariant tuning curves (as in Fig. 1a), with neurons ranked according to their preferred stimulus. The red and pink curves correspond to the population response to the same input on two different trials. We assumed that the variability is such that the hill simply translated sideways from trial to trial. When this was the case, correlations were mostly proportional to the product of the derivatives of the tuning curves. The two blue neurons were positively correlated because the derivatives of their tuning curves were negative for both neurons, and the product of their derivatives was therefore positive. In contrast, the green neuron was negatively correlated with either of the blue neurons because its derivative had the opposite sign as theirs. (b) Population patterns of activity, as in a, can be thought of as points in an N-dimensional space, in which each axis corresponds to the activity of one neuron. Only two neural dimensions are shown here, out of N. As the hill of activity shifts with s, the mean population activity traces out a curve (the black curve labeled f(s)). Pure information-limiting noise looks like a sideways shift of the hill, corresponding to movement along the curve. In this case, the activity has a probability distribution that wraps along the curve f(s), as shown in yellow. If the variability is small compared with the curvature of the manifold, the yellow distribution can be approximated by the blue distribution that lies along the tangent to the curve, corresponding to the f′(s) direction, with a resultant covariance matrix proportional to f′(s)f′(s)T.
Figure 4
Figure 4
Shared connections do not necessarily induce information limiting correlations. (a) Network architecture. The network consists of two layers. The input layer contains N neurons. These are modeled as white noise process and they project to N output neurons. The output neurons, which were not recurrently connected, were modeled as non-leaky integrate-and-fire neurons. The output neurons also receive independent and identically distributed (i.i.d.) noise. A parameter f controls the probability that a connection is shared by two neurons. (b) The Fano factor was roughly constant at around 0.9. Not shown are the firing rates; these were held at 50 Hz, independent of network size, by adjusting the mean drive to each neuron. (c) Correlations were also roughly constant as the size of the network increases. (d) Information in the output layer (which was the same as in the input layer; data not shown). Unlike in Figure 1c, where positive correlations led to information saturation as a function of the number of neurons, information increased linearly with network size. Thus, the correlations induced by the shared connectivity do not limit information in this case. In b–d, the solid lines indicate analytical predictions, the dots show the results of simulations and error bars represent s.e.m.
Figure 5
Figure 5
Differential correlations. (a) Average correlation coefficient as a function of the difference in preferred stimuli for a population of neurons with differential correlations (as shown in Fig. 3a) plus independent Poisson noise. The average was taken over all pairs of neurons with the same difference in preferred stimuli, δs. Correlations were negative for large δs, in contrast to the correlations found in vivo (Fig. 1b). (b) Correlation coefficient between two pairs of neurons as a function the stimulus. The blue line represents a pair of neurons with preferred stimuli −0.5 and 0.5. The red line represents a pair with preferred stimuli −0.25 and 0.25. Correlations were strongly modulated by the value of the stimulus.
Figure 6
Figure 6
Small differential correlations can have a large effect on information. (a) A population code with tuning curves of varying amplitudes. (b) Information as a function of the number of neurons for the population code shown in a. The two curves correspond to two different covariance matrices, Σ0 (red) and Σ0 + εf′f′T (blue), where Σ0 is a covariance matrix in which pairwise covariances follow a decaying exponential function of the cosine of the difference in preferred stimuli (as in Fig. 1b; see Supplementary Modeling). Information saturated in the presence of differential correlations (the εf′f′T component). (c) Empirically estimated correlations from 1,000 trials plotted as function of the difference in preferred stimuli for the two covariance matrices Σ0 (red) and Σ0 + εf′f′T (blue). The two distributions of correlation coefficients were nearly indistinguishable even though the amount of information was very different (b). (d) Percentage correct in a binary classification task as a function of the number of neurons. The red and blue curves correspond to two different covariance matrices, Σ0 (red) and Σ0 + εΔf ΔfT (blue), where Σ0 is as described in b and Δf is the mean difference of activity of the neurons for the two stimulus classes; the black curve corresponds to the performance of an independent population. Performance was limited to 72% when ΔfΔfT correlations were present, but reached the value predicted for an independent code when there were no ΔfΔfT correlations (Supplementary Modeling). Data points and error bars represent mean and s.e.m., respectively.
Figure 7
Figure 7
Estimating information from neuronal populations with differential correlations (Supplementary Modeling). (a–c) Empirical estimate of Fisher information versus the number of sampled neurons (N) and number of trials (M). In all panels, the solid blue line shows the average Fisher information as a function of the number of simultaneously sampled neurons, with the average calculated over 20 random sets of N neurons; the black horizontal line shows the true Fisher information for the entire population, which is assumed to have infinite size. Error bars represent s.e.m. The same color code for M is used in all panels. (a) Fisher information computed from equation (3), using a covariance matrix in which only N/2 correlation values were measured experimentally. The missing entries were estimated from the empirical measurements by requiring that they have approximately the same statistics as the observed correlation coefficients. Regardless of the number of trials, the estimated information failed to reveal the saturation of information, thereby missing the presence of differential correlations. (b) Fisher information again computed from equation (3), but using a covariance matrix estimated from N simultaneously recorded neurons. The estimated information still missed the information saturation even for thousands of trials. (c) Fisher information estimated with a locally optimal linear estimator trained with early stopping to prevent overfitting. This method consistently returned a lower bound on the true information for the entire population (horizontal black line), even for a small number of trials and neurons. As a result, it revealed the presence of information limiting correlations. Data points represent mean values in all panels.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Tolhurst DJ, Movshon JA, Dean AD. The statistical reliability of signals in single neurons in cat and monkey visual cortex. Vision Res. 1983;23:775–785. - PubMed
    1. Averbeck BB, Latham PE, Pouget A. Neural correlations, population coding and computation. Nat. Rev. Neurosci. 2006;7:358–366. - PubMed
    1. Zohary E, Shadlen MN, Newsome WT. Correlated neuronal discharge rate and its implication for psychophysical performance. Nature. 1994;370:140–143. - PubMed
    1. Maynard EM, et al. Neuronal interactions improve cortical population coding of movement direction. J. Neurosci. 1999;19:8083–8093. - PMC - PubMed
    1. Averbeck BB, Lee D. Neural noise and movement-related codes in the macaque supplementary motor area. J. Neurosci. 2003;23:7630–7641. - PMC - PubMed

Publication types

LinkOut - more resources

-