Towards a learning-theoretic analysis of spike-timing dependent plasticity (original) (raw)
Related papers
Reinforcement Learning With Modulated Spike Timing–Dependent Synaptic Plasticity
Journal of Neurophysiology, 2007
Spike timing–dependent synaptic plasticity (STDP) has emerged as the preferred framework linking patterns of pre- and postsynaptic activity to changes in synaptic strength. Although synaptic plasticity is widely believed to be a major component of learning, it is unclear how STDP itself could serve as a mechanism for general purpose learning. On the other hand, algorithms for reinforcement learning work on a wide variety of problems, but lack an experimentally established neural implementation. Here, we combine these paradigms in a novel model in which a modified version of STDP achieves reinforcement learning. We build this model in stages, identifying a minimal set of conditions needed to make it work. Using a performance-modulated modification of STDP in a two-layer feedforward network, we can train output neurons to generate arbitrarily selected spike trains or population responses. Furthermore, a given network can learn distinct responses to several different input patterns. We...
Spike-timing-dependent synaptic plasticity: from single spikes to spike trains
Neurocomputing, 2004
We present a neurobiologically motivated model of a neuron with active dendrites and dynamic synapses, and a training algorithm which builds upon single spike-timing-dependent synaptic plasticity derived from neurophysiological evidence. We show that in the presence of a moderate level of noise, the plasticity rule can be extended from single to multiple presynaptic spikes and applied to effectively train a neuron in detecting temporal sequences of spike trains. The trained neuron responds reliably under different regimes and types of noise.
Procedia Computer Science, 2018
The goal of this work is to study the influence of the neuron model choice on the results of STDP learning on base of simple toy tasks. As shown, the resulting mean output firing rate after STDP learning with restricted symmetric spike pairing scheme does not depend on the mean input rates for such neuron models as Leaky Integrate-and-Fire, Traub, and static neuron. Then this effect, being used to solve a typical classification task of Fishers Iris, demonstrates that the classification accuracy does not depend significantly on the choice of the neuron model. Thus, the independence of learning results on the neuron model gives the possibility to use simpler neuron models in further investigations.
Learning of Precise Spike Times with Membrane Potential Dependent Synaptic Plasticity
2014
Precise spatio-temporal patterns of neuronal action potentials underly e.g. sensory representations and control of muscle activities. However, it is not known how the synaptic efficacies in the neuronal networks of the brain adapt such that they can reliably generate spikes at specific points in time. Existing activitydependent plasticity rules like Spike-Timing-Dependent Plasticity are agnostic to the goal of learning spike times. On the other hand, the existing formal and supervised learning algorithms perform a temporally precise comparison of projected activity with the target, but there is no known biologically plausible implementation of this comparison. Here, we propose a simple and local unsupervised synaptic plasticity mechanism that is derived from the requirement of a balanced membrane potential. Since the relevant signal for synaptic change is the postsynaptic voltage rather than spike times, we call the plasticity rule Membrane Potential Dependent Plasticity (MPDP). Combining our plasticity mechanism with spike after-hyperpolarization causes a sensitivity of synaptic change to pre-and postsynaptic spike times which can reproduce Hebbian spike timing dependent plasticity for inhibitory synapses as was found in experiments. In addition, the sensitivity of MPDP to the time course of the voltage when generating a spike allows MPDP to distinguish between weak (spurious) and strong (teacher) spikes, which therefore provides a neuronal basis for the comparison of actual and target activity. For spatio-temporal input spike patterns our conceptually simple plasticity rule achieves a surprisingly high storage capacity for spike associations. The sensitivity of the MPDP to the subthreshold membrane potential during training allows robust memory retrieval after learning even in the presence of activity corrupted by noise. We propose that MPDP represents a biologically realistic mechanism to learn temporal target activity patterns.
Spatially and temporally local spike-timing-dependent plasticity rule
Proceedings of the International Joint Conference on Neural Networks, 2005
Recent neurophysiological research has focused on the temporal relationships between neuronal firing and plasticity, and has shown the phenomenon of spike-timing-dependent plasticity (STDP). Various models were suggested to implement the STDP-like learning rule in artificial networks based on spiking neuronal representations. Here we present and analyze a simple rule that only depends on the information that is available at the synapse at the time of synaptic modification. This rule is further extended by addition of four different types of gating derived from conventionally used types of gated decay in learning rules for continuous firing rate neural networks. The results show that the advantages of using these gatings are transferred to the new rule without sacrificing its dependency on spike-timing.
Neurocomputing, 2002
We provide a mathematical method to determine the distribution of synaptic strengths formed by any types of spike-timing-dependent plasticity (STDP). This becomes possible by applying the theory of Ornstein-Uhlenbeck process in determining the Fokker-Planck equation that characterizes the distribution. We verify our novel method by reproducing quantitative properties of STDP observed in previous simulation results. We apply our method to CA1-type window function and electric ÿsh-type window function to demonstrate possible implications of STDP. Moreover, we derive basic properties of STDP from our formalism. Especially, we determine the optimal window function for synaptic competition.
Spike-timing-dependent plasticity: common themes and divergent vistas
2002
Abstract. Recent experimental observations of spike-timing-dependent synaptic plasticity (STDP) have revitalized the study of synaptic learning rules. The most surprising aspect of these experiments lies in the observation that synapses activated shortly after the occurrence of a postsynaptic spike are weakened. Thus, synaptic plasticity is sensitive to the temporal ordering of pre-and postsynaptic activation. This temporal asymmetry has been suggested to underlie a range of learning tasks.
Temporal modulation of spike-timing-dependent plasticity
2010
Spike-timing-dependent plasticity (STDP) has attracted considerable experimental and theoretical attention over the last decade. In the most basic formulation, STDP provides a fundamental unit-a spike pair-for quantifying the induction of long-term changes in synaptic strength. However, many factors, both pre-and postsynaptic, can affect synaptic transmission and integration, especially when multiple spikes are considered. Here we review the experimental evidence for multiple types of nonlinear temporal interactions in STDP, focusing on the contributions of individual spike pairs, overall spike rate, and precise spike timing for modification of cortical and hippocampal excitatory synapses. We discuss the underlying processes that determine the specific learning rules at different synapses, such as postsynaptic excitability and short-term depression. Finally, we describe the success of efforts toward building predictive, quantitative models of how complex and natural spike trains induce long-term synaptic modifications.
An implementation of reinforcement learning based on spike timing dependent plasticity
Biological Cybernetics, 2008
An explanatory model is developed to show how synaptic learning mechanisms modeled through spike-timing dependent plasticity (STDP) can result in long-term adaptations consistent with reinforcement learning models. In particular, the reinforcement learning model known as temporal difference (TD) learning has been used to model neuronal behavior in the orbitofrontal cortex (OFC) and ventral tegmental area (VTA) of macaque monkey during reinforcement learning. While some research has observed, empirically, a connection between STDP and TD, there has not been an explanatory model directly connecting TD to STDP. Through analysis of the learning dynamics that results from a general form of a STDP learning rule, the connection between STDP and TD is explained. We further demonstrate that a STDP learning rule drives the spike probability of a reward predicting neuronal population to a stable equilibrium. The equilibrium solution has an increasing slope where the steepness of the slope predicts the probability of the reward, similar to the results from electrophysiological recordings suggesting a different slope that predicts the value of the anticipated reward of Montague and Berns ]. This connection begins to shed light into more recent data gathered from VTA and OFC which are not well modeled by TD. We suggest that STDP provides the underlying mechanism for explaining reinforcement learning and other higher level perceptual and cognitive function.
Neural Computation, 2004
Experimental evidence indicates that synaptic modification depends on the timing relationship between the presynaptic inputs and the output spikes that they generate. In this letter, results are presented for models of spike-timing-dependent plasticity (STDP) whose weight dynamics is determined by a stable fixed point. Four classes of STDP are identified on the basis of the time extent of their input-output interactions. The effect on the potentiation of synapses with different rates of input is investigated to elucidate the relationship of STDP with classical studies of long-term potentiation and depression and rate-based Hebbian learning. The selective potentiation of higher-rate synaptic inputs is found only for models where the time extent of the input-output interactions is input restricted (i.e., restricted to time domains delimited by adjacent synaptic inputs) and that have a time-asymmetric learning window with a longer time constant for depression than for potentiation. The analysis provides an account of learning dynamics determined by an input-selective stable fixed point. The effect of suppressive interspike interactions on STDP is also analyzed and shown to modify the synaptic dynamics.