Gert Cauwenberghs | University of California, San Diego (original) (raw)
Uploads
Papers by Gert Cauwenberghs
Frontiers in neuroscience, 2018
Embedded, continual learning for autonomous and adaptive behavior is a key application of neuromo... more Embedded, continual learning for autonomous and adaptive behavior is a key application of neuromorphic hardware. However, neuromorphic implementations of embedded learning at large scales that are both flexible and efficient have been hindered by a lack of a suitable algorithmic framework. As a result, most neuromorphic hardware are trained off-line on large clusters of dedicated processors or GPUs and transferred to the device. We address this by introducing the neural and synaptic array transceiver (NSAT), a neuromorphic computational framework facilitating flexible and efficient embedded learning by matching algorithmic requirements and neural and synaptic dynamics. NSAT supports event-driven supervised, unsupervised and reinforcement learning algorithms including deep learning. We demonstrate the NSAT in a wide range of tasks, including the simulation of Mihalas-Niebur neuron, dynamic neural fields, event-driven random back-propagation for event-based deep learning, event-based ...
Frontiers in neuroscience, 2018
Error backpropagation is a highly effective mechanism for learning high-quality hierarchical feat... more Error backpropagation is a highly effective mechanism for learning high-quality hierarchical features in deep networks. Updating the features or weights in one layer, however, requires waiting for the propagation of error signals from higher layers. Learning using delayed and non-local errors makes it hard to reconcile backpropagation with the learning mechanisms observed in biological neural networks as it requires the neurons to maintain a memory of the input long enough until the higher-layer errors arrive. In this paper, we propose an alternative learning mechanism where errors are generated locally in each layer using fixed, random auxiliary classifiers. Lower layers could thus be trained independently of higher layers and training could either proceed layer by layer, or simultaneously in all layers using local error information. We address biological plausibility concerns such as weight symmetry requirements and show that the proposed learning mechanism based on fixed, broad, ...
Nature biomedical engineering, 2017
Advanced capabilities in electrical recording are essential for the treatment of heart-rhythm dis... more Advanced capabilities in electrical recording are essential for the treatment of heart-rhythm diseases. The most advanced technologies use flexible integrated electronics; however, the penetration of biological fluids into the underlying electronics and any ensuing electrochemical reactions pose significant safety risks. Here, we show that an ultrathin, leakage-free, biocompatible dielectric layer can completely seal an underlying layer of flexible electronics while allowing for electrophysiological measurements through capacitive coupling between tissue and the electronics, and thus without the need for direct metal contact. The resulting current-leakage levels and operational lifetimes are, respectively, four orders of magnitude smaller and between two and three orders of magnitude longer than those of any other flexible-electronics technology. Systematic electrophysiological studies with normal, paced and arrhythmic conditions in Langendorff hearts highlight the capabilities of t...
Frontiers in neuroscience, 2017
Quantification of dynamic causal interactions among brain regions constitutes an important compon... more Quantification of dynamic causal interactions among brain regions constitutes an important component of conducting research and developing applications in experimental and translational neuroscience. Furthermore, cortical networks with dynamic causal connectivity in brain-computer interface (BCI) applications offer a more comprehensive view of brain states implicated in behavior than do individual brain regions. However, models of cortical network dynamics are difficult to generalize across subjects because current electroencephalography (EEG) signal analysis techniques are limited in their ability to reliably localize sources across subjects. We propose an algorithmic and computational framework for identifying cortical networks across subjects in which dynamic causal connectivity is modeled among user-selected cortical regions of interest (ROIs). We demonstrate the strength of the proposed framework using a "reach/saccade to spatial target" cognitive task performed by 10...
Frontiers in neuroscience, 2016
Recent studies have shown that synaptic unreliability is a robust and sufficient mechanism for in... more Recent studies have shown that synaptic unreliability is a robust and sufficient mechanism for inducing the stochasticity observed in cortex. Here, we introduce Synaptic Sampling Machines (S2Ms), a class of neural network models that uses synaptic stochasticity as a means to Monte Carlo sampling and unsupervised learning. Similar to the original formulation of Boltzmann machines, these models can be viewed as a stochastic counterpart of Hopfield networks, but where stochasticity is induced by a random mask over the connections. Synaptic stochasticity plays the dual role of an efficient mechanism for sampling, and a regularizer during learning akin to DropConnect. A local synaptic plasticity rule implementing an event-driven form of contrastive divergence enables the learning of generative models in an on-line fashion. S2Ms perform equally well using discrete-timed artificial units (as in Hopfield networks) or continuous-timed leaky integrate and fire neurons. The learned representat...
IEEE transactions on circuits and systems. I, Regular papers : a publication of the IEEE Circuits and Systems Society, 2016
A parallel micro-power mixed-signal VLSI implementation of independent component analysis (ICA) w... more A parallel micro-power mixed-signal VLSI implementation of independent component analysis (ICA) with reconfigurable outer-product learning rules is presented. With the gradient sensing of the acoustic field over a miniature microphone array as a pre-processing method, the proposed ICA implementation can separate and localize up to 3 sources in mild reverberant environment. The ICA processor is implemented in 0.5 µm CMOS technology and occupies 3 mm × 3 mm area. At 16 kHz sampling rate, ASIC consumes 195 µW power from a 3 V supply. The outer-product implementation of natural gradient and Herault-Jutten ICA update rules demonstrates comparable performance to benchmark FastICA algorithm in ideal conditions and more robust performance in noisy and reverberant environment. Experiments demonstrate perceptually clear separation and precise localization over wide range of separation angles of two speech sources presented through speakers positioned at 1.5 m from the array on a conference ro...
international symposium on circuits and systems, May 31, 1998
Conference proceedings: ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference
Frontiers in neuroscience, 2015
2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2014
2011 IEEE Biomedical Circuits and Systems Conference (BioCAS), 2011
Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference, 2014
The planning of goal-directed movement towards targets in different parts of space is an importan... more The planning of goal-directed movement towards targets in different parts of space is an important function of the brain. Such visuo-motor planning and execution is known to involve multiple brain regions, including visual, parietal, and frontal cortices. To understand how these brain regions work together to both plan and execute goal-directed movement, it is essential to describe the dynamic causal interactions among them. Here we model causal interactions of distributed cortical source activity derived from non-invasively recorded EEG, using a combination of ICA, minimum-norm distributed source localization (cLORETA), and dynamical modeling within the Source Information Flow Toolbox (SIFT). We differentiate network causal connectivity of reach planning and execution, by comparing the causal network in a speeded reaching task with that for a control task not requiring goal-directed movement. Analysis of a pilot dataset (n=5) shows the utility of this technique and reveals increase...
Annals of Biomedical Engineering, 2014
IEEE Transactions on Circuits and Systems I: Regular Papers, 2010
IEEE Transactions on Audio, Speech and Language Processing, 2007
Frontiers in Neuroscience, 2014
Home; About; Submit; Advertise &a... more Home; About; Submit; Advertise & PR. Register; Login. Science: Genetics: Applied Genetic Epidemiology; Behavioral and Psychiatric Genetics; Bioinformatics and Computational Biology; Epigenomics; Evolutionary and Genomic Microbiology; ...
Frontiers in neuroscience, 2018
Embedded, continual learning for autonomous and adaptive behavior is a key application of neuromo... more Embedded, continual learning for autonomous and adaptive behavior is a key application of neuromorphic hardware. However, neuromorphic implementations of embedded learning at large scales that are both flexible and efficient have been hindered by a lack of a suitable algorithmic framework. As a result, most neuromorphic hardware are trained off-line on large clusters of dedicated processors or GPUs and transferred to the device. We address this by introducing the neural and synaptic array transceiver (NSAT), a neuromorphic computational framework facilitating flexible and efficient embedded learning by matching algorithmic requirements and neural and synaptic dynamics. NSAT supports event-driven supervised, unsupervised and reinforcement learning algorithms including deep learning. We demonstrate the NSAT in a wide range of tasks, including the simulation of Mihalas-Niebur neuron, dynamic neural fields, event-driven random back-propagation for event-based deep learning, event-based ...
Frontiers in neuroscience, 2018
Error backpropagation is a highly effective mechanism for learning high-quality hierarchical feat... more Error backpropagation is a highly effective mechanism for learning high-quality hierarchical features in deep networks. Updating the features or weights in one layer, however, requires waiting for the propagation of error signals from higher layers. Learning using delayed and non-local errors makes it hard to reconcile backpropagation with the learning mechanisms observed in biological neural networks as it requires the neurons to maintain a memory of the input long enough until the higher-layer errors arrive. In this paper, we propose an alternative learning mechanism where errors are generated locally in each layer using fixed, random auxiliary classifiers. Lower layers could thus be trained independently of higher layers and training could either proceed layer by layer, or simultaneously in all layers using local error information. We address biological plausibility concerns such as weight symmetry requirements and show that the proposed learning mechanism based on fixed, broad, ...
Nature biomedical engineering, 2017
Advanced capabilities in electrical recording are essential for the treatment of heart-rhythm dis... more Advanced capabilities in electrical recording are essential for the treatment of heart-rhythm diseases. The most advanced technologies use flexible integrated electronics; however, the penetration of biological fluids into the underlying electronics and any ensuing electrochemical reactions pose significant safety risks. Here, we show that an ultrathin, leakage-free, biocompatible dielectric layer can completely seal an underlying layer of flexible electronics while allowing for electrophysiological measurements through capacitive coupling between tissue and the electronics, and thus without the need for direct metal contact. The resulting current-leakage levels and operational lifetimes are, respectively, four orders of magnitude smaller and between two and three orders of magnitude longer than those of any other flexible-electronics technology. Systematic electrophysiological studies with normal, paced and arrhythmic conditions in Langendorff hearts highlight the capabilities of t...
Frontiers in neuroscience, 2017
Quantification of dynamic causal interactions among brain regions constitutes an important compon... more Quantification of dynamic causal interactions among brain regions constitutes an important component of conducting research and developing applications in experimental and translational neuroscience. Furthermore, cortical networks with dynamic causal connectivity in brain-computer interface (BCI) applications offer a more comprehensive view of brain states implicated in behavior than do individual brain regions. However, models of cortical network dynamics are difficult to generalize across subjects because current electroencephalography (EEG) signal analysis techniques are limited in their ability to reliably localize sources across subjects. We propose an algorithmic and computational framework for identifying cortical networks across subjects in which dynamic causal connectivity is modeled among user-selected cortical regions of interest (ROIs). We demonstrate the strength of the proposed framework using a "reach/saccade to spatial target" cognitive task performed by 10...
Frontiers in neuroscience, 2016
Recent studies have shown that synaptic unreliability is a robust and sufficient mechanism for in... more Recent studies have shown that synaptic unreliability is a robust and sufficient mechanism for inducing the stochasticity observed in cortex. Here, we introduce Synaptic Sampling Machines (S2Ms), a class of neural network models that uses synaptic stochasticity as a means to Monte Carlo sampling and unsupervised learning. Similar to the original formulation of Boltzmann machines, these models can be viewed as a stochastic counterpart of Hopfield networks, but where stochasticity is induced by a random mask over the connections. Synaptic stochasticity plays the dual role of an efficient mechanism for sampling, and a regularizer during learning akin to DropConnect. A local synaptic plasticity rule implementing an event-driven form of contrastive divergence enables the learning of generative models in an on-line fashion. S2Ms perform equally well using discrete-timed artificial units (as in Hopfield networks) or continuous-timed leaky integrate and fire neurons. The learned representat...
IEEE transactions on circuits and systems. I, Regular papers : a publication of the IEEE Circuits and Systems Society, 2016
A parallel micro-power mixed-signal VLSI implementation of independent component analysis (ICA) w... more A parallel micro-power mixed-signal VLSI implementation of independent component analysis (ICA) with reconfigurable outer-product learning rules is presented. With the gradient sensing of the acoustic field over a miniature microphone array as a pre-processing method, the proposed ICA implementation can separate and localize up to 3 sources in mild reverberant environment. The ICA processor is implemented in 0.5 µm CMOS technology and occupies 3 mm × 3 mm area. At 16 kHz sampling rate, ASIC consumes 195 µW power from a 3 V supply. The outer-product implementation of natural gradient and Herault-Jutten ICA update rules demonstrates comparable performance to benchmark FastICA algorithm in ideal conditions and more robust performance in noisy and reverberant environment. Experiments demonstrate perceptually clear separation and precise localization over wide range of separation angles of two speech sources presented through speakers positioned at 1.5 m from the array on a conference ro...
international symposium on circuits and systems, May 31, 1998
Conference proceedings: ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference
Frontiers in neuroscience, 2015
2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2014
2011 IEEE Biomedical Circuits and Systems Conference (BioCAS), 2011
Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference, 2014
The planning of goal-directed movement towards targets in different parts of space is an importan... more The planning of goal-directed movement towards targets in different parts of space is an important function of the brain. Such visuo-motor planning and execution is known to involve multiple brain regions, including visual, parietal, and frontal cortices. To understand how these brain regions work together to both plan and execute goal-directed movement, it is essential to describe the dynamic causal interactions among them. Here we model causal interactions of distributed cortical source activity derived from non-invasively recorded EEG, using a combination of ICA, minimum-norm distributed source localization (cLORETA), and dynamical modeling within the Source Information Flow Toolbox (SIFT). We differentiate network causal connectivity of reach planning and execution, by comparing the causal network in a speeded reaching task with that for a control task not requiring goal-directed movement. Analysis of a pilot dataset (n=5) shows the utility of this technique and reveals increase...
Annals of Biomedical Engineering, 2014
IEEE Transactions on Circuits and Systems I: Regular Papers, 2010
IEEE Transactions on Audio, Speech and Language Processing, 2007
Frontiers in Neuroscience, 2014
Home; About; Submit; Advertise &a... more Home; About; Submit; Advertise & PR. Register; Login. Science: Genetics: Applied Genetic Epidemiology; Behavioral and Psychiatric Genetics; Bioinformatics and Computational Biology; Epigenomics; Evolutionary and Genomic Microbiology; ...