Metrics That Learn Relevance (original) (raw)

Information-theoretic metric learning: 2-D linear projections of neural data for visualization

2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2013

Intracortical neural recordings are typically highdimensional due to many electrodes, channels, or units and high sampling rates, making it very difficult to visually inspect differences among responses to various conditions. By representing the neural response in a low-dimensional space, a researcher can visually evaluate the amount of information the response carries about the conditions. We consider a linear projection to 2-D space that also parametrizes a metric between neural responses. The projection, and corresponding metric, should preserve class-relevant information pertaining to different behavior or stimuli. We find the projection as a solution to the information-theoretic optimization problem of maximizing the information between the projected data and the class labels. The method is applied to two datasets using different types of neural responses: motor cortex neuronal firing rates of a macaque during a center-out reaching task, and local field potentials in the somatosensory cortex of a rat during tactile stimulation of the forepaw. In both cases, projected data points preserve the natural topology of targets or peripheral touch sites. Using the learned metric on the neural responses increases the nearest-neighbor classification rate versus the original data; thus, the metric is tuned to distinguish among the conditions.

Learning multiscale neural metrics via entropy minimization

2013 6th International IEEE/EMBS Conference on Neural Engineering (NER), 2013

In order to judiciously compare neural responses between repeated trials or stimuli, a well-suited distance metric is necessary. With multi-electrode recordings, a neural response is a spatiotemporal pattern, but not all of the dimensions of space and time should be treated equally. In order to understand which dimensions of the input are more discriminative and to improve the classification performance, we propose a metric-learning approach that can be used across scales. This extends previous work that used a linear projection into lower dimensional space; here, multiscale metrics or kernels are learned as the weighted combinations of different metrics or kernels on each of the neural response's dimensions. Preliminary results are explored on a cortical recording of a rat during a tactile stimulation experiment. Metrics on both local field potential and spiking data are explored. The learned weights reveal important dimensions of the response, and the learned metrics improve nearest-neighbor classification performance.

Bias Reduction and Metric Learning for Nearest-Neighbor Estimation of Kullback-Leibler Divergence

Neural computation, 2018

Nearest-neighbor estimators for the Kullback-Leiber (KL) divergence that are asymptotically unbiased have recently been proposed and demonstrated in a number of applications. However, with a small number of samples, nonparametric methods typically suffer from large estimation bias due to the nonlocality of information derived from nearest-neighbor statistics. In this letter, we show that this estimation bias can be mitigated by modifying the metric function, and we propose a novel method for learning a locally optimal Mahalanobis distance function from parametric generative models of the underlying density distributions. Using both simulations and experiments on a variety of data sets, we demonstrate that this interplay between approximate generative models and nonparametric techniques can significantly improve the accuracy of nearest-neighbor-based estimation of the KL divergence.

The Use of an Adaptive Distance Measure in Generalizing Pattern Learning

Starting from a situation where we want to infer a functional relationship between input and output patterns from a given set of examples (supervised generalization or prototype learning), we approach this problem by approximating the metric and consequently the topological space of the input patterns. In other words, we try to find a distance measure for the input patterns which is implicitly constrained by the function table given by the example set. The output pattern for every new input pattern within the range of a well-defined input pattern can be read off the network then (nearest-neighbor method), bis analysis, besides providing a novel way for the implementation of a weightless pattern ling network, offers a new look at the relation between training sets and the generallion behavior of an artificial neural network (such as back-propagation or LVQ). The fcscussion presented here is limited to binary patterns and pattern classification.

A Kernel-Based Calculation of Information on a Metric Space

Entropy, 2013

Kernel density estimation is a technique for approximating probability distributions. Here, it is applied to the calculation of mutual information on a metric space. This is motivated by the problem in neuroscience of calculating the mutual information between stimuli and spiking responses; the space of these responses is a metric space. It is shown that kernel density estimation on a metric space resembles the k-nearest-neighbor approach. This approach is applied to a toy dataset designed to mimic electrophysiological data.

A Metric for Evaluating Neural Input Representation in Supervised Learning Networks

Frontiers in Neuroscience, 2018

Supervised learning has long been attributed to several feed-forward neural circuits within the brain, with particular attention being paid to the cerebellar granular layer. The focus of this study is to evaluate the input activity representation of these feed-forward neural networks. The activity of cerebellar granule cells is conveyed by parallel fibers and translated into Purkinje cell activity, which constitutes the sole output of the cerebellar cortex. The learning process at this parallel-fiber-to-Purkinje-cell connection makes each Purkinje cell sensitive to a set of specific cerebellar states, which are roughly determined by the granule-cell activity during a certain time window. A Purkinje cell becomes sensitive to each neural input state and, consequently, the network operates as a function able to generate a desired output for each provided input by means of supervised learning. However, not all sets of Purkinje cell responses can be assigned to any set of input states due to the network's own limitations (inherent to the network neurobiological substrate), that is, not all input-output mapping can be learned. A key limiting factor is the representation of the input states through granule-cell activity. The quality of this representation (e.g., in terms of heterogeneity) will determine the capacity of the network to learn a varied set of outputs. Assessing the quality of this representation is interesting when developing and studying models of these networks to identify those neuron or network characteristics that enhance this representation. In this study we present an algorithm for evaluating quantitatively the level of compatibility/interference amongst a set of given cerebellar states according to their representation (granule-cell activation patterns) without the need for actually conducting simulations and network training. The algorithm input consists of a real-number matrix that codifies the activity level of every considered granule-cell in each state. The capability of this representation to generate a varied set of outputs is evaluated geometrically, thus resulting in a real number that assesses the goodness of the representation.

PARTIAL RETRAINING: A NEW APPROACH TO INPUT RELEVANCE DETERMINATION

International Journal of Neural Systems, 1999

In this article we introduce partial retraining, an algorithm to determine the relevance of the input variables of a trained neural network. We place this algorithm in the context of other approaches to relevance determination. Numerical experiments on both arti cial and real-world problems show that partial retraining outperforms its competitors, which include methods based on constant substitution, analysis of weight magnitudes, and \optimal brain surgeon".

Kernel-based distance metric learning in the output space

The 2013 International Joint Conference on Neural Networks (IJCNN), 2013

In this paper we present two related, kernel-based Distance Metric Learning (DML) methods. Their respective models non-linearly map data from their original space to an output space, and subsequent distance measurements are performed in the output space via a Mahalanobis metric. The dimensionality of the output space can be directly controlled to facilitate the learning of a low-rank metric. Both methods allow for simultaneous inference of the associated metric and the mapping to the output space, which can be used to visualize the data, when the output space is 2-or 3-dimensional. Experimental results for a collection of classification tasks illustrate the advantages of the proposed methods over other traditional and kernel-based DML approaches.