Unsupervised Learning of Invariant Representations in Hierarchical Architectures (original) (raw)
Abstract:The present phase of Machine Learning is characterized by supervised learning algorithms relying on large sets of labeled examples ($n \to \infty$). The next phase is likely to focus on algorithms capable of learning from very few labeled examples ($n \to 1$), like humans seem able to do. We propose an approach to this problem and describe the underlying theory, based on the unsupervised, automatic learning of a ``good'' representation for supervised learning, characterized by small sample complexity ($n$). We consider the case of visual object recognition though the theory applies to other domains. The starting point is the conjecture, proved in specific cases, that image representations which are invariant to translations, scaling and other transformations can considerably reduce the sample complexity of learning. We prove that an invariant and unique (discriminative) signature can be computed for each image patch, III, in terms of empirical distributions of the dot-products between III and a set of templates stored during unsupervised learning. A module performing filtering and pooling, like the simple and complex cells described by Hubel and Wiesel, can compute such estimates. Hierarchical architectures consisting of this basic Hubel-Wiesel moduli inherit its properties of invariance, stability, and discriminability while capturing the compositional organization of the visual world in terms of wholes and parts. The theory extends existing deep learning convolutional architectures for image and speech recognition. It also suggests that the main computational goal of the ventral stream of visual cortex is to provide a hierarchical representation of new objects/images which is invariant to transformations, stable, and discriminative for recognition---and that this representation may be continuously learned in an unsupervised way during development and visual experience.
Submission history
From: Fabio Anselmi [view email]
[v1] Sun, 17 Nov 2013 13:22:44 UTC (6,820 KB)
[v2] Thu, 21 Nov 2013 10:41:41 UTC (6,820 KB)
[v3] Wed, 18 Dec 2013 20:57:24 UTC (6,020 KB)
[v4] Sun, 12 Jan 2014 04:06:29 UTC (3,260 KB)
[v5] Tue, 11 Mar 2014 19:56:59 UTC (3,787 KB)