Nonlinear multilayered sequence models (original) (raw)

Abstract

Feature discovery is a fundamental problem, for the performance of any learning algorithm depends on the features it is given. In this thesis we focus on the problem of feature discovery for sequential data, such as video or speech, which is important for many practical problems, including that of constructing an intelligent agent. The hidden variables of probabilistic models that are fitted to the data often act as useful features for discrimination tasks that are not known during learning. We introduce several powerful probabilistic sequence models with efficient learning and inference algorithms that can learn accurate models of highly complex, high-dimensional sequence data and produce hierarchical features.