Jan Lellmann | University of Cambridge (original) (raw)
Papers by Jan Lellmann
International Journal of Computer Vision, 2008
Inferring scene geometry from a sequence of camera images is one of the central problems in compu... more Inferring scene geometry from a sequence of camera images is one of the central problems in computer vision. While the overwhelming majority of related research focuses on diffuse surface models, there are cases when this is not a viable assumption: in many industrial applications, one has to deal with metal or coated surfaces exhibiting a strong specular behavior. We propose a novel and generalized constrained gradient descent method to determine the shape of a purely specular object from the reflection of a calibrated scene and additional data required to find a unique solution. This data is exemplarily provided by optical flow measurements obtained by small scale motion of the specular object, with camera and scene remaining stationary. We present a non-approximative general forward model to predict the optical flow of specular surfaces, covering rigid body motion as well as elastic deformation, and allowing for a characterization of problematic points. We demonstrate the applicability of our method by numerical experiments on synthetic and real data.
The saddle point framework provides a convenient way to formulate many convex variational problem... more The saddle point framework provides a convenient way to formulate many convex variational problems that occur in computer vision. The framework unifies a broad range of data and regularization terms, and is particularly suited for nonsmooth problems such as Total Variation-based approaches to image labeling. However, for many interesting problems the constraint sets involved are difficult to handle numerically. State-of-the-art methods rely on using nested iterative projections, which induces both theoretical and practical convergence issues. We present a dual multiple-constraint Douglas-Rachford splitting approach that is globally convergent, avoids inner iterative loops, enforces the constraints exactly, and requires only basic operations that can be easily parallelized. The method outperforms existing methods by a factor of 4 − 20 while considerably increasing the numerical robustness.
Optimization Methods & Software, 2012
We present the Convex Optimization Algorithms Library (COAL), a flexible C++framework for modelli... more We present the Convex Optimization Algorithms Library (COAL), a flexible C++framework for modelling and solving convex optimization problems in connection with variational problems of image analysis. COAL connects solver implementations with specific models via an extensible set of properties, without enforcing a specific standard form. This allows to exploit the problem structure and to handle large-scale problems while supporting rapid prototyping and modifications of the model. Based on predefined building blocks, a broad range of functionals encountered in image analysis can be implemented and be reliably optimized using state-of-the-art algorithms, without the need to know algorithmic details. We demonstrate the use of COAL on four representative variational problems of image analysis.
We introduce a linearly weighted variant of the total variation for vector fields in order to for... more We introduce a linearly weighted variant of the total variation for vector fields in order to formulate regularizers for multi-class labeling problems with non-trivial interclass distances. We characterize the possible distances, show that Euclidean distances can be exactly represented, and review some methods to approximate non-Euclidean distances in order to define novel total variation based regularizers. We show that the convex relaxed problem can be efficiently optimized to a prescribed accuracy with optimality certificates using Nesterov's method, and evaluate and compare our approach on several synthetical and real-world examples.
Computing Research Repository, 2011
We study convex relaxations of the image labeling problem on a continuous domain with regularizer... more We study convex relaxations of the image labeling problem on a continuous domain with regularizers based on metric interaction potentials. The generic framework ensures existence of minimizers and covers a wide range of relaxations of the originally combinatorial problem. We focus on two specific relaxations that differ in flexibility and simplicity -- one can be used to tightly relax any metric interaction potential, while the other one only covers Euclidean metrics but requires less computational effort. For solving the nonsmooth discretized problem, we propose a globally convergent Douglas-Rachford scheme, and show that a sequence of dual iterates can be recovered in order to provide a posteriori optimality bounds. In a quantitative comparison to two other first-order methods, the approach shows competitive performance on synthetical and real-world images. By combining the method with an improved binarization technique for nonstandard potentials, we were able to routinely recover discrete solutions within 1%--5% of the global optimum for the combinatorial image labeling problem.
Multi-class labeling is one of the core problems in image analysis. We show how this combinatoria... more Multi-class labeling is one of the core problems in image analysis. We show how this combinatorial problem can be approximately solved using tools from convex optimization. We suggest a novel functional based on a multidimensional total variation formulation, allowing for a broad range of data terms. Optimization is carried out in the operator splitting framework using Douglas-Rachford Splitting. In this connection, we compare two methods to solve the Rudin-Osher-Fatemi type subproblems and demonstrate the performance of our approach on single- and multichannel images.
Variational relaxations can be used to compute approximate minimizers of optimal partitioning and... more Variational relaxations can be used to compute approximate minimizers of optimal partitioning and multiclass labeling problems on continuous domains. While the resulting relaxed convex problem can be solved globally optimal, in order to obtain a discrete solution a rounding step is required, which may increase the objective and lead to suboptimal solutions. We analyze a probabilistic rounding method and prove that it allows to obtain discrete solutions with an a priori upper bound on the objective, ensuring the quality of the result from the viewpoint of optimization. We show that the approach can be interpreted as an approximate, multiclass variant of the coarea formula.
International Journal of Computer Vision, 2008
Inferring scene geometry from a sequence of camera images is one of the central problems in compu... more Inferring scene geometry from a sequence of camera images is one of the central problems in computer vision. While the overwhelming majority of related research focuses on diffuse surface models, there are cases when this is not a viable assumption: in many industrial applications, one has to deal with metal or coated surfaces exhibiting a strong specular behavior. We propose a novel and generalized constrained gradient descent method to determine the shape of a purely specular object from the reflection of a calibrated scene and additional data required to find a unique solution. This data is exemplarily provided by optical flow measurements obtained by small scale motion of the specular object, with camera and scene remaining stationary. We present a non-approximative general forward model to predict the optical flow of specular surfaces, covering rigid body motion as well as elastic deformation, and allowing for a characterization of problematic points. We demonstrate the applicability of our method by numerical experiments on synthetic and real data.
The saddle point framework provides a convenient way to formulate many convex variational problem... more The saddle point framework provides a convenient way to formulate many convex variational problems that occur in computer vision. The framework unifies a broad range of data and regularization terms, and is particularly suited for nonsmooth problems such as Total Variation-based approaches to image labeling. However, for many interesting problems the constraint sets involved are difficult to handle numerically. State-of-the-art methods rely on using nested iterative projections, which induces both theoretical and practical convergence issues. We present a dual multiple-constraint Douglas-Rachford splitting approach that is globally convergent, avoids inner iterative loops, enforces the constraints exactly, and requires only basic operations that can be easily parallelized. The method outperforms existing methods by a factor of 4 − 20 while considerably increasing the numerical robustness.
Optimization Methods & Software, 2012
We present the Convex Optimization Algorithms Library (COAL), a flexible C++framework for modelli... more We present the Convex Optimization Algorithms Library (COAL), a flexible C++framework for modelling and solving convex optimization problems in connection with variational problems of image analysis. COAL connects solver implementations with specific models via an extensible set of properties, without enforcing a specific standard form. This allows to exploit the problem structure and to handle large-scale problems while supporting rapid prototyping and modifications of the model. Based on predefined building blocks, a broad range of functionals encountered in image analysis can be implemented and be reliably optimized using state-of-the-art algorithms, without the need to know algorithmic details. We demonstrate the use of COAL on four representative variational problems of image analysis.
We introduce a linearly weighted variant of the total variation for vector fields in order to for... more We introduce a linearly weighted variant of the total variation for vector fields in order to formulate regularizers for multi-class labeling problems with non-trivial interclass distances. We characterize the possible distances, show that Euclidean distances can be exactly represented, and review some methods to approximate non-Euclidean distances in order to define novel total variation based regularizers. We show that the convex relaxed problem can be efficiently optimized to a prescribed accuracy with optimality certificates using Nesterov's method, and evaluate and compare our approach on several synthetical and real-world examples.
Computing Research Repository, 2011
We study convex relaxations of the image labeling problem on a continuous domain with regularizer... more We study convex relaxations of the image labeling problem on a continuous domain with regularizers based on metric interaction potentials. The generic framework ensures existence of minimizers and covers a wide range of relaxations of the originally combinatorial problem. We focus on two specific relaxations that differ in flexibility and simplicity -- one can be used to tightly relax any metric interaction potential, while the other one only covers Euclidean metrics but requires less computational effort. For solving the nonsmooth discretized problem, we propose a globally convergent Douglas-Rachford scheme, and show that a sequence of dual iterates can be recovered in order to provide a posteriori optimality bounds. In a quantitative comparison to two other first-order methods, the approach shows competitive performance on synthetical and real-world images. By combining the method with an improved binarization technique for nonstandard potentials, we were able to routinely recover discrete solutions within 1%--5% of the global optimum for the combinatorial image labeling problem.
Multi-class labeling is one of the core problems in image analysis. We show how this combinatoria... more Multi-class labeling is one of the core problems in image analysis. We show how this combinatorial problem can be approximately solved using tools from convex optimization. We suggest a novel functional based on a multidimensional total variation formulation, allowing for a broad range of data terms. Optimization is carried out in the operator splitting framework using Douglas-Rachford Splitting. In this connection, we compare two methods to solve the Rudin-Osher-Fatemi type subproblems and demonstrate the performance of our approach on single- and multichannel images.
Variational relaxations can be used to compute approximate minimizers of optimal partitioning and... more Variational relaxations can be used to compute approximate minimizers of optimal partitioning and multiclass labeling problems on continuous domains. While the resulting relaxed convex problem can be solved globally optimal, in order to obtain a discrete solution a rounding step is required, which may increase the objective and lead to suboptimal solutions. We analyze a probabilistic rounding method and prove that it allows to obtain discrete solutions with an a priori upper bound on the objective, ensuring the quality of the result from the viewpoint of optimization. We show that the approach can be interpreted as an approximate, multiclass variant of the coarea formula.