Christopher Gilliam - Academia.edu (original) (raw)

Motion Estimation using Local All-Pass Filters by Christopher Gilliam

IEEE International Conference on Image Processing (ICIP), Sep 2017

We propose an efficient method for image registration based on iteratively fitting a parametric m... more We propose an efficient method for image registration based on iteratively fitting a parametric model to the output of an elastic registration. It combines the flexibility of elastic registration able to estimate complex deformations-with the robustness of parametric registration-able to estimate very large displacement. Our approach is made feasible by using the recent Local All-Pass (LAP) algorithm; a fast and accurate filter-based method for estimating the local deformation between two images. Moreover, at each iteration we fit a linear parametric model to the local deformation which is equivalent to solving a linear system of equations (very fast and efficient). We use a quadratic polynomial model however the framework can easily be extended to more complicated models. The significant advantage of the proposed method is its robustness to model mis-match (e.g. noise and blurring). Experimental results on synthetic images and real images demonstrate that the proposed algorithm is highly accurate and outperforms a selection of image registration approaches .

IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Apr 2015

The optical flow is a velocity field that describes the motion of pixels within a sequence (or se... more The optical flow is a velocity field that describes the motion of pixels within a sequence (or set) of images. Its estimation plays an important role in areas such as motion compensation, object tracking and image registration. In this paper, we present a novel framework to estimate the optical flow using local all-pass filters. Instead of using the optical flow equation, the framework is based on relating one image to another, on a local level, using an all-pass filter and then extracting the optical flow from the filter. Using this framework, we present a fast novel algorithm for estimating a smoothly varying optical flow, which we term the Local All-Pass (LAP) algorithm. We demonstrate that this algorithm is consistent and accurate, and that it outperforms three state-of-the-art algorithms when estimating constant and smoothly varying flows. We also show initial competitive results for real images.

IEEE International Symposium on Biomedical Imaging, Apr 2016

Fast and accurate motion estimation is an important tool in biomedical imaging applications such ... more Fast and accurate motion estimation is an important tool in biomedical imaging applications such as motion compensation and image registration. In this paper, we present a novel algorithm to estimate motion in volumetric images based on the recently developed Local All-Pass (LAP) optical flow framework. The framework is built upon the idea that any motion can be regarded as a local rigid displacement
and is hence equivalent to all-pass filtering. Accordingly, our algorithm aims to relate two images, on a local level, using a 3D all-pass filter and then extract the local motion flow from the filter. As this process is based on filtering, it can be efficiently repeated over the whole image volume allowing fast estimation of a dense 3D
motion. We demonstrate the effectiveness of this algorithm on both synthetic motion flows and in-vivo MRI data involving respiratory motion. In particular, the algorithm obtains greater accuracy for significantly reduced computation time when compared to competing approaches.

IEEE International Conference on Image Processing (ICIP), Sep 2015

Accurate and robust spot tracking is a necessary tool for quantitative motion analysis in fluores... more Accurate and robust spot tracking is a necessary tool for quantitative motion analysis in fluorescence microscopy images. Few trackers however consider the underlying dynamics present in biological systems. For example, the collective motion of cells often exhibits both fast dynamics, i.e. Brownian motion, and slow dynamics, i.e. time-invariant stationary motion. In this paper, we propose a novel,
multi-frame, tracker that exploits this stationary motion. More precisely, we first estimate the stationary motion and then use it to guide the spot tracker. We obtain the stationary motion by adapting a recent optical flow algorithm that relates one image to another locally using an all-pass filter. We perform this operation over all the image frames simultaneously and estimate a single, stationary optical flow. We compare the proposed tracker with two existing techniques and show that our approach is more robust to high noise and varying structure. In addition, we also show initial experiments on real microscopy images.

IEEE International Conference on Image Processing (ICIP), Sep 2015

Estimating the displacements between two images is often addressed using a small displacement ass... more Estimating the displacements between two images is often addressed using a small displacement assumption, which leads to what is known as the optical flow equation. We study the quality of the underlying approximation for the recently developed Local All-Pass (LAP) optical flow algorithm, which is based on another approach—displacements result from filtering. While the simplest version of LAP computes only first-order differences, we show that the order of LAP approximation
is quadratic, unlike standard optical flow equation based algorithms for which this approximation is only linear. More generally, the order of approximation of the
LAP algorithm is twice larger than the differentiation order involved. The key step in the derivation is the use of Pad´e approximants.

Finite Rate of Innovation by Christopher Gilliam

IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Mar 2016

Recently, sampling theory has been broadened to include a class of non-bandlimited signals that p... more Recently, sampling theory has been broadened to include a class of non-bandlimited signals that possess finite rate of innovation (FRI). In this paper, we consider the problem of determining the minimum rate of innovation (RI) in a noisy setting. First, we adapt a recent
model-fitting algorithm for FRI recovery and demonstrate that it achieves the Cram´er-Rao bounds. Using this algorithm, we then present a framework to estimate the minimum RI based on fitting the sparsest model to the noisy samples whilst satisfying a mean squared
error (MSE) criterion - a signal is recovered if the output MSE is less than the input MSE. Specifically, given a RI, we use the MSE criterion to judge whether our model-fitting has been a success or a failure. Using this output, we present a Dichotomic algorithm that performs a binary search for the minimum RI and demonstrate that it obtains a sparser RI estimate than an existing information criterion approach.

IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), May 2014

Recently, classical sampling theory has been broadened to include a class of non-bandlimited sign... more Recently, classical sampling theory has been broadened to include a class of non-bandlimited signals that possess finite rate of innovation (FRI). In this paper we consider the reconstruction of a periodic stream of Diracs from noisy samples. We demonstrate that its noiseless FRI samples can be represented as a ratio of two polynomials. Using this structure as a model, we propose recovering the FRI signal using a model fitting approach rather than an annihilation method. We present an algorithm that fits this model to the noisy samples and demonstrate that it has low computation cost and is more reliable than two state-of-the-art methods.

IEEE Transactions on Signal Processing, Nov 2015

Finite rate of innovation (FRI) is a recent framework for sampling and reconstruction of a large ... more Finite rate of innovation (FRI) is a recent framework for sampling and reconstruction of a large class of parametric signals that are characterized by finite number of innovations (parameters) per unit interval. In the absence of noise, exact recovery of FRI signals has been demonstrated. In the noisy scenario, there exist techniques to deal with non-ideal measurements. Yet, the accuracy and resiliency to noise and model mismatch are still challenging problems for real-world applications. We address the reconstruction of FRI signals, specifically a stream of Diracs, from few signal samples degraded by noise and we propose a new FRI reconstruction method that is based on a model—fitting approach related to the structured—TLS problem. The model—fitting method is based on minimizing the training error, that is, the error between the computed and the recovered moments (i.e., the FRI-samples of the signal), subject to an annihilation system. We present our framework for three different constraints of the annihilation system. Moreover, we propose a model order selection framework to determine the innovation rate of the signal; i.e., the number of Diracs by estimating the noise level through the training error curve. We compare the performance of the model—fitting approach with known FRI reconstruction algorithms and Cramér–Rao’s lower bound (CRLB) to validate these contributions.

Plenoptic Sampling by Christopher Gilliam

Emerging Technologies for 3D Video: Creation, Coding, Transmission and Rendering, 2013

ABSTRACT Image-based rendering (IBR) is a technique for producing arbitrary views of a scene usin... more ABSTRACT Image-based rendering (IBR) is a technique for producing arbitrary views of a scene using multiple images instead of exact object models. The central concept is that each image comprises a collection of light rays and a new view is interpolated from these light rays. If we modelled the light rays using a seven-dimensional function, known as the plenoptic function, then IBR can be viewed in terms of sampling and reconstruction. Therefore the important goal of minimizing the number of images required in IBR, whilst maintaining rendering quality, can be examined through sampling analysis of the plenoptic function. In this context, the chapter examines the state of the art in plenoptic sampling theory. It focuses on both uniform and adaptive sampling of the plenoptic function. In particular, it presents theoretical results for uniform sampling based on spectral analysis of the plenoptic function and algorithms for adaptive plenoptic sampling.

The plenoptic function is a powerful tool to analyze the properties of multi-view image datasets.... more The plenoptic function is a powerful tool to analyze the properties of multi-view image datasets. In particular, the understanding of the spectral properties of the plenoptic function is essential in many computer vision applications including image-based rendering. In this paper, we derive for the first time an exact closed-form expression of the plenoptic spectrum of a slanted plane with finite width and use this expression as the elementary building block to derive the plenoptic spectrum of more sophisticated scenes. This is achieved by approximating the geometry of the scene with a set of slanted planes and evaluating the closed-form expression for each plane in the set. We then use this closed-form expression to revisit uniform plenoptic sampling. In this context, we derive a new Nyquist rate for the plenoptic sampling of a slanted plane and a new reconstruction filter. Through numerical simulations, on both real and synthetic scenes, we show that the new filter outperforms alternative existing filters.

IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Mar 2012

Image based rendering is a technique for producing arbitrary viewpoints of a scene using multiple... more Image based rendering is a technique for producing arbitrary viewpoints of a scene using multiple images instead of exact object models. The recent emergence of low-price, fast, and reliable cameras for measuring depth makes possible the augmentation of traditional color images with depth images. This combination promises to improve the rendering quality of an arbitrary viewpoint and thus have a great impact on IBR. A key issue is to understand, for any particular scene of interest, how many depth images and how many color images are necessary in order to obtain good rendering results. In this paper, using a framework akin to the plenoptic function, we perform a spectral analysis of multi-view depth images in order to determine the relationship between the number of depth and color images required. Our analysis is then validated using both synthetic and real images.

IEEE International Conference on Image Processing (ICIP), Sep 2011

The plenoptic function enables Image-based rendering (IBR) to be viewed in terms of sampling and ... more The plenoptic function enables Image-based rendering (IBR) to be viewed in terms of sampling and reconstruction. Thus the spatial sampling rate can be determined through spectral analysis of the plenoptic function. In this paper we present a method of nonuniformly sampling a scene, with a smoothly varying surface, given a finite number of samples. This method approximates such a scene with a set of slanted planes subject to the constraint of finite number of samples. We use the recent spectral analysis of a single slanted plane to determine a piecewise constant spatial sampling rate for the scene. Finally, we show that this sampling rate results in a nonuniform sampling scheme that reconstructs the plenoptic function beyond that of uniform sampling.

IEEE International Conference on Image Processing (ICIP), Sep 2010

The plenoptic function enables Image-based rendering (IBR) to be viewed in terms of sampling and ... more The plenoptic function enables Image-based rendering (IBR) to be
viewed in terms of sampling and reconstruction. Thus the spatial sampling rate can be determined through spectral analysis of the plenoptic function. In this paper we examine the bandwidth of the plenoptic function when both the field of view and the scene width are finite. This analysis is carried out on two planar Lambertian scenes, a fronto-parallel plane and a slanted plane, and in both cases the texture is bandlimited. We derive an exact closed-form expression for the plenoptic spectrum of a slanted plane with sinusoidal texture. We show that in both cases the finite constraints lead to band-unlimited spectra. By determining the essential bandwidth, we derive a sampling curve that gives an adequate camera spacing for a given distance between the scene and the camera line.

IEEE International Conference on Image Processing (ICIP), Sep 2017

We propose an efficient method for image registration based on iteratively fitting a parametric m... more We propose an efficient method for image registration based on iteratively fitting a parametric model to the output of an elastic registration. It combines the flexibility of elastic registration able to estimate complex deformations-with the robustness of parametric registration-able to estimate very large displacement. Our approach is made feasible by using the recent Local All-Pass (LAP) algorithm; a fast and accurate filter-based method for estimating the local deformation between two images. Moreover, at each iteration we fit a linear parametric model to the local deformation which is equivalent to solving a linear system of equations (very fast and efficient). We use a quadratic polynomial model however the framework can easily be extended to more complicated models. The significant advantage of the proposed method is its robustness to model mis-match (e.g. noise and blurring). Experimental results on synthetic images and real images demonstrate that the proposed algorithm is highly accurate and outperforms a selection of image registration approaches .

IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Apr 2015

The optical flow is a velocity field that describes the motion of pixels within a sequence (or se... more The optical flow is a velocity field that describes the motion of pixels within a sequence (or set) of images. Its estimation plays an important role in areas such as motion compensation, object tracking and image registration. In this paper, we present a novel framework to estimate the optical flow using local all-pass filters. Instead of using the optical flow equation, the framework is based on relating one image to another, on a local level, using an all-pass filter and then extracting the optical flow from the filter. Using this framework, we present a fast novel algorithm for estimating a smoothly varying optical flow, which we term the Local All-Pass (LAP) algorithm. We demonstrate that this algorithm is consistent and accurate, and that it outperforms three state-of-the-art algorithms when estimating constant and smoothly varying flows. We also show initial competitive results for real images.

IEEE International Symposium on Biomedical Imaging, Apr 2016

Fast and accurate motion estimation is an important tool in biomedical imaging applications such ... more Fast and accurate motion estimation is an important tool in biomedical imaging applications such as motion compensation and image registration. In this paper, we present a novel algorithm to estimate motion in volumetric images based on the recently developed Local All-Pass (LAP) optical flow framework. The framework is built upon the idea that any motion can be regarded as a local rigid displacement
and is hence equivalent to all-pass filtering. Accordingly, our algorithm aims to relate two images, on a local level, using a 3D all-pass filter and then extract the local motion flow from the filter. As this process is based on filtering, it can be efficiently repeated over the whole image volume allowing fast estimation of a dense 3D
motion. We demonstrate the effectiveness of this algorithm on both synthetic motion flows and in-vivo MRI data involving respiratory motion. In particular, the algorithm obtains greater accuracy for significantly reduced computation time when compared to competing approaches.

IEEE International Conference on Image Processing (ICIP), Sep 2015

Accurate and robust spot tracking is a necessary tool for quantitative motion analysis in fluores... more Accurate and robust spot tracking is a necessary tool for quantitative motion analysis in fluorescence microscopy images. Few trackers however consider the underlying dynamics present in biological systems. For example, the collective motion of cells often exhibits both fast dynamics, i.e. Brownian motion, and slow dynamics, i.e. time-invariant stationary motion. In this paper, we propose a novel,
multi-frame, tracker that exploits this stationary motion. More precisely, we first estimate the stationary motion and then use it to guide the spot tracker. We obtain the stationary motion by adapting a recent optical flow algorithm that relates one image to another locally using an all-pass filter. We perform this operation over all the image frames simultaneously and estimate a single, stationary optical flow. We compare the proposed tracker with two existing techniques and show that our approach is more robust to high noise and varying structure. In addition, we also show initial experiments on real microscopy images.

IEEE International Conference on Image Processing (ICIP), Sep 2015

Estimating the displacements between two images is often addressed using a small displacement ass... more Estimating the displacements between two images is often addressed using a small displacement assumption, which leads to what is known as the optical flow equation. We study the quality of the underlying approximation for the recently developed Local All-Pass (LAP) optical flow algorithm, which is based on another approach—displacements result from filtering. While the simplest version of LAP computes only first-order differences, we show that the order of LAP approximation
is quadratic, unlike standard optical flow equation based algorithms for which this approximation is only linear. More generally, the order of approximation of the
LAP algorithm is twice larger than the differentiation order involved. The key step in the derivation is the use of Pad´e approximants.

IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Mar 2016

Recently, sampling theory has been broadened to include a class of non-bandlimited signals that p... more Recently, sampling theory has been broadened to include a class of non-bandlimited signals that possess finite rate of innovation (FRI). In this paper, we consider the problem of determining the minimum rate of innovation (RI) in a noisy setting. First, we adapt a recent
model-fitting algorithm for FRI recovery and demonstrate that it achieves the Cram´er-Rao bounds. Using this algorithm, we then present a framework to estimate the minimum RI based on fitting the sparsest model to the noisy samples whilst satisfying a mean squared
error (MSE) criterion - a signal is recovered if the output MSE is less than the input MSE. Specifically, given a RI, we use the MSE criterion to judge whether our model-fitting has been a success or a failure. Using this output, we present a Dichotomic algorithm that performs a binary search for the minimum RI and demonstrate that it obtains a sparser RI estimate than an existing information criterion approach.

IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), May 2014

Recently, classical sampling theory has been broadened to include a class of non-bandlimited sign... more Recently, classical sampling theory has been broadened to include a class of non-bandlimited signals that possess finite rate of innovation (FRI). In this paper we consider the reconstruction of a periodic stream of Diracs from noisy samples. We demonstrate that its noiseless FRI samples can be represented as a ratio of two polynomials. Using this structure as a model, we propose recovering the FRI signal using a model fitting approach rather than an annihilation method. We present an algorithm that fits this model to the noisy samples and demonstrate that it has low computation cost and is more reliable than two state-of-the-art methods.

IEEE Transactions on Signal Processing, Nov 2015

Finite rate of innovation (FRI) is a recent framework for sampling and reconstruction of a large ... more Finite rate of innovation (FRI) is a recent framework for sampling and reconstruction of a large class of parametric signals that are characterized by finite number of innovations (parameters) per unit interval. In the absence of noise, exact recovery of FRI signals has been demonstrated. In the noisy scenario, there exist techniques to deal with non-ideal measurements. Yet, the accuracy and resiliency to noise and model mismatch are still challenging problems for real-world applications. We address the reconstruction of FRI signals, specifically a stream of Diracs, from few signal samples degraded by noise and we propose a new FRI reconstruction method that is based on a model—fitting approach related to the structured—TLS problem. The model—fitting method is based on minimizing the training error, that is, the error between the computed and the recovered moments (i.e., the FRI-samples of the signal), subject to an annihilation system. We present our framework for three different constraints of the annihilation system. Moreover, we propose a model order selection framework to determine the innovation rate of the signal; i.e., the number of Diracs by estimating the noise level through the training error curve. We compare the performance of the model—fitting approach with known FRI reconstruction algorithms and Cramér–Rao’s lower bound (CRLB) to validate these contributions.

Emerging Technologies for 3D Video: Creation, Coding, Transmission and Rendering, 2013

ABSTRACT Image-based rendering (IBR) is a technique for producing arbitrary views of a scene usin... more ABSTRACT Image-based rendering (IBR) is a technique for producing arbitrary views of a scene using multiple images instead of exact object models. The central concept is that each image comprises a collection of light rays and a new view is interpolated from these light rays. If we modelled the light rays using a seven-dimensional function, known as the plenoptic function, then IBR can be viewed in terms of sampling and reconstruction. Therefore the important goal of minimizing the number of images required in IBR, whilst maintaining rendering quality, can be examined through sampling analysis of the plenoptic function. In this context, the chapter examines the state of the art in plenoptic sampling theory. It focuses on both uniform and adaptive sampling of the plenoptic function. In particular, it presents theoretical results for uniform sampling based on spectral analysis of the plenoptic function and algorithms for adaptive plenoptic sampling.

The plenoptic function is a powerful tool to analyze the properties of multi-view image datasets.... more The plenoptic function is a powerful tool to analyze the properties of multi-view image datasets. In particular, the understanding of the spectral properties of the plenoptic function is essential in many computer vision applications including image-based rendering. In this paper, we derive for the first time an exact closed-form expression of the plenoptic spectrum of a slanted plane with finite width and use this expression as the elementary building block to derive the plenoptic spectrum of more sophisticated scenes. This is achieved by approximating the geometry of the scene with a set of slanted planes and evaluating the closed-form expression for each plane in the set. We then use this closed-form expression to revisit uniform plenoptic sampling. In this context, we derive a new Nyquist rate for the plenoptic sampling of a slanted plane and a new reconstruction filter. Through numerical simulations, on both real and synthetic scenes, we show that the new filter outperforms alternative existing filters.

IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Mar 2012

Image based rendering is a technique for producing arbitrary viewpoints of a scene using multiple... more Image based rendering is a technique for producing arbitrary viewpoints of a scene using multiple images instead of exact object models. The recent emergence of low-price, fast, and reliable cameras for measuring depth makes possible the augmentation of traditional color images with depth images. This combination promises to improve the rendering quality of an arbitrary viewpoint and thus have a great impact on IBR. A key issue is to understand, for any particular scene of interest, how many depth images and how many color images are necessary in order to obtain good rendering results. In this paper, using a framework akin to the plenoptic function, we perform a spectral analysis of multi-view depth images in order to determine the relationship between the number of depth and color images required. Our analysis is then validated using both synthetic and real images.

IEEE International Conference on Image Processing (ICIP), Sep 2011

The plenoptic function enables Image-based rendering (IBR) to be viewed in terms of sampling and ... more The plenoptic function enables Image-based rendering (IBR) to be viewed in terms of sampling and reconstruction. Thus the spatial sampling rate can be determined through spectral analysis of the plenoptic function. In this paper we present a method of nonuniformly sampling a scene, with a smoothly varying surface, given a finite number of samples. This method approximates such a scene with a set of slanted planes subject to the constraint of finite number of samples. We use the recent spectral analysis of a single slanted plane to determine a piecewise constant spatial sampling rate for the scene. Finally, we show that this sampling rate results in a nonuniform sampling scheme that reconstructs the plenoptic function beyond that of uniform sampling.

IEEE International Conference on Image Processing (ICIP), Sep 2010

The plenoptic function enables Image-based rendering (IBR) to be viewed in terms of sampling and ... more The plenoptic function enables Image-based rendering (IBR) to be
viewed in terms of sampling and reconstruction. Thus the spatial sampling rate can be determined through spectral analysis of the plenoptic function. In this paper we examine the bandwidth of the plenoptic function when both the field of view and the scene width are finite. This analysis is carried out on two planar Lambertian scenes, a fronto-parallel plane and a slanted plane, and in both cases the texture is bandlimited. We derive an exact closed-form expression for the plenoptic spectrum of a slanted plane with sinusoidal texture. We show that in both cases the finite constraints lead to band-unlimited spectra. By determining the essential bandwidth, we derive a sampling curve that gives an adequate camera spacing for a given distance between the scene and the camera line.