Thomas Lampert | Université de Strasbourg (original) (raw)
Papers by Thomas Lampert
Although agreement between annotators who mark feature locations within images has been studied i... more Although agreement between annotators who mark feature locations within images has been studied in the past from a statistical viewpoint, little work has attempted to quantify the extent to which this phenomenon affects the evaluation of foreground-background segmentation algorithms. Many researchers utilise ground truth in experimentation and more often than not this ground truth is derived from one annotator's opinion. How does the difference in opinion affect an algorithm's evaluation? A methodology is applied to four image processing problems to quantify the inter-annotator variance and to offer insight into the mechanisms behind agreement and the use of ground truth. It is found that when detecting linear structures annotator agreement is very low. The agreement in a structure's position can be partially explained through basic image properties. Automatic segmentation algorithms are compared to annotator agreement and it is found that there is a clear relation between the two. Several ground truth estimation methods are used to infer a number of algorithm performances. It is found that: the rank of a detector is highly dependent upon the method used to form the ground truth; and that although STAPLE and LSML appear to represent the mean of the performance measured using individual annotations, when there are few annotations, or there is a large variance in them, these estimates tend to degrade. Furthermore, one of the most commonly adopted combination methods---consensus voting---accentuates more obvious features, resulting in an overestimation of performance. It is concluded that in some datasets it is not possible to confidently infer an algorithm ranking when evaluating upon one ground truth.
The automatic identification of sleep stages in electroencephalography (EEG) time-series is a lon... more The automatic identification of sleep stages in electroencephalography (EEG) time-series is a long desired goal for researchers concerned with the study of sleep disorders. This paper presents advances towards achieving this goal, with particular application to EEG time-series recorded from mice. Approaches in the literature apply supervised learning classifiers, however, these do not reach the performance levels required for use within a laboratory. In this paper, detection reliability is increased, most notably in the case of REM stage identification, by naturally decomposing the problem and applying a support vector machine (SVM) based classifier to each of the EEG channels. Their outputs are integrated within a multiple classifier system. Furthermore, there exists no general consensus on the ideal choice of parameter values in such systems. Therefore, an investigation into the effects upon the classification performance is presented by varying parameters such as: the epoch length; features size; number of training samples; and the method for calculating the power spectral density estimate. Finally, the results of these investigations are brought together to demonstrate the performance of the proposed classification algorithm in two cases: intra-animal classification and inter-animal classification. It is shown that, within a dataset of ten EEG recordings, and using less than one percent of an EEG as training data, a mean classification error of Awake 6.45%, NREM 5.82%, and REM 6.65% (with standard deviations less than 0.6%) is achieved in intra-animal analysis and, when using the equivalent of seven percent of one EEG as training data, Awake 10.19%, NREM 7.75%, and REM 17.43% is achieved in inter-animal analysis (with mean standard deviations of 6.42%, 2.89%, and 9.69% respectively). A software package implementing the proposed approach will be made available through Cybula Ltd.
While a problem's skew is often assumed to be constant, this paper discusses three settings where... more While a problem's skew is often assumed to be constant, this paper discusses three settings where this assumption does not hold. Consequently, incorrectly assuming skew to be constant in these contradicting cases results in an over or under estimation of an algorithm's performance. The area under a precision-recall curve (AUCPR) is a common summary measurement used to report the performance of machine learning algorithms. It is well known that precision is dependent upon class skew, which often varies between datasets. In addition to this, it is demonstrated herein that under certain circumstances the relative ranking of an algorithm (as measured by AUCPR) is not constant and is instead also dependent upon skew. The skew at which the performance of two algorithms inverts and the relationship between precision measured at different skews are defined. This is extended to account for temporal skew characteristics and situations in which skew cannot be precisely defined. Formal proofs for these findings are presented, desirable properties are proved and their application demonstrated.
This paper proposes an active contour algorithm for spectrogram track detection. It extends upon ... more This paper proposes an active contour algorithm for spectrogram track detection. It extends upon previously published work in a number of areas, previously published internal and potential energy models are refined and theoretical motivations for these changes are offered. These refinements offer a marked improvement in detection performance, including a notable reduction in the probability of false positive detections. The result is feature extraction at signal-to-noise ratios as low as− 1dB in the frequency domain.
Being the first stage of analysis within an image, low-level feature detection (edge and line det... more Being the first stage of analysis within an image, low-level feature detection (edge and line detection) is a crucial step in the image analysis process and, as such, deserves suitable attention. This paper presents a systematic investigation into low-level feature detection in spectrogram images. The result of which is the identification of frequency tracks. Analysis of the literature identifies different strategies for accomplishing low-level feature detection. Nevertheless, the advantages and disadvantages of each are not explicitly investigated. Three model based detection strategies are outlined, each extracting an increasing amount of information from the spectrogram, and, through ROC analysis, it is shown that at increasing levels of extraction the detection rates increase. Nevertheless, further investigation suggests that model based detection has a limitation---it is not computationally feasible to fully evaluate the model of even a simple sinusoidal track. Therefore, alternative approaches, such as dimensionality reduction, are investigated to reduce the complex search space. It is shown that, if carefully selected, these techniques can approach the detection rates of model based strategies that perform the same level of information extraction.
This paper proposes an active contour framework for spectrogram track detection. A potential ener... more This paper proposes an active contour framework for spectrogram track detection. A potential energy is proposed which results in feature extraction at a signal-to-noise ratio (SNR) of 0.5 dB. We show, through complexity analysis, that this is achievable in real-time.
The detection of tracks in spectrograms is an important step in remote sensing applications such ... more The detection of tracks in spectrograms is an important step in remote sensing applications such as the analysis of marine mammal calls and remote sensing data in underwater environments. Recent advances in technology and the abundance of data requires the development of more sensitive detection methods. This problem has attracted researchers’ interest from a variety of backgrounds ranging between image processing, signal processing, simulated annealing and Bayesian filtering. Most of the literature is concentrated in three areas: image processing, neural networks, and statistical models such as the Hidden Markov model. There has not been a review paper which describes and critically analyses the application of these key algorithms. This paper presents an extensive survey and an algorithm taxonomy, additionally each algorithm is reviewed according to a set of criteria relating to their success in application. These criteria are defined to be their ability to cope with noise variation over time, track association, high variability in track shape, closely separated tracks, multiple tracks, the birth/death of tracks, low signal-to-noise ratios, that they have no a priori assumption of track shape and that they are computationally cheap. Our analysis concludes that none of these algorithms fully meets these criteria.
Pattern Recognition, 2008. ICPR …, Jan 1, 2008
Computer Recognition Systems 3, Jan 1, 2009
Typically, acoustic data received via passive sonar systems in underwater environments is transfo... more Typically, acoustic data received via passive sonar systems in underwater environments is transformed into the frequency domain using the Short-Time Fourier Transform. This allows for the construction of a spectrogram image in which time and frequency are the axes and intensity represents the power at a particular time and frequency. It follows from this that if a (stationary or non-stationary) periodic component is present during some consecutive time frames a track or line will be present within the spectrogram. The problem of automatic ...
2009 Advanced Video and …, Jan 1, 2009
Computer Recognition Systems 3, Jan 1, 2009
Although agreement between annotators who mark feature locations within images has been studied i... more Although agreement between annotators who mark feature locations within images has been studied in the past from a statistical viewpoint, little work has attempted to quantify the extent to which this phenomenon affects the evaluation of foreground-background segmentation algorithms. Many researchers utilise ground truth in experimentation and more often than not this ground truth is derived from one annotator's opinion. How does the difference in opinion affect an algorithm's evaluation? A methodology is applied to four image processing problems to quantify the inter-annotator variance and to offer insight into the mechanisms behind agreement and the use of ground truth. It is found that when detecting linear structures annotator agreement is very low. The agreement in a structure's position can be partially explained through basic image properties. Automatic segmentation algorithms are compared to annotator agreement and it is found that there is a clear relation between the two. Several ground truth estimation methods are used to infer a number of algorithm performances. It is found that: the rank of a detector is highly dependent upon the method used to form the ground truth; and that although STAPLE and LSML appear to represent the mean of the performance measured using individual annotations, when there are few annotations, or there is a large variance in them, these estimates tend to degrade. Furthermore, one of the most commonly adopted combination methods---consensus voting---accentuates more obvious features, resulting in an overestimation of performance. It is concluded that in some datasets it is not possible to confidently infer an algorithm ranking when evaluating upon one ground truth.
The automatic identification of sleep stages in electroencephalography (EEG) time-series is a lon... more The automatic identification of sleep stages in electroencephalography (EEG) time-series is a long desired goal for researchers concerned with the study of sleep disorders. This paper presents advances towards achieving this goal, with particular application to EEG time-series recorded from mice. Approaches in the literature apply supervised learning classifiers, however, these do not reach the performance levels required for use within a laboratory. In this paper, detection reliability is increased, most notably in the case of REM stage identification, by naturally decomposing the problem and applying a support vector machine (SVM) based classifier to each of the EEG channels. Their outputs are integrated within a multiple classifier system. Furthermore, there exists no general consensus on the ideal choice of parameter values in such systems. Therefore, an investigation into the effects upon the classification performance is presented by varying parameters such as: the epoch length; features size; number of training samples; and the method for calculating the power spectral density estimate. Finally, the results of these investigations are brought together to demonstrate the performance of the proposed classification algorithm in two cases: intra-animal classification and inter-animal classification. It is shown that, within a dataset of ten EEG recordings, and using less than one percent of an EEG as training data, a mean classification error of Awake 6.45%, NREM 5.82%, and REM 6.65% (with standard deviations less than 0.6%) is achieved in intra-animal analysis and, when using the equivalent of seven percent of one EEG as training data, Awake 10.19%, NREM 7.75%, and REM 17.43% is achieved in inter-animal analysis (with mean standard deviations of 6.42%, 2.89%, and 9.69% respectively). A software package implementing the proposed approach will be made available through Cybula Ltd.
While a problem's skew is often assumed to be constant, this paper discusses three settings where... more While a problem's skew is often assumed to be constant, this paper discusses three settings where this assumption does not hold. Consequently, incorrectly assuming skew to be constant in these contradicting cases results in an over or under estimation of an algorithm's performance. The area under a precision-recall curve (AUCPR) is a common summary measurement used to report the performance of machine learning algorithms. It is well known that precision is dependent upon class skew, which often varies between datasets. In addition to this, it is demonstrated herein that under certain circumstances the relative ranking of an algorithm (as measured by AUCPR) is not constant and is instead also dependent upon skew. The skew at which the performance of two algorithms inverts and the relationship between precision measured at different skews are defined. This is extended to account for temporal skew characteristics and situations in which skew cannot be precisely defined. Formal proofs for these findings are presented, desirable properties are proved and their application demonstrated.
This paper proposes an active contour algorithm for spectrogram track detection. It extends upon ... more This paper proposes an active contour algorithm for spectrogram track detection. It extends upon previously published work in a number of areas, previously published internal and potential energy models are refined and theoretical motivations for these changes are offered. These refinements offer a marked improvement in detection performance, including a notable reduction in the probability of false positive detections. The result is feature extraction at signal-to-noise ratios as low as− 1dB in the frequency domain.
Being the first stage of analysis within an image, low-level feature detection (edge and line det... more Being the first stage of analysis within an image, low-level feature detection (edge and line detection) is a crucial step in the image analysis process and, as such, deserves suitable attention. This paper presents a systematic investigation into low-level feature detection in spectrogram images. The result of which is the identification of frequency tracks. Analysis of the literature identifies different strategies for accomplishing low-level feature detection. Nevertheless, the advantages and disadvantages of each are not explicitly investigated. Three model based detection strategies are outlined, each extracting an increasing amount of information from the spectrogram, and, through ROC analysis, it is shown that at increasing levels of extraction the detection rates increase. Nevertheless, further investigation suggests that model based detection has a limitation---it is not computationally feasible to fully evaluate the model of even a simple sinusoidal track. Therefore, alternative approaches, such as dimensionality reduction, are investigated to reduce the complex search space. It is shown that, if carefully selected, these techniques can approach the detection rates of model based strategies that perform the same level of information extraction.
This paper proposes an active contour framework for spectrogram track detection. A potential ener... more This paper proposes an active contour framework for spectrogram track detection. A potential energy is proposed which results in feature extraction at a signal-to-noise ratio (SNR) of 0.5 dB. We show, through complexity analysis, that this is achievable in real-time.
The detection of tracks in spectrograms is an important step in remote sensing applications such ... more The detection of tracks in spectrograms is an important step in remote sensing applications such as the analysis of marine mammal calls and remote sensing data in underwater environments. Recent advances in technology and the abundance of data requires the development of more sensitive detection methods. This problem has attracted researchers’ interest from a variety of backgrounds ranging between image processing, signal processing, simulated annealing and Bayesian filtering. Most of the literature is concentrated in three areas: image processing, neural networks, and statistical models such as the Hidden Markov model. There has not been a review paper which describes and critically analyses the application of these key algorithms. This paper presents an extensive survey and an algorithm taxonomy, additionally each algorithm is reviewed according to a set of criteria relating to their success in application. These criteria are defined to be their ability to cope with noise variation over time, track association, high variability in track shape, closely separated tracks, multiple tracks, the birth/death of tracks, low signal-to-noise ratios, that they have no a priori assumption of track shape and that they are computationally cheap. Our analysis concludes that none of these algorithms fully meets these criteria.
Pattern Recognition, 2008. ICPR …, Jan 1, 2008
Computer Recognition Systems 3, Jan 1, 2009
Typically, acoustic data received via passive sonar systems in underwater environments is transfo... more Typically, acoustic data received via passive sonar systems in underwater environments is transformed into the frequency domain using the Short-Time Fourier Transform. This allows for the construction of a spectrogram image in which time and frequency are the axes and intensity represents the power at a particular time and frequency. It follows from this that if a (stationary or non-stationary) periodic component is present during some consecutive time frames a track or line will be present within the spectrogram. The problem of automatic ...
2009 Advanced Video and …, Jan 1, 2009
Computer Recognition Systems 3, Jan 1, 2009