Xavier Clady | University Pierre and Marie Curie (original) (raw)

Papers by Xavier Clady

doi: 10.3389/fnins.2015.00046 Spatiotemporal features for asynchronous event-based data

HAL (Le Centre pour la Communication Scientifique Directe), 2002

International audienceno abstrac

HAL (Le Centre pour la Communication Scientifique Directe), 2006

HAL (Le Centre pour la Communication Scientifique Directe), Apr 1, 2012

L'objet de cet article est de présenter le projet MIRAS: Robot avec interaction multimodale pour ... more L'objet de cet article est de présenter le projet MIRAS: Robot avec interaction multimodale pour l'assistance à la déambulation. Les objectifs de ce projet sont exposés dans un premier temps ; puis chaque axe de travail est détaillé : description des scenarii et fonctions multimodales associées, présentation du travail de conception et de réalisation du robot, synthèse de l'étude de sécurité et enfin présentation des travaux menés pour la validation clinique. Les résultats des premiers essais pilotes sont également analysés pour souligner les points d'amélioration. Enfin, une discussion sur l'intérêt de l'étude et ses problématiques est menée avant de conclure l'article.

Lecture Notes in Computer Science, Aug 12, 2008

This paper presents a framework for multiclass vehicle type (Make and Model) identification based... more This paper presents a framework for multiclass vehicle type (Make and Model) identification based on oriented contour points. A method to construct a model from several frontal vehicle images is presented. Employing this model, three voting algorithms and a distance error allows to measure the similarity between an input instance and the data bases classes. These scores could be combined to design a discriminant function. We present too a second classification stage that employ scores like vectors. A nearest-neighbor algorithm is used to determine the vehicle type. This method have been tested on a realistic data set (830 images containing 50 different vehicle classes) obtaining similar results for equivalent recognition frameworks with different features selections [12]. The system also shows to be robust to partial occlusions.

IEEE transactions on neural networks and learning systems, Dec 1, 2015

Object tracking is an important step in many artificial vision tasks. The current state-of-the-ar... more Object tracking is an important step in many artificial vision tasks. The current state-of-the-art implementations remain too computationally demanding for the problem to be solved in real time with high dynamics. This paper presents a novel real-time method for visual part-based tracking of complex objects from the output of an asynchronous event-based camera. This paper extends the pictorial structures model introduced by Fischler and Elschlager 40 years ago and introduces a new formulation of the problem, allowing the dynamic processing of visual input in real time at high temporal resolution using a conventional PC. It relies on the concept of representing an object as a set of basic elements linked by springs. These basic elements consist of simple trackers capable of successfully tracking a target with an ellipse-like shape at several kilohertz on a conventional computer. For each incoming event, the method updates the elastic connections established between the trackers and guarantees a desired geometric structure corresponding to the tracked object in real time. This introduces a high temporal elasticity to adapt to projective deformations of the tracked object in the focal plane. The elastic energy of this virtual mechanical system provides a quality criterion for tracking and can be used to determine whether the measured deformations are caused by the perspective projection of the perceived object or by occlusions. Experiments on real-world data show the robustness of the method in the context of dynamic face tracking.

HAL (Le Centre pour la Communication Scientifique Directe), 2011

In this communication, we present an algorithm for the onboard vision vehicle detection problem, ... more In this communication, we present an algorithm for the onboard vision vehicle detection problem, using the attentional cascade architecture proposed by Viola and Jones in 2001[23]. Three families of features are compared : the rectangular filters (Haar’s features), the histograms of oriented gradient (Histograms of Gradients, HoG) and their combination (a concatenation of the two preceding features). A comparative study of the results of the generative (HoG features), discriminative (Haar’s features) d etectors, and of their fusion, is presented. These results sh ow that the last detector combines the advantages of the two

Frontiers in Neuroscience, 2017

This paper introduces an event-based luminance-free feature from the output of asynchronous event... more This paper introduces an event-based luminance-free feature from the output of asynchronous event-based neuromorphic retinas. The feature consists in mapping the distribution of the optical flow along the contours of the moving objects in the visual scene into a matrix. Asynchronous event-based neuromorphic retinas are composed of autonomous pixels, each of them asynchronously generating "spiking" events that encode relative changes in pixels' illumination at high temporal resolutions. The optical flow is computed at each event, and is integrated locally or globally in a speed and direction coordinate frame based grid, using speed-tuned temporal kernels. The latter ensures that the resulting feature equitably represents the distribution of the normal motion along the current moving edges, whatever their respective dynamics. The usefulness and the generality of the proposed feature are demonstrated in pattern recognition applications: local corner detection and global gesture recognition.

Neural computation, Jan 20, 2015

This letter introduces a study to precisely measure what an increase in spike timing precision ca... more This letter introduces a study to precisely measure what an increase in spike timing precision can add to spike-driven pattern recognition algorithms. The concept of generating spikes from images by converting gray levels into spike timings is currently at the basis of almost every spike-based modeling of biological visual systems. The use of images naturally leads to generating incorrect artificial and redundant spike timings and, more important, also contradicts biological findings indicating that visual processing is massively parallel, asynchronous with high temporal resolution. A new concept for acquiring visual information through pixel-individual asynchronous level-crossing sampling has been proposed in a recent generation of asynchronous neuromorphic visual sensors. Unlike conventional cameras, these sensors acquire data not at fixed points in time for the entire array but at fixed amplitude changes of their input, resulting optimally sparse in space and time-pixel individua...

Frontiers in psychology, 2018

The kinematics of hand movements (spatial use, curvature, acceleration, and velocity) of infants ... more The kinematics of hand movements (spatial use, curvature, acceleration, and velocity) of infants with their mothers in an interactive setting are significantly associated with age in cohorts of typical and at-risk infantsdiffer significantly at 5-6 months of age, depending on the context: relating either with an object or a person.Environmental and developmental factors shape the developmental trajectories of hand movements in different cohorts: environment for infants with VIMs; stage of development for premature infants and those with West syndrome; and both factors for infants with orality disorders.The curvature of hand movements specifically reflects atypical development in infants with West syndrome when developmental age is considered. We aimed to discriminate between typical and atypical developmental trajectory patterns of at-risk infants in an interactive setting in this observational and longitudinal study, with the assumption that hand movements (HM) reflect preverbal co...

-Nous présentons dans cet article, un nouvel algorithme de traitement d'images, la "Transformée C... more -Nous présentons dans cet article, un nouvel algorithme de traitement d'images, la "Transformée Chinoise", permettant d'estimer la localisation des doigts d'une main. Cette approche utilise une technique inspirée de la Transformée de Hough qui prend en compte la disposition des pixels de contour ainsi que l'orientation du gradient en ces pixels. Elle a été intégrée dans un système d'acquisition visuelle monoculaire des gestes humains de préhension.

IEEE Transactions on Neural Networks and Learning Systems

This paper introduces an event-based luminancefree algorithm for line and segment detection from ... more This paper introduces an event-based luminancefree algorithm for line and segment detection from the output of asynchronous event-based neuromorphic retinas. These recent biomimetic vision sensors are composed of autonomous pixels, each of them asynchronously generating visual events that encode relative changes in pixels' illumination at high temporal resolutions. This frame-free approach results in an increased energy efficiency and in real-time operation, making these sensors especially suitable for applications such as autonomous robotics. The proposed algorithm is based on an iterative event-based weighted least squares fitting, and it is consequently well suited to the high temporal resolution and asynchronous acquisition of neuromorphic cameras: parameters of a current line are updated for each event attributed (i.e., spatio-temporally close) to it, while implicitly forgetting the contribution of older events according to a speed-tuned exponentially decaying function. A detection occurs if a measure of activity, i.e., implicit measure of the number of contributing events and using the same decay function, exceeds a given threshold. The speed-tuned decreasing function is based on a measure of the apparent motion, i.e., the optical flow computed around each event. This latter ensures that the algorithm behaves independently of the edges' dynamics. Line segments are then extracted from the lines, allowing for the tracking of the corresponding endpoints. We provide experiments showing the accuracy of our algorithm and study the influence of the apparent velocity and relative orientation of the observed edges. Finally, evaluations of its computational efficiency show that this algorithm can be envisioned for high-speed applications, such as vision-based robotic navigation.

This paper introduces an event-based luminance-free feature from the output of asynchronous event... more This paper introduces an event-based luminance-free feature from the output of asynchronous event-based neuromorphic retinas. The feature consists in mapping the distribution of the optical flow along the contours of the moving objects in the visual scene into a matrix. Asynchronous event-based neuromorphic retinas are composed of autonomous pixels, each of them asynchronously generating " spiking " events that encode relative changes in pixels' illumination at high temporal resolutions. The optical flow is computed at each event, and is integrated locally or globally in a speed and direction coordinate frame based grid, using speed-tuned temporal kernels. The latter ensures that the resulting feature equitably represents the distribution of the normal motion along the current moving edges, whatever their respective dynamics. The usefulness and the generality of the proposed feature are demonstrated in pattern recognition applications: local corner detection and global gesture recognition.

In this paper, visual perception in car driving assistance i s c onsidered. The work concerns the... more In this paper, visual perception in car driving assistance i s c onsidered. The work concerns the development of a system combining a Pan-Tilt-Zoom (PTZ) camera and a standard c amera, in order to track front vehicles. The standard c amera has a small focal length, and is devoted to the analyze of the whole frontal scene. Here, the PTZ camera is used t o t r ack the closest vehicle. An original approach to control it is integrated. Camera r otations and zoom are c ontrolled by visual servoing and by an eecient real time target tracking algorithm. The aim of this work is to keep the rear view image of target vehicle stable in scale and position. The methods presented h e r e were tested o n r eal road sequences within the VELAC 1 demonstration vehicle. Experimental results show the eeectiveness of such an approach.

ACM SIGGRAPH 2006 Research posters on - SIGGRAPH '06, 2006

This communication describes a system to capture paintings in progress. An adapted algorithm base... more This communication describes a system to capture paintings in progress. An adapted algorithm based on statistical foreground substraction is used to mask the artist hand and his tools in image sequences. This allows a better understanding of the artist gesture and his mind.

doi: 10.3389/fnins.2015.00046 Spatiotemporal features for asynchronous event-based data

HAL (Le Centre pour la Communication Scientifique Directe), 2002

International audienceno abstrac

HAL (Le Centre pour la Communication Scientifique Directe), 2006

HAL (Le Centre pour la Communication Scientifique Directe), Apr 1, 2012

L'objet de cet article est de présenter le projet MIRAS: Robot avec interaction multimodale pour ... more L'objet de cet article est de présenter le projet MIRAS: Robot avec interaction multimodale pour l'assistance à la déambulation. Les objectifs de ce projet sont exposés dans un premier temps ; puis chaque axe de travail est détaillé : description des scenarii et fonctions multimodales associées, présentation du travail de conception et de réalisation du robot, synthèse de l'étude de sécurité et enfin présentation des travaux menés pour la validation clinique. Les résultats des premiers essais pilotes sont également analysés pour souligner les points d'amélioration. Enfin, une discussion sur l'intérêt de l'étude et ses problématiques est menée avant de conclure l'article.

Lecture Notes in Computer Science, Aug 12, 2008

This paper presents a framework for multiclass vehicle type (Make and Model) identification based... more This paper presents a framework for multiclass vehicle type (Make and Model) identification based on oriented contour points. A method to construct a model from several frontal vehicle images is presented. Employing this model, three voting algorithms and a distance error allows to measure the similarity between an input instance and the data bases classes. These scores could be combined to design a discriminant function. We present too a second classification stage that employ scores like vectors. A nearest-neighbor algorithm is used to determine the vehicle type. This method have been tested on a realistic data set (830 images containing 50 different vehicle classes) obtaining similar results for equivalent recognition frameworks with different features selections [12]. The system also shows to be robust to partial occlusions.

IEEE transactions on neural networks and learning systems, Dec 1, 2015

Object tracking is an important step in many artificial vision tasks. The current state-of-the-ar... more Object tracking is an important step in many artificial vision tasks. The current state-of-the-art implementations remain too computationally demanding for the problem to be solved in real time with high dynamics. This paper presents a novel real-time method for visual part-based tracking of complex objects from the output of an asynchronous event-based camera. This paper extends the pictorial structures model introduced by Fischler and Elschlager 40 years ago and introduces a new formulation of the problem, allowing the dynamic processing of visual input in real time at high temporal resolution using a conventional PC. It relies on the concept of representing an object as a set of basic elements linked by springs. These basic elements consist of simple trackers capable of successfully tracking a target with an ellipse-like shape at several kilohertz on a conventional computer. For each incoming event, the method updates the elastic connections established between the trackers and guarantees a desired geometric structure corresponding to the tracked object in real time. This introduces a high temporal elasticity to adapt to projective deformations of the tracked object in the focal plane. The elastic energy of this virtual mechanical system provides a quality criterion for tracking and can be used to determine whether the measured deformations are caused by the perspective projection of the perceived object or by occlusions. Experiments on real-world data show the robustness of the method in the context of dynamic face tracking.

HAL (Le Centre pour la Communication Scientifique Directe), 2011

In this communication, we present an algorithm for the onboard vision vehicle detection problem, ... more In this communication, we present an algorithm for the onboard vision vehicle detection problem, using the attentional cascade architecture proposed by Viola and Jones in 2001[23]. Three families of features are compared : the rectangular filters (Haar’s features), the histograms of oriented gradient (Histograms of Gradients, HoG) and their combination (a concatenation of the two preceding features). A comparative study of the results of the generative (HoG features), discriminative (Haar’s features) d etectors, and of their fusion, is presented. These results sh ow that the last detector combines the advantages of the two

Frontiers in Neuroscience, 2017

This paper introduces an event-based luminance-free feature from the output of asynchronous event... more This paper introduces an event-based luminance-free feature from the output of asynchronous event-based neuromorphic retinas. The feature consists in mapping the distribution of the optical flow along the contours of the moving objects in the visual scene into a matrix. Asynchronous event-based neuromorphic retinas are composed of autonomous pixels, each of them asynchronously generating "spiking" events that encode relative changes in pixels' illumination at high temporal resolutions. The optical flow is computed at each event, and is integrated locally or globally in a speed and direction coordinate frame based grid, using speed-tuned temporal kernels. The latter ensures that the resulting feature equitably represents the distribution of the normal motion along the current moving edges, whatever their respective dynamics. The usefulness and the generality of the proposed feature are demonstrated in pattern recognition applications: local corner detection and global gesture recognition.

Neural computation, Jan 20, 2015

This letter introduces a study to precisely measure what an increase in spike timing precision ca... more This letter introduces a study to precisely measure what an increase in spike timing precision can add to spike-driven pattern recognition algorithms. The concept of generating spikes from images by converting gray levels into spike timings is currently at the basis of almost every spike-based modeling of biological visual systems. The use of images naturally leads to generating incorrect artificial and redundant spike timings and, more important, also contradicts biological findings indicating that visual processing is massively parallel, asynchronous with high temporal resolution. A new concept for acquiring visual information through pixel-individual asynchronous level-crossing sampling has been proposed in a recent generation of asynchronous neuromorphic visual sensors. Unlike conventional cameras, these sensors acquire data not at fixed points in time for the entire array but at fixed amplitude changes of their input, resulting optimally sparse in space and time-pixel individua...

Frontiers in psychology, 2018

The kinematics of hand movements (spatial use, curvature, acceleration, and velocity) of infants ... more The kinematics of hand movements (spatial use, curvature, acceleration, and velocity) of infants with their mothers in an interactive setting are significantly associated with age in cohorts of typical and at-risk infantsdiffer significantly at 5-6 months of age, depending on the context: relating either with an object or a person.Environmental and developmental factors shape the developmental trajectories of hand movements in different cohorts: environment for infants with VIMs; stage of development for premature infants and those with West syndrome; and both factors for infants with orality disorders.The curvature of hand movements specifically reflects atypical development in infants with West syndrome when developmental age is considered. We aimed to discriminate between typical and atypical developmental trajectory patterns of at-risk infants in an interactive setting in this observational and longitudinal study, with the assumption that hand movements (HM) reflect preverbal co...

-Nous présentons dans cet article, un nouvel algorithme de traitement d'images, la "Transformée C... more -Nous présentons dans cet article, un nouvel algorithme de traitement d'images, la "Transformée Chinoise", permettant d'estimer la localisation des doigts d'une main. Cette approche utilise une technique inspirée de la Transformée de Hough qui prend en compte la disposition des pixels de contour ainsi que l'orientation du gradient en ces pixels. Elle a été intégrée dans un système d'acquisition visuelle monoculaire des gestes humains de préhension.

IEEE Transactions on Neural Networks and Learning Systems

This paper introduces an event-based luminancefree algorithm for line and segment detection from ... more This paper introduces an event-based luminancefree algorithm for line and segment detection from the output of asynchronous event-based neuromorphic retinas. These recent biomimetic vision sensors are composed of autonomous pixels, each of them asynchronously generating visual events that encode relative changes in pixels' illumination at high temporal resolutions. This frame-free approach results in an increased energy efficiency and in real-time operation, making these sensors especially suitable for applications such as autonomous robotics. The proposed algorithm is based on an iterative event-based weighted least squares fitting, and it is consequently well suited to the high temporal resolution and asynchronous acquisition of neuromorphic cameras: parameters of a current line are updated for each event attributed (i.e., spatio-temporally close) to it, while implicitly forgetting the contribution of older events according to a speed-tuned exponentially decaying function. A detection occurs if a measure of activity, i.e., implicit measure of the number of contributing events and using the same decay function, exceeds a given threshold. The speed-tuned decreasing function is based on a measure of the apparent motion, i.e., the optical flow computed around each event. This latter ensures that the algorithm behaves independently of the edges' dynamics. Line segments are then extracted from the lines, allowing for the tracking of the corresponding endpoints. We provide experiments showing the accuracy of our algorithm and study the influence of the apparent velocity and relative orientation of the observed edges. Finally, evaluations of its computational efficiency show that this algorithm can be envisioned for high-speed applications, such as vision-based robotic navigation.

This paper introduces an event-based luminance-free feature from the output of asynchronous event... more This paper introduces an event-based luminance-free feature from the output of asynchronous event-based neuromorphic retinas. The feature consists in mapping the distribution of the optical flow along the contours of the moving objects in the visual scene into a matrix. Asynchronous event-based neuromorphic retinas are composed of autonomous pixels, each of them asynchronously generating " spiking " events that encode relative changes in pixels' illumination at high temporal resolutions. The optical flow is computed at each event, and is integrated locally or globally in a speed and direction coordinate frame based grid, using speed-tuned temporal kernels. The latter ensures that the resulting feature equitably represents the distribution of the normal motion along the current moving edges, whatever their respective dynamics. The usefulness and the generality of the proposed feature are demonstrated in pattern recognition applications: local corner detection and global gesture recognition.

In this paper, visual perception in car driving assistance i s c onsidered. The work concerns the... more In this paper, visual perception in car driving assistance i s c onsidered. The work concerns the development of a system combining a Pan-Tilt-Zoom (PTZ) camera and a standard c amera, in order to track front vehicles. The standard c amera has a small focal length, and is devoted to the analyze of the whole frontal scene. Here, the PTZ camera is used t o t r ack the closest vehicle. An original approach to control it is integrated. Camera r otations and zoom are c ontrolled by visual servoing and by an eecient real time target tracking algorithm. The aim of this work is to keep the rear view image of target vehicle stable in scale and position. The methods presented h e r e were tested o n r eal road sequences within the VELAC 1 demonstration vehicle. Experimental results show the eeectiveness of such an approach.

ACM SIGGRAPH 2006 Research posters on - SIGGRAPH '06, 2006

This communication describes a system to capture paintings in progress. An adapted algorithm base... more This communication describes a system to capture paintings in progress. An adapted algorithm based on statistical foreground substraction is used to mask the artist hand and his tools in image sequences. This allows a better understanding of the artist gesture and his mind.