Andrews Sobral | Université De La Rochelle (original) (raw)

Papers by Andrews Sobral

Pattern Recognition Letters, 2017

Background model initialization is commonly the first step of the background subtraction process.... more Background model initialization is commonly the first step of the background subtraction process. In practice, several challenges appear and perturb this process, such as dynamic background, bootstrapping, illumination changes, noise image, etc. In this context, we investigate the background model initialization as a reconstruction problem from missing data. This problem can be formulated as a matrix or tensor completion task where the image sequence (or video) is revealed as partially observed data. In this paper, the missing entries are induced from the moving regions through a simple joint motion-detection and frame-selection operation. The redundant frames are eliminated, and the moving regions are represented by zeros in our observation model. The second stage involves evaluating twenty-three state-of-the-art algorithms comprising of thirteen matrix completion and ten tensor completion algorithms. These algorithms aim to recover the low-rank component (or background model) from partially observed data. The Scene Background Initialization data set was selected in order to evaluate this proposal with respect to the background model challenges. Our experimental results show the good performance of LRGeomCG method over its direct competitors.

Sensors, 2016

Human pose estimation refers to the estimation of the location of body parts and how they are con... more Human pose estimation refers to the estimation of the location of body parts and how they are connected in an image. Human pose estimation from monocular images has wide applications (e.g., image indexing). Several surveys on human pose estimation can be found in the literature, but they focus on a certain category; for example, model-based approaches or human motion analysis, etc. As far as we know, an overall review of this problem domain has yet to be provided. Furthermore, recent advancements based on deep learning have brought novel algorithms for this problem. In this paper, a comprehensive survey of human pose estimation from monocular images is carried out including milestone works and recent advancements. Based on one standard pipeline for the solution of computer vision problems, this survey splits the problem into several modules: feature extraction and description, human body models, and modeling methods. Problem modeling methods are approached based on two means of categorization in this survey. One way to categorize includes top-down and bottom-up methods, and another way includes generative and discriminative methods. Considering the fact that one direct application of human pose estimation is to provide initialization for automatic video surveillance, there are additional sections for motion-related methods in all modules: motion features, motion models, and motion-based methods. Finally, the paper also collects 26 publicly available data sets for validation and provides error measurement methods that are frequently used.

Handbook of Robust Low-Rank and Sparse Matrix Decomposition, 2016

ABSTRACT

2015 IEEE International Conference on Computer Vision Workshop (ICCVW), 2015

Background subtraction process plays a very essential role for various computer vision tasks. The... more Background subtraction process plays a very essential role for various computer vision tasks. The process becomes more critical when the input scene contains variation of pixels such as swaying trees, rippling of water, illumination variations, etc. Recent methods of matrix decomposition into low-rank (e.g., corresponds to the background) and sparse (e.g., constitutes the moving objects) components such as Robust Principal Component Analysis (RPCA), have been shown to be very efficient framework for background subtraction. However, when the size of the input data grows and due to the lack of sparsityconstraints, these methods cannot cope with the real-time challenges and always show a weak performance due to the erroneous foreground regions. In order to address the above mentioned issues, this paper presents a superpixelbased matrix decomposition method together with maximum norm (max-norm) regularizations and structured sparsity constraints. The low-rank component estimated from each homogeneous region is more perfect, reliable, and efficient, since each superpixel provides different characteristics with a reduced value of rank. Online max-norm based matrix decomposition is employed on each segmented superpixel to separate the low rank and initial outliers support. And then, the structured sparsity constraints such as the generalized fussed lasso (GFL) are adopted for exploiting structural information continuously as the foreground pixels are both spatially connected and sparse. We propose an online single unified optimization framework for detecting foreground and learning the background model simultaneously. Rigorous experimental evaluations on challenging datasets demonstrate the superior performance of the proposed scheme in terms of both accuracy and computational time.

The BGSLibrary provides a free easy-to-use C++ open source framework to perform background subtra... more The BGSLibrary provides a free easy-to-use C++ open source framework to perform background subtraction (BGS). Currently the library provides 29 BGS algorithms. In this work the library is described and the benchmark and performance evaluation of all BGS algorithms are shown. It is expected that the results presented here can help to choice the most suitable background subtraction method.

2015 12th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 2015

The development of automated video-surveillance applications for maritime environment is a very d... more The development of automated video-surveillance applications for maritime environment is a very difficult task due to the complexity of the scenes: moving water, waves, etc. The motion of the objects of interest (i.e. ships or boats) can be mixed with the dynamic behavior of the background (non-regular patterns). In this paper, a double-constrained Robust Principal Component Analysis (RPCA), named SCM-RPCA (Shape and Confidence Mapbased RPCA), is proposed to improve the object foreground detection in maritime scenes. The sparse component is constrained by shape and confidence maps both extracted from spatial saliency maps. The experimental results in the UCSD and MarDT data sets indicate a better enhancement of the object foreground mask when compared with some related RPCA methods.

Lecture Notes in Computer Science, 2015

Background model initialization is commonly the first step of the background subtraction process.... more Background model initialization is commonly the first step of the background subtraction process. In practice, several challenges appear and perturb this process such as dynamic background, bootstrapping, illumination changes, noise image, etc. In this context, this work aims to investigate the background model initialization as a matrix completion problem. Thus, we consider the image sequence (or video) as a partially observed matrix. First, a simple joint motiondetection and frame-selection operation is done. The redundant frames are eliminated, and the moving regions are represented by zeros in our observation matrix. The second stage involves evaluating nine popular matrix completion algorithms with the Scene Background Initialization (SBI) data set, and analyze them with respect to the background model challenges. The experimental results shows the good performance of LRGeomCG [17] method over its direct competitors.

Proceedings of the 30th Annual ACM Symposium on Applied Computing, 2015

ABSTRACT Background modeling and foreground object detection is the first step in visual surveill... more ABSTRACT Background modeling and foreground object detection is the first step in visual surveillance system. The task becomes more difficult when the background scene contains significant variations, such as water surface, waving trees and sudden illumination conditions, etc. Recently, subspace learning model such as Robust Principal Component Analysis (RPCA) provides a very nice framework for separating the moving objects from the stationary scenes. However, due to its batch optimization process, high dimensional data should be processed. As a result, huge computational complexity and memory problems occur in traditional RPCA based approaches. In contrast, Online Robust PCA (ORPCA) has the ability to process such large dimensional data via stochastic manners. OR-PCA processes one frame per time instance and updates the subspace basis accordingly when a new frame arrives. However, due to the lack of features, the sparse component of OR-PCA is not always robust to handle various background modeling challenges. As a consequence, the system shows a very weak performance, which is not desirable for real applications. To handle these challenges, this paper presents a multi-feature based ORPCA scheme. A multi-feature model is able to build a robust low-rank background model of the scene. In addition, a very nice feature selection process is designed to dynamically select a useful set of features frame by frame, according to the weighted sum of total features. Experimental results on challenging datasets such as Wallflower, I2R and BMC 2012 show that the proposed scheme outperforms the state of the art approaches for the background subtraction task.

This work proposes a framework for facial expression recognition based on generalized procrustes ... more This work proposes a framework for facial expression recognition based on generalized procrustes analysis. The proposed system classifies seven different facial expressions: happiness, anger, sadness, surprise, disgust, fear and neutral. The proposed system was evaluated with the MUG Facial Expression database. Experimental results shows that the proposed method achieves 97.78% of recognition rate.

Background Modeling and Foreground Detection for Video Surveillance, 2014

ABSTRACT The BGSLibrary has been designed to provides an easy-to-use C++ framework and tools to p... more ABSTRACT The BGSLibrary has been designed to provides an easy-to-use C++ framework and tools to perform background subtraction. First released in March 2012, currently the library offers 32 background subtraction algorithms. The source code is available under GNU GPL v3 license and the library is free for non-commercial use, open source and platform independent. Note that the license of the algorithms included in BGSLibrary not necessarily have the same license of the library. Some authors do not allow that their algorithms will be used for a commercial purpose, first is needed to contact them to ask permission. However, by default, we decided to adopt the GPL-v3 license. The BGSLibrary also provides one Java based GUI (Graphical User Interface) allowing the users to configure the input video-source, region of interest, and the parameters of each BS algorithm. A MFC-based GUI is also provided for a quick access in Windows computers. But, a QT-based GUI is coming for a platform independent usage. To build/run the BGSLibrary, it is necessary to have the OpenCV library installed previously. Everyone is invited to collaborate with the BGSLibrary. In this chapter some efforts has been made for how to make and add your contributions in the library. To purchase the handbook: http://www.crcpress.com/product/isbn/9781482205374

Lecture Notes in Computer Science, 2015

Accurate and efficient foreground detection is an important task in video surveillance system. Th... more Accurate and efficient foreground detection is an important task in video surveillance system. The task becomes more critical when the background scene shows more variations, such as water surface, waving trees, varying illumination conditions, etc. Recently, Robust Principal Components Analysis (RPCA) shows a very nice framework for moving object detection. The background sequence is modeled by a low-dimensional subspace called low-rank matrix and sparse error constitutes the foreground objects. But RPCA presents the limitations of computational complexity and memory storage due to batch optimization methods, as a result it is difficult to apply for real-time system. To handle these challenges, this paper presents a robust foreground detection algorithm via Online Robust PCA (OR-PCA) using image decomposition along with continuous constraint such as Markov Random Field (MRF). OR-PCA with good initialization scheme using image decomposition approach improves the accuracy of foreground detection and the computation time as well. Moreover, solving MRF with graph-cuts exploits structural information using spatial neighborhood system and similarities to further improve the foreground segmentation in highly dynamic backgrounds. Experimental results on challenging datasets such as Wallflower, I2R, BMC 2012 and Change Detection 2014 dataset demonstrate that our proposed scheme significantly outperforms the state of the art approaches and works effectively on a wide range of complex background scenes.

Lecture Notes in Computer Science, 2014

Background subtraction (BS) is the art of separating moving objects from their background. The Ba... more Background subtraction (BS) is the art of separating moving objects from their background. The Background Modeling (BM) is one of the main steps of the BS process. Several subspace learning (SL) algorithms based on matrix and tensor tools have been used to perform the BM of the scenes. However, several SL algorithms work on a batch process increasing memory consumption when data size is very large. Moreover, these algorithms are not suitable for streaming data when the full size of the data is unknown. In this work, we propose an incremental tensor subspace learning that uses only a small part of the entire data and updates the low-rank model incrementally when new data arrive. In addition, the multi-feature model allows us to build a robust low-rank background model of the scene. Experimental results shows that the proposed method achieves interesting results for background subtraction task.

Computer Vision and Image Understanding, 2014

ABSTRACT Background subtraction (BS) is a crucial step in many computer vision systems, as it is ... more ABSTRACT Background subtraction (BS) is a crucial step in many computer vision systems, as it is first applied to detect moving objects within a video stream. Many algorithms have been designed to segment the foreground objects from the background of a sequence. In this article, we propose to use the BMC (Background Models Challenge) dataset, and to compare the 29 methods implemented in the BGSLibrary. From this large set of various BG methods, we have conducted a relevant experimental analysis to evaluate both their robustness and their practical performance in terms of processor/memory requirements.

2015 IEEE International Conference on Computer Vision Workshop (ICCVW), 2015

Background subtraction is an important task for visual surveillance systems. However, this task b... more Background subtraction is an important task for visual surveillance systems. However, this task becomes more complex when the data size grows since the real-world scenario requires larger data to be processed in a more efficient way, and in some cases, in a continuous manner. Until now, most of background subtraction algorithms were designed for mono or trichromatic cameras within the visible spectrum or near infrared part. Recent advances in multispectral imaging technologies give the possibility to record multispectral videos for video surveillance applications. Due to the specific nature of these data, many of the bands within multispectral images are often strongly correlated. In addition, processing multispectral images with hundreds of bands can be computationally burdensome. In order to address these major difficulties of multispectral imaging for video surveillance, this paper propose an online stochastic framework for tensor decomposition of multispectral video sequences (OSTD). First, the experimental evaluations on synthetic generated data show the robustness of the OSTD with other state of the art approaches then, we apply the same idea on seven multispectral video bands to show that only RGB features are not sufficient to tackle color saturation, illumination variations and shadows problem, but the addition of six visible spectral bands together with one near infra-red spectra provides a better background/foreground separation.

Computer Science Review, 2017

Background/foreground separation is the first step in video surveillance system to detect moving ... more Background/foreground separation is the first step in video surveillance system to detect moving objects. Recent research on problem formulations based on decomposition into low-rank plus sparse matrices shows a suitable framework to separate moving objects from the background. The most representative problem formulation is the Robust Principal Component Analysis (RPCA) solved via Principal Component Pursuit (PCP) which decomposes a data matrix in a low-rank matrix and a sparse matrix. However, similar robust implicit or explicit decompositions can be made in the following problem formulations: Robust Non-negative Matrix Factorization (RNMF), Robust Matrix Completion (RMC), Robust Subspace Recovery (RSR), Robust Subspace Tracking (RST) and Robust Low-Rank Minimization (RLRM). The main goal of these similar problem formulations is to obtain explicitly or implicitly a decomposition into low-rank matrix plus additive matrices. These formulation problems differ from the implicit or explicit decomposition, the loss function, the optimization problem and the solvers. As the problem formulation can be NP-hard in its original formulation, and it can be convex or not following the constraints and the loss functions used, the key challenges concern the design of efficient relaxed models and solvers which have to be with iterations as few as possible, and as efficient as possible. In the application of background/foreground separation, constraints inherent to the specificities of the background and the foreground as the temporal and spatial properties need

This work proposes a holistic method for highway traffic video classification based on vehicle cr... more This work proposes a holistic method for highway traffic video classification based on vehicle crowd properties. The method classifies the traffic congestion into three classes: light, medium and heavy. This is done by usage of average crowd density and crowd speed. Firstly, the crowd density is estimated by background subtraction and the crowd speed is performed by pyramidal Kanade-Lucas-Tomasi (KLT) tracker algorithm. The features classification with neural networks show 94.50% of accuracy on experimental results from 254 highway traffic videos of UCSD data set.

This thesis introduces the recent advances on decomposition into low-rank plus sparse matrices an... more This thesis introduces the recent advances on decomposition into low-rank plus sparse matrices and tensors, as well as the main contributions to face the principal issues in moving object detection. First, we present an overview of the state-of-the-art methods for low-rank and sparse decomposition, as well as their application to background modeling and foreground segmentation tasks. Next, we address the problem of background model initialization as a reconstruction process from missing/corrupted data. A novel methodology is presented showing an attractive potential for background modeling initialization in video surveillance. Subsequently, we propose a double-constrained version of robust principal component analysis to improve the foreground detection in maritime environments for automated video-surveillance applications. The algorithm makes use of double constraints extracted from spatial saliency maps to enhance object foreground detection in dynamic scenes. We also developed tw...

ArXiv, 2021

Finding the best mathematical equation to deal with the different challenges found in complex sce... more Finding the best mathematical equation to deal with the different challenges found in complex scenarios requires a thorough understanding of the scenario and a trial and error process carried out by experts. In recent years, most state-of-the-art equation discovery methods have been widely applied in modeling and identification systems. However, equation discovery approaches can be very useful in computer vision, particularly in the field of feature extraction. In this paper, we focus on recent AI advances to present a novel framework for automatically discovering equations from scratch with little human intervention to deal with the different challenges encountered in real-world scenarios. In addition, our proposal can reduce human bias by proposing a search space design through generative network instead of hand-designed. As a proof of concept, the equations discovered by our framework are used to distinguish moving objects from the background in video sequences. Experimental resu...

This work proposes an automatic human-face expression recognition system that classifies seven di... more This work proposes an automatic human-face expression recognition system that classifies seven different facial expressions: happiness, anger, sadness, surprise, disgust, fear and neutral. The experimental results show that the proposed system achieves the best hit hate using a linear discriminant classifier, 99.71% and 99.55% for MUG and FEEDTUM databases respectively.

Pattern Recognition Letters, 2017

Sensors, 2016

Handbook of Robust Low-Rank and Sparse Matrix Decomposition, 2016

ABSTRACT

2015 IEEE International Conference on Computer Vision Workshop (ICCVW), 2015

2015 12th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 2015

Lecture Notes in Computer Science, 2015

Background model initialization is commonly the first step of the background subtraction process.... more Background model initialization is commonly the first step of the background subtraction process. In practice, several challenges appear and perturb this process such as dynamic background, bootstrapping, illumination changes, noise image, etc. In this context, this work aims to investigate the background model initialization as a matrix completion problem. Thus, we consider the image sequence (or video) as a partially observed matrix. First, a simple joint motiondetection and frame-selection operation is done. The redundant frames are eliminated, and the moving regions are represented by zeros in our observation matrix. The second stage involves evaluating nine popular matrix completion algorithms with the Scene Background Initialization (SBI) data set, and analyze them with respect to the background model challenges. The experimental results shows the good performance of LRGeomCG [17] method over its direct competitors.

Proceedings of the 30th Annual ACM Symposium on Applied Computing, 2015

Background Modeling and Foreground Detection for Video Surveillance, 2014

Lecture Notes in Computer Science, 2015

Lecture Notes in Computer Science, 2014

Computer Vision and Image Understanding, 2014

2015 IEEE International Conference on Computer Vision Workshop (ICCVW), 2015

Computer Science Review, 2017

ArXiv, 2021

The LRSLibrary provides a collection of low-rank and sparse decomposition algorithms in MATLAB. T... more The LRSLibrary provides a collection of low-rank and sparse decomposition algorithms in MATLAB. The library was designed for motion segmentation in videos, but it can be also used or adapted for other computer vision problems. Currently the LRSLibrary contains a total of 70 matrix-based and tensor-based algorithms. The LRSLibrary was tested successfully in MATLAB R2013b both x86 and x64 versions.

The BGSLibrary provides a C++ framework to perform background subtraction (BGS). The code works e... more The BGSLibrary provides a C++ framework to perform background subtraction (BGS). The code works either on Windows or on Linux. Currently the library offers 36 BGS algorithms. A large amount of algorithms were provided by several authors. The source code is available under GNU GPL v3 license, the library is free and open source for academic purposes. Any user can be download latest project source code using SVN client. In Windows, a demo project for Visual Studio 2010 is provided. An executable version of BGSLibrary is available for Windows 32 bits and 64 bits. For Linux and Mac users, a Makefile can be used to compile all files and generate an executable example.

The recent advances in robust matrix and tensor factorization are fundamental and can be applied ... more The recent advances in robust matrix and tensor factorization are fundamental and can be applied to background modeling and foreground detection for video surveillance. It was for this reason that the LRSLibrary was developed. The goal is to provide an easy-to-use library to apply low-rank and sparse decomposition tools for background modeling and subtraction in videos. The library is open-source and free for academic/research purpose (non-commercial).

The BGSLibrary has been designed to provides an easy-to-use C++ framework and tools to perform ba... more The BGSLibrary has been designed to provides an easy-to-use C++ framework and tools to perform background subtraction. First released in March 2012, currently the library offers 32 background subtraction algorithms. The source code is available under GNU GPL v3 license and the library is free for non-commercial use, open source and platform independent. Note that the license of the algorithms included in BGSLibrary not necessarily have the same license of the library. Some authors do not allow that their algorithms will be used for a commercial purpose, first is needed to contact them to ask permission. However, by default, we decided to adopt the GPL-v3 license.
The BGSLibrary also provides one Java based GUI (Graphical User Interface) allowing the users to configure the input video-source, region of interest, and the parameters of each BS algorithm. A MFC-based GUI is also provided for a quick access in Windows computers. But, a QT-based GUI is coming for a platform independent usage. To build/run the BGSLibrary, it is necessary to have the OpenCV library installed previously. Everyone is invited to collaborate with the BGSLibrary. In this chapter some efforts has been made for how to make and add your contributions in the library."

Atualmente, sistemas inteligentes utilizados para monitoração de tráfego urbano têm sido adotados... more Atualmente, sistemas inteligentes utilizados para monitoração de tráfego urbano têm sido adotados, cada vez mais, com maior frequência. As soluções tradicionais produzem estatísticas através da detecção e contagem individual de veículos presentes no trânsito. Porém, estes sistemas comumente falham, especialmente em cenas que possuem uma grande quantidade de veículos em movimento (e.g. alto congestionamento) por conta do aumento da oclusão entre os veículos. Muitas vezes a oclusão acaba prejudicando a predição exata da quantidade de veículos presentes na cena e a correta identificação do real estado do trânsito. Métodos alternativos analisam o vídeo de forma global considerando o trânsito como uma única entidade -- nuvem ou aglomerado de veículos que possuem um comportamento único. Através da análise do comportamento da nuvem de veículos, os métodos baseados em contexto global procuram extrair informações relevantes tais como a densidade, velocidade, localização e sentido dos veículos presentes na cena, favorecendo a identificação do real estado do trânsito. Considerando esta abordagem, o presente trabalho propõe um método para classificação do estado do trânsito. Para determinar o estado do trânsito, o método proposto utiliza duas propriedades para classificar o trânsito em três níveis de congestionamento: baixo, médio e alto. Tais propriedades são representadas pela densidade média da nuvem de veículos e sua respectiva velocidade média. Neste trabalho, decidiu-se por combinar estas duas propriedades em um vetor de características que foi utilizado para compor o conjunto de treinamento. Os resultados experimentais demonstram uma taxa de acerto de 94.5\% em um conjunto de 254 vídeos de trânsito utilizando redes neurais artificiais.