Andrzej Cichocki - Profile on Academia.edu (original) (raw)
Papers by Andrzej Cichocki
In tomographic image reconstruction from limited-view projections the underlying inverse problem ... more In tomographic image reconstruction from limited-view projections the underlying inverse problem is ill-posed with the rank-deficient system matrix. The minimal-norm least squares solution may considerably differs from the true solution, and hence a priori knowledge is needed to improve the reconstruction. In our approach, we assume that the true image presents sparse features with uniform spacial smoothness. The sparsity constraints are imposed with the p diversity measure that is minimized with the FOCUSS algorithm. The spacial smoothness is enforced with the adaptive Wiener noise removing implemented in each FOCUSS iterations. The simulation results demonstrate the benefits of the proposed approach.
A novel approach combining a time-frequency representation of brain activity in form of recorded ... more A novel approach combining a time-frequency representation of brain activity in form of recorded EEG signals together with nonnegative matrix factorization (NMF) post-processing section in brain computer interface (BCI) training paradigm is presented. Such a combination of two emerging signal analysis techniques enables us to find and enhance very small oscillations related to presented visual stimuli. Presented results confirm validity of the chosen approach.
IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2000
Linear discriminant analysis (LDA) has been widely adopted to classify event-related potential (E... more Linear discriminant analysis (LDA) has been widely adopted to classify event-related potential (ERP) in braincomputer interface (BCI). Good classification performance of the ERP-based BCI usually requires sufficient data recordings for effective training of the LDA classifier, and hence a long system calibration time which however may depress the system practicability and cause the users resistance to the BCI system. In this study, we introduce a spatial-temporal discriminant analysis (STDA) to ERP classification. As a multiway extension of the LDA, the STDA method tries to maximize the discriminant information between target and non-target classes through finding two projection matrices from spatial and temporal dimensions collaboratively, which reduces effectively the feature dimensionality in the discriminant analysis, and hence decreases significantly the number of required training samples. The proposed STDA method was validated with dataset II of the BCI Competition III and dataset recorded from our own experiments, and compared to the state-of-the-art algorithms for ERP classification. Online experiments were additionally implemented for the validation. The superior classification performance in using few training samples shows that the STDA is effective to reduce the system calibration time and improve the classification accuracy, thereby enhancing the practicability of ERP-based BCI.
L1-Regularized Multiway Canonical Correlation Analysis for SSVEP-Based BCI
IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2000
Canonical correlation analysis (CCA) between recorded electroencephalogram (EEG) and designed ref... more Canonical correlation analysis (CCA) between recorded electroencephalogram (EEG) and designed reference signals of sine-cosine waves usually works well for steady-state visual evoked potential (SSVEP) recognition in brain-computer interface (BCI) application. However, using the reference signals of sine- cosine waves without subject-specific and inter-trial information can hardly give the optimal recognition accuracy, due to possible overfitting, especially within a short time window length. This paper introduces an L1-regularized multiway canonical correlation analysis (L1-MCCA) for reference signal optimization to improve the SSVEP recognition performance further. A multiway extension of the CCA, called MCCA, is first presented, in which collaborative CCAs are exploited to optimize the reference signals in correlation analysis for SSVEP recognition alternatingly from the channel-way and trial-way arrays of constructed EEG tensor. L1-regularization is subsequently imposed on the trial-way array optimization in the MCCA, and hence results in the more powerful L1-MCCA with function of effective trial selection. Both the proposed MCCA and L1-MCCA methods are validated for SSVEP recognition with EEG data from 10 healthy subjects, and compared to the ordinary CCA without reference signal optimization. Experimental results show that the MCCA significantly outperforms the CCA for SSVEP recognition. The L1-MCCA further improves the recognition accuracy which is significantly higher than that of the MCCA.
IEEE Transactions on Neural Networks, 2000
N ONLINEAR signal processing using neural networks inherited the dominant mean-squared error crit... more N ONLINEAR signal processing using neural networks inherited the dominant mean-squared error criteria from linear adaptive filtering theory due to a number of appealing properties exhibited by second-order statistical optimality criteria under the linear model. In linear filtering of Gaussian distributed signals, second order statistics such as variance and correlation completely describe the data distribution and their interactions. In the nonlinear model framework, where signals are not normally Gaussian distributed, this is not the case anymore.
A common neural-network model for unsupervised exploratory data analysis and independent component analysis
IEEE Transactions on Neural Networks, 1998
This paper presents the derivation of an unsupervised learning algorithm, which enables the ident... more This paper presents the derivation of an unsupervised learning algorithm, which enables the identification and visualization of latent structure within ensembles of high-dimensional data. This provides a linear projection of the data onto a lower dimensional subspace to identify the characteristic structure of the observations independent latent causes. The algorithm is shown to be a very promising tool for unsupervised exploratory data analysis and data visualization. Experimental results confirm the attractiveness of this technique for exploratory data analysis and an empirical comparison is made with the recently proposed generative topographic mapping (GTM) and standard principal component analysis (PCA). Based on standard probability density models a generic nonlinearity is developed which allows both 1) identification and visualization of dichotomised clusters inherent in the observed data and 2) separation of sources with arbitrary distributions from mixtures, whose dimensionality may be greater than that of number of sources. The resulting algorithm is therefore also a generalized neural approach to independent component analysis (ICA) and it is considered to be a promising method for analysis of real-world data that will consist of sub- and super-Gaussian components such as biomedical signals.
Table of contents
IEEE Transactions on Neural Networks, 2000
A robust approach to independent component analysis of signals with high-level noise measurements
IEEE Transactions on Neural Networks, 2003
We propose a robust approach for independent component analysis (ICA) of signals where observatio... more We propose a robust approach for independent component analysis (ICA) of signals where observations are contaminated with high-level additive noise and/or outliers. The source signals may contain mixtures of both sub-Gaussian and super-Gaussian components, and the number of sources is unknown. Our robust approach includes two procedures. In the first procedure, a robust prewhitening technique is used to reduce the power of additive noise, the dimensionality and the correlation among sources. A cross-validation technique is introduced to estimate the number of sources in this first procedure. In the second procedure, a nonlinear function is derived using the parameterized t-distribution density model. This nonlinear function is robust against the undue influence of outliers fundamentally. Moreover, the stability of the proposed algorithm and the robust property of misestimating the parameters (kurtosis) have been studied. By combining the t-distribution model with a family of light-tailed distributions (sub-Gaussian) model, we can separate the mixture of sub-Gaussian and super-Gaussian source components. Through the analysis of artificially synthesized data and real-world magnetoencephalographic (MEG) data, we illustrate the efficacy of this robust approach.
IEEE Transactions on Neural Networks, 2004
Independent component analysis is to extract independent signals from their linear mixtures witho... more Independent component analysis is to extract independent signals from their linear mixtures without assuming prior knowledge of their mixing coefficients. As we know, a number of factors are likely to affect separation results in practical applications, such as the number of active sources, the distribution of source signals, and noise.The purpose of this paper to develop a general framework of blind separation from a practical point of view with special emphasis on the activation function adaptation. First, we propose the exponential generative model for probability density functions. A method of constructing an exponential generative model from the activation functions is discussed. Then, a learning algorithm is derived to update the parameters in the exponential generative model. The learning algorithm for the activation function adaptation is consistent with the one for training the demixing model. Stability analysis of the learning algorithm for the activation function is also discussed. Both theoretical analysis and simulations show that the proposed approach is universally convergent regardless of the distributions of sources. Finally, computer simulations are given to demonstrate the effectiveness and validity of the approach.
IEEE Transactions on Neural Networks, 2006
In this paper, we use a two-stage sparse factorization approach for blindly estimating the channe... more In this paper, we use a two-stage sparse factorization approach for blindly estimating the channel parameters and then estimating source components for electroencephalogram (EEG) signals. EEG signals are assumed to be linear mixtures of source components, artifacts, etc. Therefore, a raw EEG data matrix can be factored into the product of two matrices, one of which represents the mixing matrix and the other the source component matrix. Furthermore, the components are sparse in the time-frequency domain, i.e., the factorization is a sparse factorization in the time frequency domain. It is a challenging task to estimate the mixing matrix. Our extensive analysis and computational results, which were based on many sets of EEG data, not only provide firm evidences supporting the above assumption, but also prompt us to propose a new algorithm for estimating the mixing matrix. After the mixing matrix is estimated, the source components are estimated in the time frequency domain using a linear programming method. In an example of the potential applications of our approach, we analyzed the EEG data that was obtained from a modified Sternberg memory experiment. Two almost uncorrelated components obtained by applying the sparse factorization method were selected for phase synchronization analysis. Several interesting findings were obtained, especially that memory-related synchronization and desynchronization appear in the alpha band, and that the strength of alpha band synchronization is related to memory performance.
Blind Extraction of Global Signal From Multi-Channel Noisy Observations
IEEE Transactions on Neural Networks, 2000
We propose a novel efficient method of blind signal extraction from multi-sensor networks when ea... more We propose a novel efficient method of blind signal extraction from multi-sensor networks when each observed signal consists of one global signal and local uncorrelated signals. Most of existing blind signal separation and extraction methods such as independent component analysis have constraints such as statistical independence, non-Gaussianity, and underdetermination, and they are not suitable for global signal extraction problem from noisy observations. We developed an estimation algorithm based on alternating iteration and the smart weighted averaging. The proposed method does not have strong assumptions such as independence or non-Gaussianity. Experimental results using a musical signal and a real electroencephalogram demonstrate the advantage of the proposed method.
Analysis and Online Realization of the CCA Approach for Blind Source Separation
IEEE Transactions on Neural Networks, 2000
A critical analysis of the canonical correlation analysis (CCA) approach in blind source separati... more A critical analysis of the canonical correlation analysis (CCA) approach in blind source separation (BSS) is provided. It is proved that by maximizing the autocorrelation functions of the recovered signals we can separate the source signals successfully. It is further shown that the CCA approach represents the same class of generalized eigenvalue decomposition (GEVD) problems as the matrix pencil method. Finally, online realizations of the CCA approach are discussed with a linear-predictor-based algorithm studied as an example.
IEEE Transactions on Neural Networks, 2000
In this paper we present an iterative inversion (II) approach to blind source separation (BSS). I... more In this paper we present an iterative inversion (II) approach to blind source separation (BSS). It consists of a quasi-Newton method for the resolution of an estimating equation obtained from the implicit inversion of a robust estimate of the mixing system. The resulting learning rule includes several existing algorithms for BSS as particular cases giving them a novel and unified interpretation.It also provides a justification of the Cardoso and Laheld step size normalization . The II method is first presented for instantaneous mixtures and then extended to the problem of blind separation of convolutive mixtures. Finally, we derive the necessary and sufficient asymptotic stability conditions for both the instantaneous and convolutive methods to converge.
IEEE Transactions on Neural Networks, 2000
Nonnegative matrix factorization (NMF) is an unsupervised learning method useful in various appli... more Nonnegative matrix factorization (NMF) is an unsupervised learning method useful in various applications including image processing and semantic analysis of documents. This paper focuses on symmetric NMF (SNMF), which is a special case of NMF decomposition. Three parallel multiplicative update algorithms using level 3 basic linear algebra subprograms directly are developed for this problem. First, by minimizing the Euclidean distance, a multiplicative update algorithm is proposed, and its convergence under mild conditions is proved. Based on it, we further propose another two fast parallel methods: α-SNMF and β-SNMF algorithms. All of them are easy to implement. These algorithms are applied to probabilistic clustering. We demonstrate their effectiveness for facial image clustering, document categorization, and pattern clustering in gene expression.
IEEE Transactions on Neural Networks, 2004
This paper reports a study on the problem of the blind simultaneous extraction of specific groups... more This paper reports a study on the problem of the blind simultaneous extraction of specific groups of independent components from a linear mixture. This paper first presents a general overview and unification of several information theoretic criteria for the extraction of a single independent component. Then, our contribution fills the theoretical gap that exists between extraction and separation by presenting tools that extend these criteria to allow the simultaneous blind extraction of subsets with an arbitrary number of independent components. In addition, we analyze a family of learning algorithms based on Stiefel manifolds and the natural gradient ascent, present the nonlinear optimal activations (score) functions, and provide new or extended local stability conditions. Finally, we illustrate the performance and features of the proposed approach by computer-simulation experiments.
IEEE Transactions on Neural Networks, 2005
In this letter, we solve the problem of identifying matrices S and A knowing only their multiplic... more In this letter, we solve the problem of identifying matrices S and A knowing only their multiplication X = AS, under some conditions, expressed either in terms of A and sparsity of S (identifiability conditions), or in terms of X (sparse component analysis (SCA) conditions). We present algorithms for such identification and illustrate them by examples.
Simplified neural networks for solving linear least squares and total least squares problems in real time
IEEE Transactions on Neural Networks, 1994
In this paper a new class of simplified low-cost analog artificial neural networks with on chip a... more In this paper a new class of simplified low-cost analog artificial neural networks with on chip adaptive learning algorithms are proposed for solving linear systems of algebraic equations in real time. The proposed learning algorithms for linear least squares (LS), total least squares (TLS) and data least squares (DLS) problems can be considered as modifications and extensions of well known algorithms: the row-action projection-Kaczmarz algorithm and/or the LMS (Adaline) Widrow-Hoff algorithms. The algorithms can be applied to any problem which can be formulated as a linear regression problem. The correctness and high performance of the proposed neural networks are illustrated by extensive computer simulation results.
Equivalence Probability and Sparsity of Two Sparse Solutions in Sparse Representation
IEEE Transactions on Neural Networks, 2000
This paper discusses the estimation and numerical calculation of the probability that the 0-norm ... more This paper discusses the estimation and numerical calculation of the probability that the 0-norm and 1-norm solutions of underdetermined linear equations are equivalent in the case of sparse representation. First, we define the sparsity degree of a signal. Two equivalence probability estimates are obtained when the entries of the 0-norm solution have different sparsity degrees. One is for the case in which the basis matrix is given or estimated, and the other is for the case in which the basis matrix is random. However, the computational burden to calculate these probabilities increases exponentially as the number of columns of the basis matrix increases. This computational complexity problem can be avoided through a sampling method. Next, we analyze the sparsity degree of mixtures and establish the relationship between the equivalence probability and the sparsity degree of the mixtures. This relationship can be used to analyze the performance of blind source separation (BSS). Furthermore, we extend the equivalence probability estimates to the small noise case. Finally, we illustrate how to use these theoretical results to guarantee a satisfactory performance in underdetermined BSS.
A switched-capacitor interface for capacitive sensors based on relaxation oscillators
IEEE Transactions on Instrumentation and Measurement, 1990
... to the unknown capacitance C, has been achieved to be di-rectly compatible with amicroprocess... more ... to the unknown capacitance C, has been achieved to be di-rectly compatible with amicroprocessor. ... and M. S. Beck, “A high frequency stray-immune capacitance transducer based on the ... and K. Watanabe, “A switched-capacitor charge-balancing analog-to-digital converter and ...
A novel switched-capacitor four-quadrant analog multiplier-divider and some of its applications
IEEE Transactions on Instrumentation and Measurement, 2000
In tomographic image reconstruction from limited-view projections the underlying inverse problem ... more In tomographic image reconstruction from limited-view projections the underlying inverse problem is ill-posed with the rank-deficient system matrix. The minimal-norm least squares solution may considerably differs from the true solution, and hence a priori knowledge is needed to improve the reconstruction. In our approach, we assume that the true image presents sparse features with uniform spacial smoothness. The sparsity constraints are imposed with the p diversity measure that is minimized with the FOCUSS algorithm. The spacial smoothness is enforced with the adaptive Wiener noise removing implemented in each FOCUSS iterations. The simulation results demonstrate the benefits of the proposed approach.
A novel approach combining a time-frequency representation of brain activity in form of recorded ... more A novel approach combining a time-frequency representation of brain activity in form of recorded EEG signals together with nonnegative matrix factorization (NMF) post-processing section in brain computer interface (BCI) training paradigm is presented. Such a combination of two emerging signal analysis techniques enables us to find and enhance very small oscillations related to presented visual stimuli. Presented results confirm validity of the chosen approach.
IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2000
Linear discriminant analysis (LDA) has been widely adopted to classify event-related potential (E... more Linear discriminant analysis (LDA) has been widely adopted to classify event-related potential (ERP) in braincomputer interface (BCI). Good classification performance of the ERP-based BCI usually requires sufficient data recordings for effective training of the LDA classifier, and hence a long system calibration time which however may depress the system practicability and cause the users resistance to the BCI system. In this study, we introduce a spatial-temporal discriminant analysis (STDA) to ERP classification. As a multiway extension of the LDA, the STDA method tries to maximize the discriminant information between target and non-target classes through finding two projection matrices from spatial and temporal dimensions collaboratively, which reduces effectively the feature dimensionality in the discriminant analysis, and hence decreases significantly the number of required training samples. The proposed STDA method was validated with dataset II of the BCI Competition III and dataset recorded from our own experiments, and compared to the state-of-the-art algorithms for ERP classification. Online experiments were additionally implemented for the validation. The superior classification performance in using few training samples shows that the STDA is effective to reduce the system calibration time and improve the classification accuracy, thereby enhancing the practicability of ERP-based BCI.
L1-Regularized Multiway Canonical Correlation Analysis for SSVEP-Based BCI
IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2000
Canonical correlation analysis (CCA) between recorded electroencephalogram (EEG) and designed ref... more Canonical correlation analysis (CCA) between recorded electroencephalogram (EEG) and designed reference signals of sine-cosine waves usually works well for steady-state visual evoked potential (SSVEP) recognition in brain-computer interface (BCI) application. However, using the reference signals of sine- cosine waves without subject-specific and inter-trial information can hardly give the optimal recognition accuracy, due to possible overfitting, especially within a short time window length. This paper introduces an L1-regularized multiway canonical correlation analysis (L1-MCCA) for reference signal optimization to improve the SSVEP recognition performance further. A multiway extension of the CCA, called MCCA, is first presented, in which collaborative CCAs are exploited to optimize the reference signals in correlation analysis for SSVEP recognition alternatingly from the channel-way and trial-way arrays of constructed EEG tensor. L1-regularization is subsequently imposed on the trial-way array optimization in the MCCA, and hence results in the more powerful L1-MCCA with function of effective trial selection. Both the proposed MCCA and L1-MCCA methods are validated for SSVEP recognition with EEG data from 10 healthy subjects, and compared to the ordinary CCA without reference signal optimization. Experimental results show that the MCCA significantly outperforms the CCA for SSVEP recognition. The L1-MCCA further improves the recognition accuracy which is significantly higher than that of the MCCA.
IEEE Transactions on Neural Networks, 2000
N ONLINEAR signal processing using neural networks inherited the dominant mean-squared error crit... more N ONLINEAR signal processing using neural networks inherited the dominant mean-squared error criteria from linear adaptive filtering theory due to a number of appealing properties exhibited by second-order statistical optimality criteria under the linear model. In linear filtering of Gaussian distributed signals, second order statistics such as variance and correlation completely describe the data distribution and their interactions. In the nonlinear model framework, where signals are not normally Gaussian distributed, this is not the case anymore.
A common neural-network model for unsupervised exploratory data analysis and independent component analysis
IEEE Transactions on Neural Networks, 1998
This paper presents the derivation of an unsupervised learning algorithm, which enables the ident... more This paper presents the derivation of an unsupervised learning algorithm, which enables the identification and visualization of latent structure within ensembles of high-dimensional data. This provides a linear projection of the data onto a lower dimensional subspace to identify the characteristic structure of the observations independent latent causes. The algorithm is shown to be a very promising tool for unsupervised exploratory data analysis and data visualization. Experimental results confirm the attractiveness of this technique for exploratory data analysis and an empirical comparison is made with the recently proposed generative topographic mapping (GTM) and standard principal component analysis (PCA). Based on standard probability density models a generic nonlinearity is developed which allows both 1) identification and visualization of dichotomised clusters inherent in the observed data and 2) separation of sources with arbitrary distributions from mixtures, whose dimensionality may be greater than that of number of sources. The resulting algorithm is therefore also a generalized neural approach to independent component analysis (ICA) and it is considered to be a promising method for analysis of real-world data that will consist of sub- and super-Gaussian components such as biomedical signals.
Table of contents
IEEE Transactions on Neural Networks, 2000
A robust approach to independent component analysis of signals with high-level noise measurements
IEEE Transactions on Neural Networks, 2003
We propose a robust approach for independent component analysis (ICA) of signals where observatio... more We propose a robust approach for independent component analysis (ICA) of signals where observations are contaminated with high-level additive noise and/or outliers. The source signals may contain mixtures of both sub-Gaussian and super-Gaussian components, and the number of sources is unknown. Our robust approach includes two procedures. In the first procedure, a robust prewhitening technique is used to reduce the power of additive noise, the dimensionality and the correlation among sources. A cross-validation technique is introduced to estimate the number of sources in this first procedure. In the second procedure, a nonlinear function is derived using the parameterized t-distribution density model. This nonlinear function is robust against the undue influence of outliers fundamentally. Moreover, the stability of the proposed algorithm and the robust property of misestimating the parameters (kurtosis) have been studied. By combining the t-distribution model with a family of light-tailed distributions (sub-Gaussian) model, we can separate the mixture of sub-Gaussian and super-Gaussian source components. Through the analysis of artificially synthesized data and real-world magnetoencephalographic (MEG) data, we illustrate the efficacy of this robust approach.
IEEE Transactions on Neural Networks, 2004
Independent component analysis is to extract independent signals from their linear mixtures witho... more Independent component analysis is to extract independent signals from their linear mixtures without assuming prior knowledge of their mixing coefficients. As we know, a number of factors are likely to affect separation results in practical applications, such as the number of active sources, the distribution of source signals, and noise.The purpose of this paper to develop a general framework of blind separation from a practical point of view with special emphasis on the activation function adaptation. First, we propose the exponential generative model for probability density functions. A method of constructing an exponential generative model from the activation functions is discussed. Then, a learning algorithm is derived to update the parameters in the exponential generative model. The learning algorithm for the activation function adaptation is consistent with the one for training the demixing model. Stability analysis of the learning algorithm for the activation function is also discussed. Both theoretical analysis and simulations show that the proposed approach is universally convergent regardless of the distributions of sources. Finally, computer simulations are given to demonstrate the effectiveness and validity of the approach.
IEEE Transactions on Neural Networks, 2006
In this paper, we use a two-stage sparse factorization approach for blindly estimating the channe... more In this paper, we use a two-stage sparse factorization approach for blindly estimating the channel parameters and then estimating source components for electroencephalogram (EEG) signals. EEG signals are assumed to be linear mixtures of source components, artifacts, etc. Therefore, a raw EEG data matrix can be factored into the product of two matrices, one of which represents the mixing matrix and the other the source component matrix. Furthermore, the components are sparse in the time-frequency domain, i.e., the factorization is a sparse factorization in the time frequency domain. It is a challenging task to estimate the mixing matrix. Our extensive analysis and computational results, which were based on many sets of EEG data, not only provide firm evidences supporting the above assumption, but also prompt us to propose a new algorithm for estimating the mixing matrix. After the mixing matrix is estimated, the source components are estimated in the time frequency domain using a linear programming method. In an example of the potential applications of our approach, we analyzed the EEG data that was obtained from a modified Sternberg memory experiment. Two almost uncorrelated components obtained by applying the sparse factorization method were selected for phase synchronization analysis. Several interesting findings were obtained, especially that memory-related synchronization and desynchronization appear in the alpha band, and that the strength of alpha band synchronization is related to memory performance.
Blind Extraction of Global Signal From Multi-Channel Noisy Observations
IEEE Transactions on Neural Networks, 2000
We propose a novel efficient method of blind signal extraction from multi-sensor networks when ea... more We propose a novel efficient method of blind signal extraction from multi-sensor networks when each observed signal consists of one global signal and local uncorrelated signals. Most of existing blind signal separation and extraction methods such as independent component analysis have constraints such as statistical independence, non-Gaussianity, and underdetermination, and they are not suitable for global signal extraction problem from noisy observations. We developed an estimation algorithm based on alternating iteration and the smart weighted averaging. The proposed method does not have strong assumptions such as independence or non-Gaussianity. Experimental results using a musical signal and a real electroencephalogram demonstrate the advantage of the proposed method.
Analysis and Online Realization of the CCA Approach for Blind Source Separation
IEEE Transactions on Neural Networks, 2000
A critical analysis of the canonical correlation analysis (CCA) approach in blind source separati... more A critical analysis of the canonical correlation analysis (CCA) approach in blind source separation (BSS) is provided. It is proved that by maximizing the autocorrelation functions of the recovered signals we can separate the source signals successfully. It is further shown that the CCA approach represents the same class of generalized eigenvalue decomposition (GEVD) problems as the matrix pencil method. Finally, online realizations of the CCA approach are discussed with a linear-predictor-based algorithm studied as an example.
IEEE Transactions on Neural Networks, 2000
In this paper we present an iterative inversion (II) approach to blind source separation (BSS). I... more In this paper we present an iterative inversion (II) approach to blind source separation (BSS). It consists of a quasi-Newton method for the resolution of an estimating equation obtained from the implicit inversion of a robust estimate of the mixing system. The resulting learning rule includes several existing algorithms for BSS as particular cases giving them a novel and unified interpretation.It also provides a justification of the Cardoso and Laheld step size normalization . The II method is first presented for instantaneous mixtures and then extended to the problem of blind separation of convolutive mixtures. Finally, we derive the necessary and sufficient asymptotic stability conditions for both the instantaneous and convolutive methods to converge.
IEEE Transactions on Neural Networks, 2000
Nonnegative matrix factorization (NMF) is an unsupervised learning method useful in various appli... more Nonnegative matrix factorization (NMF) is an unsupervised learning method useful in various applications including image processing and semantic analysis of documents. This paper focuses on symmetric NMF (SNMF), which is a special case of NMF decomposition. Three parallel multiplicative update algorithms using level 3 basic linear algebra subprograms directly are developed for this problem. First, by minimizing the Euclidean distance, a multiplicative update algorithm is proposed, and its convergence under mild conditions is proved. Based on it, we further propose another two fast parallel methods: α-SNMF and β-SNMF algorithms. All of them are easy to implement. These algorithms are applied to probabilistic clustering. We demonstrate their effectiveness for facial image clustering, document categorization, and pattern clustering in gene expression.
IEEE Transactions on Neural Networks, 2004
This paper reports a study on the problem of the blind simultaneous extraction of specific groups... more This paper reports a study on the problem of the blind simultaneous extraction of specific groups of independent components from a linear mixture. This paper first presents a general overview and unification of several information theoretic criteria for the extraction of a single independent component. Then, our contribution fills the theoretical gap that exists between extraction and separation by presenting tools that extend these criteria to allow the simultaneous blind extraction of subsets with an arbitrary number of independent components. In addition, we analyze a family of learning algorithms based on Stiefel manifolds and the natural gradient ascent, present the nonlinear optimal activations (score) functions, and provide new or extended local stability conditions. Finally, we illustrate the performance and features of the proposed approach by computer-simulation experiments.
IEEE Transactions on Neural Networks, 2005
In this letter, we solve the problem of identifying matrices S and A knowing only their multiplic... more In this letter, we solve the problem of identifying matrices S and A knowing only their multiplication X = AS, under some conditions, expressed either in terms of A and sparsity of S (identifiability conditions), or in terms of X (sparse component analysis (SCA) conditions). We present algorithms for such identification and illustrate them by examples.
Simplified neural networks for solving linear least squares and total least squares problems in real time
IEEE Transactions on Neural Networks, 1994
In this paper a new class of simplified low-cost analog artificial neural networks with on chip a... more In this paper a new class of simplified low-cost analog artificial neural networks with on chip adaptive learning algorithms are proposed for solving linear systems of algebraic equations in real time. The proposed learning algorithms for linear least squares (LS), total least squares (TLS) and data least squares (DLS) problems can be considered as modifications and extensions of well known algorithms: the row-action projection-Kaczmarz algorithm and/or the LMS (Adaline) Widrow-Hoff algorithms. The algorithms can be applied to any problem which can be formulated as a linear regression problem. The correctness and high performance of the proposed neural networks are illustrated by extensive computer simulation results.
Equivalence Probability and Sparsity of Two Sparse Solutions in Sparse Representation
IEEE Transactions on Neural Networks, 2000
This paper discusses the estimation and numerical calculation of the probability that the 0-norm ... more This paper discusses the estimation and numerical calculation of the probability that the 0-norm and 1-norm solutions of underdetermined linear equations are equivalent in the case of sparse representation. First, we define the sparsity degree of a signal. Two equivalence probability estimates are obtained when the entries of the 0-norm solution have different sparsity degrees. One is for the case in which the basis matrix is given or estimated, and the other is for the case in which the basis matrix is random. However, the computational burden to calculate these probabilities increases exponentially as the number of columns of the basis matrix increases. This computational complexity problem can be avoided through a sampling method. Next, we analyze the sparsity degree of mixtures and establish the relationship between the equivalence probability and the sparsity degree of the mixtures. This relationship can be used to analyze the performance of blind source separation (BSS). Furthermore, we extend the equivalence probability estimates to the small noise case. Finally, we illustrate how to use these theoretical results to guarantee a satisfactory performance in underdetermined BSS.
A switched-capacitor interface for capacitive sensors based on relaxation oscillators
IEEE Transactions on Instrumentation and Measurement, 1990
... to the unknown capacitance C, has been achieved to be di-rectly compatible with amicroprocess... more ... to the unknown capacitance C, has been achieved to be di-rectly compatible with amicroprocessor. ... and M. S. Beck, “A high frequency stray-immune capacitance transducer based on the ... and K. Watanabe, “A switched-capacitor charge-balancing analog-to-digital converter and ...
A novel switched-capacitor four-quadrant analog multiplier-divider and some of its applications
IEEE Transactions on Instrumentation and Measurement, 2000
In neurofeedback, brain waves are transformed into sounds or music, graphics, and other represent... more In neurofeedback, brain waves are transformed into sounds or music,
graphics, and other representations, to provide real-time information on ongoing
waves and patterns in the brain. Here we present various forms of neurofeedback,
including sonification, sonification in combination with visualization, and at last,
immersive neurofeedback, where auditory and visual feedback is provided in a
multi-sided immersive environment in which participants are completely surrounded by virtual imagery and 3D sound. Neural feedback may potentially
improve the user’s (or patient’s) ability to control brain activity, the diagnosis of
medical conditions, and the rehabilitation of neurological or psychiatric disorders.
Several psychological and medical studies have confirmed that virtual immersive
activity is enjoyable, stimulating, and can have a healing effect. As an illustration,
neurofeedback is generated from electroencephalograms (EEG) of Alzheimer’s
disease (AD) patients and healthy subjects. The auditory, visual, and immersive
representations of Alzheimer’s EEG differ substantially from healthy EEG, potentially
yielding novel diagnostic tools. Moreover, such alternative representations of
AD EEG are natural and intuitive, and hence easily accessible to laymen
(AD patients and family members)