Amit Bhaya - Profile on Academia.edu (original) (raw)
Papers by Amit Bhaya
A control Liapunov function approach to generalized and regularized descent methods for zero finding1
International Journal of Hybrid Intelligent Systems, 2014
ABSTRACT This paper revisits a class of recently proposed so-called invariant manifold methods fo... more ABSTRACT This paper revisits a class of recently proposed so-called invariant manifold methods for zero finding of ill-posed problems, showing that they can be profitably viewed as homotopy methods, in which the homotopy parameter is interpreted as a learning parameter. Moreover, it is shown that the choice of this learning parameter can be made in a natural manner from a control Liapunov function approach CLF. From this viewpoint, maintaining manifold invariance is equivalent to ensuring that the CLF satisfies a certain ordinary differential equation, involving the learning parameter, that allows an estimate of rate of convergence. In order to illustrate this approach, algorithms recently proposed using the invariant manifold approach, are rederived, via CLFs, in a unified manner. Adaptive regularization parameters for solving linear algebraic ill-posed problems were also proposed. This paper also shows that the discretizations of the ODEs to solve the zero finding problem, as well as the different adaptive choices of the regularization parameter, yield iterative methods for linear systems, which are also derived using the Liapunov optimizing control LOC method.
A Study of the Robustness of Iterative Methods for Linear Systems
AIP Conference Proceedings, 2009
Numerical methods are implemented in digital computers using finite precision arithmetics, in whi... more Numerical methods are implemented in digital computers using finite precision arithmetics, in which real∕ complex numbers are represented by finite length words. This representation results in truncating∕ rounding off the numbers, which leads to numerical errors in the algorithms. The numerical errors can result in the loss of some properties of the numerical methods (for example, the orthogonality of the residues of the conjugate gradient), which, in turn, cause numerical instability. In this paper, a new model of the perturbations resulting ...
Comments regarding (quote)On stability of interval matrices(quote) [with reply]
Real Matrices with Positive Determinant are Homotopic to the Identity
Siam Review, Jun 1, 1998
Page 1. REAL MATRICES WITH POSITIVE DETERMINANT ARE HOMOTOPIC TO THE IDENTITY ∗ AMIT BHAYA† SIAM ... more Page 1. REAL MATRICES WITH POSITIVE DETERMINANT ARE HOMOTOPIC TO THE IDENTITY ∗ AMIT BHAYA† SIAM REV. c 1998 Society for Industrial and Applied Mathematics Vol. 40, No. 2, pp. 335–340, June 1998 012 Abstract. The statement that is the title of this note is given a novel proof using the ideas of controllability and eigenvalue assignment from linear system theory. Key words. homotopy, degree theory, determinants, matrices, controllability, eigenvalue assignment AMS subject classifications. 15A15, 55M25, 93B05, 93B55 PII. ...
A control-theoretic view of diagonal preconditioners
Ijsysc, 1995
The condition number k (S) of a matrix S is the ratio of the largest singular value of S to the s... more The condition number k (S) of a matrix S is the ratio of the largest singular value of S to the smallest, and is a very important quantity in the sensitivity and convergence analysis of many problems in numerical linear algebra. The optimal condition number of a matrix S is the minimum, over all positive diagonal matrices P, of K;(PS). In this paper we interpret the problem of finding the optimal preconditioner P that minimizes k (PS) as the equivalent problem of maximally clustering the poles of a suitably defined dynamical system by the ...
Biosystems, Mar 31, 2007
This paper develops an algorithm that extracts explanatory rules from microarray data, which we t... more This paper develops an algorithm that extracts explanatory rules from microarray data, which we treat as time series, using genetic programming (GP) and fuzzy logic. Reverse polish notation is used (RPN) to describe the rules and to facilitate the GP approach. The algorithm also allows for the insertion of prior knowledge, making it possible to find sets of rules that include the relationships between genes already known. The algorithm proposed is applied to problems arising in the construction of gene regulatory networks, using two different sets of real data from biological experiments on the Arabidopsis thaliana cold response and the rat central nervous system, respectively. The results show that the proposed technique can fit data to a pre-defined precision even in situations where the data set has thousands of features but only a limited number of points in time are available, a situation in which traditional statistical alternatives encounter difficulties, due to the scarcity of time points.
Controlling plants with delay†
Http Dx Doi Org 10 1080 0020718508961165, May 21, 2007
ABSTRACT
Proceedings of Iscas 95 International Symposium on Circuits and Systems, Dec 20, 1995
This paper investigates a continuous-valued discrete-time analog of the wellknown continuous-valu... more This paper investigates a continuous-valued discrete-time analog of the wellknown continuous-valued continuous-time Hopfield neural network model, first proposed by Takeda and Goodman. It is shown that the assumption of D-stability of the interconnection matrix, together with the standard assumptions on the activation functions, guarantee a unique equilibrium under a synchronous mode of operation as well as a class of asynchronous modes. For the synchronous mode, these assumptions are also shown to imply local asymptotic stability of the equilibrium. For the asynchronous mode of operation, two results are derived. First, using results of Kleptsyn and coworkers, it is shown that symmetry and stability of the interconnection matrix guarantee local stability of the equilibrium under a class of asynchronous modes -this is referred to as local absolute stability. Second, using results of Bhaya and coworkers, it is shown that, under the standard assumptions, if the nonnegative matrix whose elements are the absolute values of the corresponding elements of the interconnection matrix is stable, then the equilibrium is globally absolutely asymptotically stable under a class of asynchronous modes. The results obtained are discussed both from the point of view of their robustness as well as their relationship to earlier results. *
Matrix Diagonal Stability in Systems and Computation
This book gives a presentation of new methods related to dynamical systems described by linear an... more This book gives a presentation of new methods related to dynamical systems described by linear and nonlinear ordinary differential equations and difference equations. Special attention is paid to dynamical systems that are open to analysis by the Liapunov approach. The material will interest researchers and professionals in control engineering or those working in scientific computation or the stability of dynamical systems.
Unified control Liapunov function based design of neural networks that aim at global minimization of nonconvex functions
2009 International Joint Conference on Neural Networks, Jun 14, 2009
Abstract This paper presents a unified approach to the design of neural networks that aim to mini... more Abstract This paper presents a unified approach to the design of neural networks that aim to minimize scalar nonconvex functions that have continuous first-and second-order derivatives and a unique global minimum. The approach is based on interpreting the function as a controlled object, namely one that has an output (the function value) that has to be driven to its smallest value by suitable manipulation of its inputs: this is achieved by the use of the control Liapunov function (CLF) technique, well known in systems and control ...
Isnn, 2008
The problem of restoring images degraded by linear position invariant distortions and noise is so... more The problem of restoring images degraded by linear position invariant distortions and noise is solved by means of a L1-norm regularization, which is equivalent to determining a L1norm solution of an overdetermined system of linear equations, which results from a data-fitting term plus a regularization term that are both in L1 norm. This system is solved by means of a gradient-based neural network with a discontinuous activation function, which is ensured to converge to a L1-norm solution of the corresponding system of linear equations.
Automatica, 1992
Abs~-~ct--The design of decentralized integral tuning regulators (DTR) for unknown MIMO linear ti... more Abs~-~ct--The design of decentralized integral tuning regulators (DTR) for unknown MIMO linear time-invariant plants is considered. It is assumed that the dc gain matrix of the plant can be measured. Then, an algorithm for determining a sequence for implementing the local controllers is presented, according to the sensitivities of the critical poles (with respect to the tuning parameters) introduced by the integral controllers. The algorithm also calculates the static precompensators for the subsystems. Tuning is made automatic by means of an autotuner. This results in a fully automated DTR synthesis procedure.
M�todos iterativos lineares projetados atrav�s da teoria de controle e suas aplica��es
Neurocomputing, Oct 1, 2009
This paper presents two neural networks to find the optimal point in convex optimization problems... more This paper presents two neural networks to find the optimal point in convex optimization problems and variational inequality problems, respectively. The domain of the functions that define the problems is a convex set, which is determined by convex inequality constraints and affine equality constraints. The neural networks are based on gradient descent and exact penalization and the convergence analysis is based on a control Liapunov function analysis, since the dynamical system corresponding to each neural network may be viewed as a so-called variable structure closed loop control system.
Neural Networks, 2004
It is pointed out that the so called momentum method, much used in the neural network literature ... more It is pointed out that the so called momentum method, much used in the neural network literature as an acceleration of the backpropagation method, is a stationary version of the conjugate gradient method. Connections with the continuous optimization method known as heavy ball with friction are also made. In both cases, adaptive (dynamic) choices of the so called learning rate and momentum parameters are obtained using a control Liapunov function analysis of the system. q
Conjugate gradient and steepest descent constant modulus algorithms applied to a blind adaptive array
Signal Processing, Oct 1, 2010
Multi-user mobile communication systems use adaptive and linearly constrained adaptive filters fo... more Multi-user mobile communication systems use adaptive and linearly constrained adaptive filters for blind and non-blind adaptive interference cancelation, multipath reduction, equalization, and adaptive beamforming. A conjugate gradient and a steepest descent method for real-time processing are proposed and applied to blind adaptive array processor. Simulations show that the proposed algorithms have performance comparable to those of algorithms proposed earlier.
Cooperative parallel asynchronous computation of the solution of symmetric linear systems
49Th Ieee Conference on Decision and Control, 2010
Abstract This paper introduces a new paradigm, called cooperative computation, for the solution o... more Abstract This paper introduces a new paradigm, called cooperative computation, for the solution of systems of linear equations with symmetric coefficient matrices. The simplest version of the algorithm consists of two agents, each one computing the solution of the whole system, using an iterative method. Infrequent unidirectional communication occurs from one agent to the other, either periodically, or probabilistically, thus characterizing the computation as parallel and asynchronous. Every time one agent communicates its ...
Matrix Forms of Gradient Descent Algorithms Applied to Restoration of Blurred Images
International Journal of Signal Processing, Image Processing and Pattern Recognition, 2014
2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), 2008
The problem of restoring images degraded by linear position invariant distortions and noise is so... more The problem of restoring images degraded by linear position invariant distortions and noise is solved by means of a L1-norm regularization, which is equivalent to determining a L1norm solution of an overdetermined system of linear equations, which results from a data-fitting term plus a regularization term that are both in L1 norm. This system is solved by means of a gradient-based neural network with a discontinuous activation function, which is ensured to converge to a L1-norm solution of the corresponding system of linear equations.
Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290), 2000
Artificial neural networks for solving different variants of linear programming problems are prop... more Artificial neural networks for solving different variants of linear programming problems are proposed and analyzed through Liapunov direct method. An energy function with an exact penalty term is associated to each variant and leads to a discontinuous dynamic gradient system model of an artificial neural network.
A control Liapunov function approach to generalized and regularized descent methods for zero finding1
International Journal of Hybrid Intelligent Systems, 2014
ABSTRACT This paper revisits a class of recently proposed so-called invariant manifold methods fo... more ABSTRACT This paper revisits a class of recently proposed so-called invariant manifold methods for zero finding of ill-posed problems, showing that they can be profitably viewed as homotopy methods, in which the homotopy parameter is interpreted as a learning parameter. Moreover, it is shown that the choice of this learning parameter can be made in a natural manner from a control Liapunov function approach CLF. From this viewpoint, maintaining manifold invariance is equivalent to ensuring that the CLF satisfies a certain ordinary differential equation, involving the learning parameter, that allows an estimate of rate of convergence. In order to illustrate this approach, algorithms recently proposed using the invariant manifold approach, are rederived, via CLFs, in a unified manner. Adaptive regularization parameters for solving linear algebraic ill-posed problems were also proposed. This paper also shows that the discretizations of the ODEs to solve the zero finding problem, as well as the different adaptive choices of the regularization parameter, yield iterative methods for linear systems, which are also derived using the Liapunov optimizing control LOC method.
A Study of the Robustness of Iterative Methods for Linear Systems
AIP Conference Proceedings, 2009
Numerical methods are implemented in digital computers using finite precision arithmetics, in whi... more Numerical methods are implemented in digital computers using finite precision arithmetics, in which real∕ complex numbers are represented by finite length words. This representation results in truncating∕ rounding off the numbers, which leads to numerical errors in the algorithms. The numerical errors can result in the loss of some properties of the numerical methods (for example, the orthogonality of the residues of the conjugate gradient), which, in turn, cause numerical instability. In this paper, a new model of the perturbations resulting ...
Comments regarding (quote)On stability of interval matrices(quote) [with reply]
Real Matrices with Positive Determinant are Homotopic to the Identity
Siam Review, Jun 1, 1998
Page 1. REAL MATRICES WITH POSITIVE DETERMINANT ARE HOMOTOPIC TO THE IDENTITY ∗ AMIT BHAYA† SIAM ... more Page 1. REAL MATRICES WITH POSITIVE DETERMINANT ARE HOMOTOPIC TO THE IDENTITY ∗ AMIT BHAYA† SIAM REV. c 1998 Society for Industrial and Applied Mathematics Vol. 40, No. 2, pp. 335–340, June 1998 012 Abstract. The statement that is the title of this note is given a novel proof using the ideas of controllability and eigenvalue assignment from linear system theory. Key words. homotopy, degree theory, determinants, matrices, controllability, eigenvalue assignment AMS subject classifications. 15A15, 55M25, 93B05, 93B55 PII. ...
A control-theoretic view of diagonal preconditioners
Ijsysc, 1995
The condition number k (S) of a matrix S is the ratio of the largest singular value of S to the s... more The condition number k (S) of a matrix S is the ratio of the largest singular value of S to the smallest, and is a very important quantity in the sensitivity and convergence analysis of many problems in numerical linear algebra. The optimal condition number of a matrix S is the minimum, over all positive diagonal matrices P, of K;(PS). In this paper we interpret the problem of finding the optimal preconditioner P that minimizes k (PS) as the equivalent problem of maximally clustering the poles of a suitably defined dynamical system by the ...
Biosystems, Mar 31, 2007
This paper develops an algorithm that extracts explanatory rules from microarray data, which we t... more This paper develops an algorithm that extracts explanatory rules from microarray data, which we treat as time series, using genetic programming (GP) and fuzzy logic. Reverse polish notation is used (RPN) to describe the rules and to facilitate the GP approach. The algorithm also allows for the insertion of prior knowledge, making it possible to find sets of rules that include the relationships between genes already known. The algorithm proposed is applied to problems arising in the construction of gene regulatory networks, using two different sets of real data from biological experiments on the Arabidopsis thaliana cold response and the rat central nervous system, respectively. The results show that the proposed technique can fit data to a pre-defined precision even in situations where the data set has thousands of features but only a limited number of points in time are available, a situation in which traditional statistical alternatives encounter difficulties, due to the scarcity of time points.
Controlling plants with delay†
Http Dx Doi Org 10 1080 0020718508961165, May 21, 2007
ABSTRACT
Proceedings of Iscas 95 International Symposium on Circuits and Systems, Dec 20, 1995
This paper investigates a continuous-valued discrete-time analog of the wellknown continuous-valu... more This paper investigates a continuous-valued discrete-time analog of the wellknown continuous-valued continuous-time Hopfield neural network model, first proposed by Takeda and Goodman. It is shown that the assumption of D-stability of the interconnection matrix, together with the standard assumptions on the activation functions, guarantee a unique equilibrium under a synchronous mode of operation as well as a class of asynchronous modes. For the synchronous mode, these assumptions are also shown to imply local asymptotic stability of the equilibrium. For the asynchronous mode of operation, two results are derived. First, using results of Kleptsyn and coworkers, it is shown that symmetry and stability of the interconnection matrix guarantee local stability of the equilibrium under a class of asynchronous modes -this is referred to as local absolute stability. Second, using results of Bhaya and coworkers, it is shown that, under the standard assumptions, if the nonnegative matrix whose elements are the absolute values of the corresponding elements of the interconnection matrix is stable, then the equilibrium is globally absolutely asymptotically stable under a class of asynchronous modes. The results obtained are discussed both from the point of view of their robustness as well as their relationship to earlier results. *
Matrix Diagonal Stability in Systems and Computation
This book gives a presentation of new methods related to dynamical systems described by linear an... more This book gives a presentation of new methods related to dynamical systems described by linear and nonlinear ordinary differential equations and difference equations. Special attention is paid to dynamical systems that are open to analysis by the Liapunov approach. The material will interest researchers and professionals in control engineering or those working in scientific computation or the stability of dynamical systems.
Unified control Liapunov function based design of neural networks that aim at global minimization of nonconvex functions
2009 International Joint Conference on Neural Networks, Jun 14, 2009
Abstract This paper presents a unified approach to the design of neural networks that aim to mini... more Abstract This paper presents a unified approach to the design of neural networks that aim to minimize scalar nonconvex functions that have continuous first-and second-order derivatives and a unique global minimum. The approach is based on interpreting the function as a controlled object, namely one that has an output (the function value) that has to be driven to its smallest value by suitable manipulation of its inputs: this is achieved by the use of the control Liapunov function (CLF) technique, well known in systems and control ...
Isnn, 2008
The problem of restoring images degraded by linear position invariant distortions and noise is so... more The problem of restoring images degraded by linear position invariant distortions and noise is solved by means of a L1-norm regularization, which is equivalent to determining a L1norm solution of an overdetermined system of linear equations, which results from a data-fitting term plus a regularization term that are both in L1 norm. This system is solved by means of a gradient-based neural network with a discontinuous activation function, which is ensured to converge to a L1-norm solution of the corresponding system of linear equations.
Automatica, 1992
Abs~-~ct--The design of decentralized integral tuning regulators (DTR) for unknown MIMO linear ti... more Abs~-~ct--The design of decentralized integral tuning regulators (DTR) for unknown MIMO linear time-invariant plants is considered. It is assumed that the dc gain matrix of the plant can be measured. Then, an algorithm for determining a sequence for implementing the local controllers is presented, according to the sensitivities of the critical poles (with respect to the tuning parameters) introduced by the integral controllers. The algorithm also calculates the static precompensators for the subsystems. Tuning is made automatic by means of an autotuner. This results in a fully automated DTR synthesis procedure.
M�todos iterativos lineares projetados atrav�s da teoria de controle e suas aplica��es
Neurocomputing, Oct 1, 2009
This paper presents two neural networks to find the optimal point in convex optimization problems... more This paper presents two neural networks to find the optimal point in convex optimization problems and variational inequality problems, respectively. The domain of the functions that define the problems is a convex set, which is determined by convex inequality constraints and affine equality constraints. The neural networks are based on gradient descent and exact penalization and the convergence analysis is based on a control Liapunov function analysis, since the dynamical system corresponding to each neural network may be viewed as a so-called variable structure closed loop control system.
Neural Networks, 2004
It is pointed out that the so called momentum method, much used in the neural network literature ... more It is pointed out that the so called momentum method, much used in the neural network literature as an acceleration of the backpropagation method, is a stationary version of the conjugate gradient method. Connections with the continuous optimization method known as heavy ball with friction are also made. In both cases, adaptive (dynamic) choices of the so called learning rate and momentum parameters are obtained using a control Liapunov function analysis of the system. q
Conjugate gradient and steepest descent constant modulus algorithms applied to a blind adaptive array
Signal Processing, Oct 1, 2010
Multi-user mobile communication systems use adaptive and linearly constrained adaptive filters fo... more Multi-user mobile communication systems use adaptive and linearly constrained adaptive filters for blind and non-blind adaptive interference cancelation, multipath reduction, equalization, and adaptive beamforming. A conjugate gradient and a steepest descent method for real-time processing are proposed and applied to blind adaptive array processor. Simulations show that the proposed algorithms have performance comparable to those of algorithms proposed earlier.
Cooperative parallel asynchronous computation of the solution of symmetric linear systems
49Th Ieee Conference on Decision and Control, 2010
Abstract This paper introduces a new paradigm, called cooperative computation, for the solution o... more Abstract This paper introduces a new paradigm, called cooperative computation, for the solution of systems of linear equations with symmetric coefficient matrices. The simplest version of the algorithm consists of two agents, each one computing the solution of the whole system, using an iterative method. Infrequent unidirectional communication occurs from one agent to the other, either periodically, or probabilistically, thus characterizing the computation as parallel and asynchronous. Every time one agent communicates its ...
Matrix Forms of Gradient Descent Algorithms Applied to Restoration of Blurred Images
International Journal of Signal Processing, Image Processing and Pattern Recognition, 2014
2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), 2008
The problem of restoring images degraded by linear position invariant distortions and noise is so... more The problem of restoring images degraded by linear position invariant distortions and noise is solved by means of a L1-norm regularization, which is equivalent to determining a L1norm solution of an overdetermined system of linear equations, which results from a data-fitting term plus a regularization term that are both in L1 norm. This system is solved by means of a gradient-based neural network with a discontinuous activation function, which is ensured to converge to a L1-norm solution of the corresponding system of linear equations.
Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290), 2000
Artificial neural networks for solving different variants of linear programming problems are prop... more Artificial neural networks for solving different variants of linear programming problems are proposed and analyzed through Liapunov direct method. An energy function with an exact penalty term is associated to each variant and leads to a discontinuous dynamic gradient system model of an artificial neural network.