Dianhui Wang | La Trobe University (original) (raw)
Papers by Dianhui Wang
Neurocomputing, Jun 1, 2013
This paper aims to develop a neural controller using the vectorial backstepping technique for dyn... more This paper aims to develop a neural controller using the vectorial backstepping technique for dynamically positioned surface ships with uncertainties and unknown disturbances. The radial basis function networks are employed to compensate for the uncertainties of ship dynamics and disturbances in controller design. The advantage of the proposed control scheme is that there is no requirement of any priori knowledge about dynamics of ships and disturbances. It is shown that our proposed control law can regulate the position and heading of ships to the desired targets with arbitrarily small positioning error. Theoretical results on stability analysis indicate that our proposed controller guarantees uniformly ultimate boundedness of all signals of the closed-loop system. Simulation studies with comparisons on a supply ship are carried out, and results demonstrate the effectiveness of the proposed control scheme.
ArXiv, 2018
Stochastic configuration networks (SCNs) as a class of randomized learner model have been success... more Stochastic configuration networks (SCNs) as a class of randomized learner model have been successfully employed in data analytics due to its universal approximation capability and fast modelling property. The technical essence lies in stochastically configuring hidden nodes (or basis functions) based on a supervisory mechanism rather than data-independent randomization as usually adopted for building randomized neural networks. Given image data modelling tasks, the use of one-dimensional SCNs potentially demolishes the spatial information of images, and may result in undesirable performance. This paper extends the original SCNs to two-dimensional version, termed 2DSCNs, for fast building randomized learners with matrix-inputs. Some theoretical analyses on the goodness of 2DSCNs against SCNs, including the complexity of the random parameter space, and the superiority of generalization, are presented. Empirical results over one regression, four benchmark handwritten digits classificat...
Journal of IT in Asia, 2017
This paper aims at developing techniqus for design and implementation of neural classifiers. Base... more This paper aims at developing techniqus for design and implementation of neural classifiers. Based on our previous study on generalized RBF neural network architecture and learning criterion function for parameter optimization, this work addresses two realization issues, i.e. supervised input features selection and genetic computation techniques for tuning classifiers. A comparative study on classifiation performance is carried on by a set of protein sequence data.
IEEE Transactions on Cybernetics, 2017
Random vector functional-link (RVFLs) networks are randomized multi-layer perceptrons (MLPs) with... more Random vector functional-link (RVFLs) networks are randomized multi-layer perceptrons (MLPs) with a single hidden layer and a linear output layer, which can be trained by solving a linear modeling problem. In particular, they are generally trained using a closed-form solution of the (regularized) least-squares approach. This paper introduces several alternative strategies for performing full Bayesian inference (BI) of RVFL networks. Distinct from standard or classical approaches, our proposed Bayesian training algorithms allow to derive an entire probability distribution over the optimal output weights of the network, instead of a single pointwise estimate according to some given criterion (e.g., least-squares). This provides several known advantages, including the possibility of introducing additional prior knowledge in the training process, the availability of an uncertainty measure during the test phase, and the capability of automatically inferring hyper-parameters from given data. In this paper two BI algorithms for regression are firstly proposed that, under some practical assumptions, can be implemented by a simple iterative process with closed-form computations. Simulation results show that one of the proposed algorithms, B-RVFL, is able to outperform standard training algorithms for RVFL networks with a proper regularization factor selected carefully via a line search procedure. A general strategy based on variational inference is also presented, with an application to data modeling problems with noisy outputs or outliers. As we discuss in the paper, using recent advances in automatic differentiation this strategy can be applied to a wide range of additional situations in an immediate fashion.
Information Sciences, 2017
Neural networks have been widely used as predictive models to fit data distribution, and they cou... more Neural networks have been widely used as predictive models to fit data distribution, and they could be implemented through learning a collection of samples. In many applications, however, the given dataset may contain noisy samples or outliers which may result in a poor learner model in terms of generalization. This paper contributes to a development of robust stochastic configuration networks (RSCNs) for resolving uncertain data regression problems. RSCNs are built on original stochastic configuration networks with weighted least squares method for evaluating the output weights, and the input weights and biases are incrementally and randomly generated by satisfying with a set of inequality constrains. The kernel density estimation (KDE) method is employed to set the penalty weights for each training samples, so that some negative impacts, caused by noisy data or outliers, on the resulting learner model can be reduced. The alternating optimization technique is applied for updating a RSCN model with improved penalty weights computed from the kernel density estimation function. Performance evaluation is carried out by a function approximation, four benchmark datasets and a case study on engineering application. Comparisons to other robust randomised neural modelling techniques, including the probabilistic robust learning algorithm for neural networks with random weights and improved RVFL networks, indicate that the proposed RSCNs with KDE perform favourably and demonstrate good potential for real-world applications.
IEEE transactions on cybernetics, Jan 21, 2017
This paper contributes to the development of randomized methods for neural networks. The proposed... more This paper contributes to the development of randomized methods for neural networks. The proposed learner model is generated incrementally by stochastic configuration (SC) algorithms, termed SC networks (SCNs). In contrast to the existing randomized learning algorithms for single layer feed-forward networks, we randomly assign the input weights and biases of the hidden nodes in the light of a supervisory mechanism, and the output weights are analytically evaluated in either a constructive or selective manner. As fundamentals of SCN-based data modeling techniques, we establish some theoretical results on the universal approximation property. Three versions of SC algorithms are presented for data regression and classification problems in this paper. Simulation results concerning both data regression and classification indicate some remarkable merits of our proposed SCNs in terms of less human intervention on the network size setting, the scope adaptation of random parameters, fast lea...
Information Sciences, 2017
Random Vector Functional-link (RVFL) networks, a class of learner models, can be regarded as feed... more Random Vector Functional-link (RVFL) networks, a class of learner models, can be regarded as feed-forward neural networks built with a specific randomized algorithm, i.e., the input weights and biases are randomly assigned and fixed during the training phase, and the output weights are analytically evaluated by the least square method. In this paper, we provide some insights into RVFL networks and highlight some practical issues and common pitfalls associated with RVFL-based modelling techniques. Inspired by the folklore that "all high-dimensional random vectors are almost always nearly orthogonal to each other", we establish a theoretical result on the infeasibility of RVFL networks for universal approximation, if a RVFL network is built incrementally with random selection of the input weights and biases from a fixed scope, and constructive evaluation of its output weights. This work also addresses the significance of the scope setting of random weights and biases in respect to modelling performance. Two numerical examples are employed to illustrate our findings, which theoretically and empirically reveal some facts and limits of such class of randomized learning algorithms.
Understanding protein-DNA binding affinity is still a mystery for many transcription factors (TFs... more Understanding protein-DNA binding affinity is still a mystery for many transcription factors (TFs). Although several approaches have been proposed in the literature to model the DNA-binding specificity of TFs, they still have some limitations. Most of the methods require a cut-off threshold in order to classify a K-mer as a binding site (BS) and finding such a threshold is usually done by handcraft rather than a science. Some other approaches use a prior knowledge on the biological context of regulatory elements in the genome along with machine learning algorithms to build classifier models for TFBSs. Noticeably, these methods deliberately select the training and testing datasets so that they are very separable. Hence, the current methods do not actually capture the TF-DNA binding relationship. In this paper, we present a threshold-free framework based on a novel ensemble learning algorithm in order to locate TFBSs in DNA sequences. Our proposed approach creates TF-specific classifi...
Transcriptional regulation mainly controls how genes are expressed and how cells behave based on ... more Transcriptional regulation mainly controls how genes are expressed and how cells behave based on the transcription factor (TF) proteins that bind upstream of the transcription start sites (TSSs) of genes. These TF DNA binding sites (TFBSs) are usually short (5-15 base pairs) and degenerate (some positions can have multiple possible alternatives). Traditionally, computational methods scan DNA sequences using the position weight matrix (PWM) of a given TF, calculate binding scores for each K-mer against the PWM, and finally classify a K-mer as to whether it is a putative TFBS or a background sequence based on a cut-off threshold. The FSCAN system, which is proposed in this paper, employs machine learning techniques to build a learner model that is able to identify TFBSs in a set of bound sequences without the need for a cut-off threshold. Our proposed method utilizes fuzzy inference techniques along with a distribution-based filtering algorithm to predict the binding sites of a TF giv...
2015 International Joint Conference on Neural Networks (IJCNN), 2015
In this paper, we investigate the problem of music classification when training data is distribut... more In this paper, we investigate the problem of music classification when training data is distributed throughout a network of interconnected agents (e.g. computers, or mobile devices), and it is available in a sequential stream. Under the considered setting, the task is for all the nodes, after receiving any new chunk of training data, to agree on a single classifier in a decentralized fashion, without reliance on a master node. In particular, in this paper we propose a fully decentralized, sequential learning algorithm for a class of neural networks known as Random Vector Functional-Link nets. The proposed algorithm does not require the presence of a single coordinating agent, and it is formulated exclusively in term of local exchanges between neighboring nodes, thus making it useful in a wide range of realistic situations. Experimental simulations on four music classification benchmarks show that the algorithm has comparable performance with respect to a centralized solution, where a single agent collects all the local data from every node and subsequently updates the model.
Information Sciences, 2014
Negative correlation learning (NCL) aims to produce ensembles with sound generalization capabilit... more Negative correlation learning (NCL) aims to produce ensembles with sound generalization capability through controlling the disagreement among base learners' outputs. Such a learning scheme is usually implemented by using feed-forward neural networks with error back-propagation algorithms (BPNNs). However, it suffers from slow convergence, local minima problem and model uncertainties caused by the initial weights and the setting of learning parameters. To achieve a better solution, this paper employs the random vector functional link (RVFL) networks as base components, and incorporates with the NCL strategy for building neural network ensembles. The basis functions of the base models are generated randomly and the parameters of the RVFL networks can be determined by solving a linear equation system. An analytical solution is derived for these parameters, where a cost function defined for NCL and the well-known least squares method are used. To examine the merits of our proposed algorithm, a comparative study is carried out with nine benchmark datasets. Results indicate that our approach outperforms other ensembling techniques on the testing datasets in terms of both effectiveness and efficiency.
BMC Bioinformatics, 2011
Background: Discrimination of transcription factor binding sites (TFBS) from background sequences... more Background: Discrimination of transcription factor binding sites (TFBS) from background sequences plays a key role in computational motif discovery. Current clustering based algorithms employ homogeneous model for problem solving, which assumes that motifs and background signals can be equivalently characterized. This assumption has some limitations because both sequence signals have distinct properties. Results: This paper aims to develop a Self-Organizing Map (SOM) based clustering algorithm for extracting binding sites in DNA sequences. Our framework is based on a novel intra-node soft competitive procedure to achieve maximum discrimination of motifs from background signals in datasets. The intra-node competition is based on an adaptive weighting technique on two different signal models to better represent these two classes of signals. Using several real and artificial datasets, we compared our proposed method with several motif discovery tools. Compared to SOMBRERO, a state-of-the-art SOM based motif discovery tool, it is found that our algorithm can achieve significant improvements in the average precision rates (i.e., about 27%) on the real datasets without compromising its sensitivity. Our method also performed favourably comparing against other motif discovery tools. Conclusions: Motif discovery with model based clustering framework should consider the use of heterogeneous model to represent the two classes of signals in DNA sequences. Such heterogeneous model can achieve better signal discrimination compared to the homogeneous model.
IEEE transactions on cybernetics, Jul 1, 2018
Random vector functional-link (RVFLs) networks are randomized multi-layer perceptrons (MLPs) with... more Random vector functional-link (RVFLs) networks are randomized multi-layer perceptrons (MLPs) with a single hidden layer and a linear output layer, which can be trained by solving a linear modeling problem. In particular, they are generally trained using a closed-form solution of the (regularized) least-squares approach. This paper introduces several alternative strategies for performing full Bayesian inference (BI) of RVFL networks. Distinct from standard or classical approaches, our proposed Bayesian training algorithms allow to derive an entire probability distribution over the optimal output weights of the network, instead of a single pointwise estimate according to some given criterion (e.g., least-squares). This provides several known advantages, including the possibility of introducing additional prior knowledge in the training process, the availability of an uncertainty measure during the test phase, and the capability of automatically inferring hyper-parameters from given data. In this paper two BI algorithms for regression are firstly proposed that, under some practical assumptions, can be implemented by a simple iterative process with closed-form computations. Simulation results show that one of the proposed algorithms, B-RVFL, is able to outperform standard training algorithms for RVFL networks with a proper regularization factor selected carefully via a line search procedure. A general strategy based on variational inference is also presented, with an application to data modeling problems with noisy outputs or outliers. As we discuss in the paper, using recent advances in automatic differentiation this strategy can be applied to a wide range of additional situations in an immediate fashion.
Wiley Interdisciplinary Reviews-Data Mining and Knowledge Discovery, Feb 9, 2017
Neural networks, as powerful tools for data mining and knowledge engineering, can learn from data... more Neural networks, as powerful tools for data mining and knowledge engineering, can learn from data to build feature-based classifiers and nonlinear predictive models. Training neural networks involves the optimization of non-convex objective functions, and usually the learning process is costly and infeasible for applications associated with data streams. A possible, albeit counter-intuitive alternative is to randomly assign a subset of the networks' weights, so that the resulting optimization task can be formulated as a linear least-squares problem. This methodology can be applied to both feedforward and recurrent networks, and similar techniques can be used to approximate kernel functions. Many experimental results indicate that such randomized models can reach sound performance compared to fully adaptable ones, with a number of favourable benefits, including (i) simplicity of implementation, (ii) faster learning with less intervention from human beings, and (iii) possibility of leveraging over all linear regression and classification algorithms (e.g., ℓ 1 norm minimization for obtaining sparse formulations). All these points make them attractive and valuable to the data mining community, particularly for handling large scale data mining in real-time. However, the literature in the field is extremely vast and fragmented, with many results being reintroduced multiple times under different names. This overview aims at providing a self-contained, uniform introduction to the different ways in which randomization can be applied to the design of neural networks and kernel functions. A clear exposition of the basic framework underlying all these approaches helps to clarify innovative lines of research, open problems and, most importantly, foster the exchanges of well-known results throughout different communities.
Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 2017
Neural networks, as powerful tools for data mining and knowledge engineering, can learn from data... more Neural networks, as powerful tools for data mining and knowledge engineering, can learn from data to build feature-based classifiers and nonlinear predictive models. Training neural networks involves the optimization of non-convex objective functions, and usually the learning process is costly and infeasible for applications associated with data streams. A possible, albeit counter-intuitive alternative is to randomly assign a subset of the networks' weights, so that the resulting optimization task can be formulated as a linear least-squares problem. This methodology can be applied to both feedforward and recurrent networks, and similar techniques can be used to approximate kernel functions. Many experimental results indicate that such randomized models can reach sound performance compared to fully adaptable ones, with a number of favourable benefits, including (i) simplicity of implementation, (ii) faster learning with less intervention from human beings, and (iii) possibility of leveraging over all linear regression and classification algorithms (e.g., ℓ 1 norm minimization for obtaining sparse formulations). All these points make them attractive and valuable to the data mining community, particularly for handling large scale data mining in real-time. However, the literature in the field is extremely vast and fragmented, with many results being reintroduced multiple times under different names. This overview aims at providing a self-contained, uniform introduction to the different ways in which randomization can be applied to the design of neural networks and kernel functions. A clear exposition of the basic framework underlying all these approaches helps to clarify innovative lines of research, open problems and, most importantly, foster the exchanges of well-known results throughout different communities.
Information Sciences, 2015
This paper aims to develop distributed learning algorithms for Random Vector Functional-Link (RVF... more This paper aims to develop distributed learning algorithms for Random Vector Functional-Link (RVFL) networks, where training data is distributed under a decentralized information structure. Two algorithms are proposed by using Decentralized Average Consensus (DAC) and Alternating Direction Method of Multipliers (ADMM) strategies, respectively. These algorithms work in a fully distributed fashion and have no requirement on coordination from a central agent during the learning process. For distributed learning, the goal is to build a common learner model which optimizes the system performance over the whole set of local data. In this work, it is assumed that all stations know the initial weights of the input layer, the output weights of local RVFL networks can be shared through communication channels among neighboring nodes only, and local datasets are blocked strictly. The proposed learning algorithms are evaluated over five benchmark datasets. Experimental results with comparisons show that the DAC-based learning algorithm performs favorably in terms of effectiveness, efficiency and computational complexity, followed by the ADMM-based learning algorithm with promising accuracy but higher computational burden.
Neural Networks, 2016
The current big data deluge requires innovative solutions for performing efficient inference on l... more The current big data deluge requires innovative solutions for performing efficient inference on large, heterogeneous amounts of information. Apart from the known challenges deriving from high volume and velocity, real-world big data applications may impose additional technological constraints, including
Traditionally, two protein sequences are classified into the same class if their feature patterns... more Traditionally, two protein sequences are classified into the same class if their feature patterns have high homology. These feature patterns were originally extracted by sequence alignment algorithms, which measure similarity between an unseen protein sequence and identified protein sequences. Neural network approaches, while reasonably accurate at classification, give no information about the relationship between the unseen case and the classified items that is useful to biologist. In contrast, in this paper we use a generalized radial basis function (GRBF) neural network architecture that generates fuzzy classification rules that could be used for further knowledge discovery. Our proposed techniques were evaluated using protein sequences with ten classes of super-families downloaded from a public domain database, and the results compared favorably with other standard machine learning techniques.
IEEE Transactions on Systems, Man, and Cybernetics: Systems
Clinker free lime (f-CaO) content plays a crucial role in determining the quality of cement. Howe... more Clinker free lime (f-CaO) content plays a crucial role in determining the quality of cement. However, the existing methods are mainly based on laboratory analysis and with significant time delays, which makes the closed-loop control of f-CaO content impossible. In this paper, a multisource data ensemble learning-based soft sensor model is developed for online estimation of clinker f-CaO content. To build such a soft sensor model, input flame images, process variables, and the corresponding output f-CaO content data for a rotary cement kiln were collected from No. 2 rotary kiln at Jiuganghongda Cement Plant which produces 2000 tonnes of clinker per day. The raw data were preprocessed to distinguish the flame image regions of interest (ROI) and remove process variable outliers. Three types of flame image ROI features, i.e., color, global configuration, and local configuration features, were then extracted without segmentation. Further, a kernel partial least square technique was appli...
IEEE Transactions on Neural Networks and Learning Systems, 2016
SOM-based motif mining, despite being a promising approach for problem solving, mostly fails to o... more SOM-based motif mining, despite being a promising approach for problem solving, mostly fails to offer a consistent interpretation of clusters in respect to the mixed composition of signal and noise in the nodes. The main reason behind this shortcoming comes from the similarity metrics used in data assignment, specially designed with the biological interpretation for this domain, are not meant to consider the inevitable noise mixture in the clusters. This limits the explicability of the majority of clusters that are supposedly noise dominated, degrading the overall system clarity in motif discovery. This paper aims to improve the explicability aspect of learning process by introducing a Composite Similarity Function (CSF) that is specially designed for the k-mer-to-cluster similarity measure in respect to the degree of motif properties and embedded noise in the cluster. Our proposed motif finding algorithm in this paper is built on our previous work READ [1] and termed as READ csf , that performs slightly better than READ and shows some remarkable improvements over SOM-based SOMBRERO and SOMEA tools respectively in terms of F-measure on the testing datasets. A real dataset containing multiple motifs is used to explore the potential of the READ csf for more challenging biological data mining tasks. Visual comparisons with the verified logos extracted from JASPAR database demonstrate that our algorithm is promising to discover multiple motifs simultaneously.
Neurocomputing, Jun 1, 2013
This paper aims to develop a neural controller using the vectorial backstepping technique for dyn... more This paper aims to develop a neural controller using the vectorial backstepping technique for dynamically positioned surface ships with uncertainties and unknown disturbances. The radial basis function networks are employed to compensate for the uncertainties of ship dynamics and disturbances in controller design. The advantage of the proposed control scheme is that there is no requirement of any priori knowledge about dynamics of ships and disturbances. It is shown that our proposed control law can regulate the position and heading of ships to the desired targets with arbitrarily small positioning error. Theoretical results on stability analysis indicate that our proposed controller guarantees uniformly ultimate boundedness of all signals of the closed-loop system. Simulation studies with comparisons on a supply ship are carried out, and results demonstrate the effectiveness of the proposed control scheme.
ArXiv, 2018
Stochastic configuration networks (SCNs) as a class of randomized learner model have been success... more Stochastic configuration networks (SCNs) as a class of randomized learner model have been successfully employed in data analytics due to its universal approximation capability and fast modelling property. The technical essence lies in stochastically configuring hidden nodes (or basis functions) based on a supervisory mechanism rather than data-independent randomization as usually adopted for building randomized neural networks. Given image data modelling tasks, the use of one-dimensional SCNs potentially demolishes the spatial information of images, and may result in undesirable performance. This paper extends the original SCNs to two-dimensional version, termed 2DSCNs, for fast building randomized learners with matrix-inputs. Some theoretical analyses on the goodness of 2DSCNs against SCNs, including the complexity of the random parameter space, and the superiority of generalization, are presented. Empirical results over one regression, four benchmark handwritten digits classificat...
Journal of IT in Asia, 2017
This paper aims at developing techniqus for design and implementation of neural classifiers. Base... more This paper aims at developing techniqus for design and implementation of neural classifiers. Based on our previous study on generalized RBF neural network architecture and learning criterion function for parameter optimization, this work addresses two realization issues, i.e. supervised input features selection and genetic computation techniques for tuning classifiers. A comparative study on classifiation performance is carried on by a set of protein sequence data.
IEEE Transactions on Cybernetics, 2017
Random vector functional-link (RVFLs) networks are randomized multi-layer perceptrons (MLPs) with... more Random vector functional-link (RVFLs) networks are randomized multi-layer perceptrons (MLPs) with a single hidden layer and a linear output layer, which can be trained by solving a linear modeling problem. In particular, they are generally trained using a closed-form solution of the (regularized) least-squares approach. This paper introduces several alternative strategies for performing full Bayesian inference (BI) of RVFL networks. Distinct from standard or classical approaches, our proposed Bayesian training algorithms allow to derive an entire probability distribution over the optimal output weights of the network, instead of a single pointwise estimate according to some given criterion (e.g., least-squares). This provides several known advantages, including the possibility of introducing additional prior knowledge in the training process, the availability of an uncertainty measure during the test phase, and the capability of automatically inferring hyper-parameters from given data. In this paper two BI algorithms for regression are firstly proposed that, under some practical assumptions, can be implemented by a simple iterative process with closed-form computations. Simulation results show that one of the proposed algorithms, B-RVFL, is able to outperform standard training algorithms for RVFL networks with a proper regularization factor selected carefully via a line search procedure. A general strategy based on variational inference is also presented, with an application to data modeling problems with noisy outputs or outliers. As we discuss in the paper, using recent advances in automatic differentiation this strategy can be applied to a wide range of additional situations in an immediate fashion.
Information Sciences, 2017
Neural networks have been widely used as predictive models to fit data distribution, and they cou... more Neural networks have been widely used as predictive models to fit data distribution, and they could be implemented through learning a collection of samples. In many applications, however, the given dataset may contain noisy samples or outliers which may result in a poor learner model in terms of generalization. This paper contributes to a development of robust stochastic configuration networks (RSCNs) for resolving uncertain data regression problems. RSCNs are built on original stochastic configuration networks with weighted least squares method for evaluating the output weights, and the input weights and biases are incrementally and randomly generated by satisfying with a set of inequality constrains. The kernel density estimation (KDE) method is employed to set the penalty weights for each training samples, so that some negative impacts, caused by noisy data or outliers, on the resulting learner model can be reduced. The alternating optimization technique is applied for updating a RSCN model with improved penalty weights computed from the kernel density estimation function. Performance evaluation is carried out by a function approximation, four benchmark datasets and a case study on engineering application. Comparisons to other robust randomised neural modelling techniques, including the probabilistic robust learning algorithm for neural networks with random weights and improved RVFL networks, indicate that the proposed RSCNs with KDE perform favourably and demonstrate good potential for real-world applications.
IEEE transactions on cybernetics, Jan 21, 2017
This paper contributes to the development of randomized methods for neural networks. The proposed... more This paper contributes to the development of randomized methods for neural networks. The proposed learner model is generated incrementally by stochastic configuration (SC) algorithms, termed SC networks (SCNs). In contrast to the existing randomized learning algorithms for single layer feed-forward networks, we randomly assign the input weights and biases of the hidden nodes in the light of a supervisory mechanism, and the output weights are analytically evaluated in either a constructive or selective manner. As fundamentals of SCN-based data modeling techniques, we establish some theoretical results on the universal approximation property. Three versions of SC algorithms are presented for data regression and classification problems in this paper. Simulation results concerning both data regression and classification indicate some remarkable merits of our proposed SCNs in terms of less human intervention on the network size setting, the scope adaptation of random parameters, fast lea...
Information Sciences, 2017
Random Vector Functional-link (RVFL) networks, a class of learner models, can be regarded as feed... more Random Vector Functional-link (RVFL) networks, a class of learner models, can be regarded as feed-forward neural networks built with a specific randomized algorithm, i.e., the input weights and biases are randomly assigned and fixed during the training phase, and the output weights are analytically evaluated by the least square method. In this paper, we provide some insights into RVFL networks and highlight some practical issues and common pitfalls associated with RVFL-based modelling techniques. Inspired by the folklore that "all high-dimensional random vectors are almost always nearly orthogonal to each other", we establish a theoretical result on the infeasibility of RVFL networks for universal approximation, if a RVFL network is built incrementally with random selection of the input weights and biases from a fixed scope, and constructive evaluation of its output weights. This work also addresses the significance of the scope setting of random weights and biases in respect to modelling performance. Two numerical examples are employed to illustrate our findings, which theoretically and empirically reveal some facts and limits of such class of randomized learning algorithms.
Understanding protein-DNA binding affinity is still a mystery for many transcription factors (TFs... more Understanding protein-DNA binding affinity is still a mystery for many transcription factors (TFs). Although several approaches have been proposed in the literature to model the DNA-binding specificity of TFs, they still have some limitations. Most of the methods require a cut-off threshold in order to classify a K-mer as a binding site (BS) and finding such a threshold is usually done by handcraft rather than a science. Some other approaches use a prior knowledge on the biological context of regulatory elements in the genome along with machine learning algorithms to build classifier models for TFBSs. Noticeably, these methods deliberately select the training and testing datasets so that they are very separable. Hence, the current methods do not actually capture the TF-DNA binding relationship. In this paper, we present a threshold-free framework based on a novel ensemble learning algorithm in order to locate TFBSs in DNA sequences. Our proposed approach creates TF-specific classifi...
Transcriptional regulation mainly controls how genes are expressed and how cells behave based on ... more Transcriptional regulation mainly controls how genes are expressed and how cells behave based on the transcription factor (TF) proteins that bind upstream of the transcription start sites (TSSs) of genes. These TF DNA binding sites (TFBSs) are usually short (5-15 base pairs) and degenerate (some positions can have multiple possible alternatives). Traditionally, computational methods scan DNA sequences using the position weight matrix (PWM) of a given TF, calculate binding scores for each K-mer against the PWM, and finally classify a K-mer as to whether it is a putative TFBS or a background sequence based on a cut-off threshold. The FSCAN system, which is proposed in this paper, employs machine learning techniques to build a learner model that is able to identify TFBSs in a set of bound sequences without the need for a cut-off threshold. Our proposed method utilizes fuzzy inference techniques along with a distribution-based filtering algorithm to predict the binding sites of a TF giv...
2015 International Joint Conference on Neural Networks (IJCNN), 2015
In this paper, we investigate the problem of music classification when training data is distribut... more In this paper, we investigate the problem of music classification when training data is distributed throughout a network of interconnected agents (e.g. computers, or mobile devices), and it is available in a sequential stream. Under the considered setting, the task is for all the nodes, after receiving any new chunk of training data, to agree on a single classifier in a decentralized fashion, without reliance on a master node. In particular, in this paper we propose a fully decentralized, sequential learning algorithm for a class of neural networks known as Random Vector Functional-Link nets. The proposed algorithm does not require the presence of a single coordinating agent, and it is formulated exclusively in term of local exchanges between neighboring nodes, thus making it useful in a wide range of realistic situations. Experimental simulations on four music classification benchmarks show that the algorithm has comparable performance with respect to a centralized solution, where a single agent collects all the local data from every node and subsequently updates the model.
Information Sciences, 2014
Negative correlation learning (NCL) aims to produce ensembles with sound generalization capabilit... more Negative correlation learning (NCL) aims to produce ensembles with sound generalization capability through controlling the disagreement among base learners' outputs. Such a learning scheme is usually implemented by using feed-forward neural networks with error back-propagation algorithms (BPNNs). However, it suffers from slow convergence, local minima problem and model uncertainties caused by the initial weights and the setting of learning parameters. To achieve a better solution, this paper employs the random vector functional link (RVFL) networks as base components, and incorporates with the NCL strategy for building neural network ensembles. The basis functions of the base models are generated randomly and the parameters of the RVFL networks can be determined by solving a linear equation system. An analytical solution is derived for these parameters, where a cost function defined for NCL and the well-known least squares method are used. To examine the merits of our proposed algorithm, a comparative study is carried out with nine benchmark datasets. Results indicate that our approach outperforms other ensembling techniques on the testing datasets in terms of both effectiveness and efficiency.
BMC Bioinformatics, 2011
Background: Discrimination of transcription factor binding sites (TFBS) from background sequences... more Background: Discrimination of transcription factor binding sites (TFBS) from background sequences plays a key role in computational motif discovery. Current clustering based algorithms employ homogeneous model for problem solving, which assumes that motifs and background signals can be equivalently characterized. This assumption has some limitations because both sequence signals have distinct properties. Results: This paper aims to develop a Self-Organizing Map (SOM) based clustering algorithm for extracting binding sites in DNA sequences. Our framework is based on a novel intra-node soft competitive procedure to achieve maximum discrimination of motifs from background signals in datasets. The intra-node competition is based on an adaptive weighting technique on two different signal models to better represent these two classes of signals. Using several real and artificial datasets, we compared our proposed method with several motif discovery tools. Compared to SOMBRERO, a state-of-the-art SOM based motif discovery tool, it is found that our algorithm can achieve significant improvements in the average precision rates (i.e., about 27%) on the real datasets without compromising its sensitivity. Our method also performed favourably comparing against other motif discovery tools. Conclusions: Motif discovery with model based clustering framework should consider the use of heterogeneous model to represent the two classes of signals in DNA sequences. Such heterogeneous model can achieve better signal discrimination compared to the homogeneous model.
IEEE transactions on cybernetics, Jul 1, 2018
Random vector functional-link (RVFLs) networks are randomized multi-layer perceptrons (MLPs) with... more Random vector functional-link (RVFLs) networks are randomized multi-layer perceptrons (MLPs) with a single hidden layer and a linear output layer, which can be trained by solving a linear modeling problem. In particular, they are generally trained using a closed-form solution of the (regularized) least-squares approach. This paper introduces several alternative strategies for performing full Bayesian inference (BI) of RVFL networks. Distinct from standard or classical approaches, our proposed Bayesian training algorithms allow to derive an entire probability distribution over the optimal output weights of the network, instead of a single pointwise estimate according to some given criterion (e.g., least-squares). This provides several known advantages, including the possibility of introducing additional prior knowledge in the training process, the availability of an uncertainty measure during the test phase, and the capability of automatically inferring hyper-parameters from given data. In this paper two BI algorithms for regression are firstly proposed that, under some practical assumptions, can be implemented by a simple iterative process with closed-form computations. Simulation results show that one of the proposed algorithms, B-RVFL, is able to outperform standard training algorithms for RVFL networks with a proper regularization factor selected carefully via a line search procedure. A general strategy based on variational inference is also presented, with an application to data modeling problems with noisy outputs or outliers. As we discuss in the paper, using recent advances in automatic differentiation this strategy can be applied to a wide range of additional situations in an immediate fashion.
Wiley Interdisciplinary Reviews-Data Mining and Knowledge Discovery, Feb 9, 2017
Neural networks, as powerful tools for data mining and knowledge engineering, can learn from data... more Neural networks, as powerful tools for data mining and knowledge engineering, can learn from data to build feature-based classifiers and nonlinear predictive models. Training neural networks involves the optimization of non-convex objective functions, and usually the learning process is costly and infeasible for applications associated with data streams. A possible, albeit counter-intuitive alternative is to randomly assign a subset of the networks' weights, so that the resulting optimization task can be formulated as a linear least-squares problem. This methodology can be applied to both feedforward and recurrent networks, and similar techniques can be used to approximate kernel functions. Many experimental results indicate that such randomized models can reach sound performance compared to fully adaptable ones, with a number of favourable benefits, including (i) simplicity of implementation, (ii) faster learning with less intervention from human beings, and (iii) possibility of leveraging over all linear regression and classification algorithms (e.g., ℓ 1 norm minimization for obtaining sparse formulations). All these points make them attractive and valuable to the data mining community, particularly for handling large scale data mining in real-time. However, the literature in the field is extremely vast and fragmented, with many results being reintroduced multiple times under different names. This overview aims at providing a self-contained, uniform introduction to the different ways in which randomization can be applied to the design of neural networks and kernel functions. A clear exposition of the basic framework underlying all these approaches helps to clarify innovative lines of research, open problems and, most importantly, foster the exchanges of well-known results throughout different communities.
Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 2017
Neural networks, as powerful tools for data mining and knowledge engineering, can learn from data... more Neural networks, as powerful tools for data mining and knowledge engineering, can learn from data to build feature-based classifiers and nonlinear predictive models. Training neural networks involves the optimization of non-convex objective functions, and usually the learning process is costly and infeasible for applications associated with data streams. A possible, albeit counter-intuitive alternative is to randomly assign a subset of the networks' weights, so that the resulting optimization task can be formulated as a linear least-squares problem. This methodology can be applied to both feedforward and recurrent networks, and similar techniques can be used to approximate kernel functions. Many experimental results indicate that such randomized models can reach sound performance compared to fully adaptable ones, with a number of favourable benefits, including (i) simplicity of implementation, (ii) faster learning with less intervention from human beings, and (iii) possibility of leveraging over all linear regression and classification algorithms (e.g., ℓ 1 norm minimization for obtaining sparse formulations). All these points make them attractive and valuable to the data mining community, particularly for handling large scale data mining in real-time. However, the literature in the field is extremely vast and fragmented, with many results being reintroduced multiple times under different names. This overview aims at providing a self-contained, uniform introduction to the different ways in which randomization can be applied to the design of neural networks and kernel functions. A clear exposition of the basic framework underlying all these approaches helps to clarify innovative lines of research, open problems and, most importantly, foster the exchanges of well-known results throughout different communities.
Information Sciences, 2015
This paper aims to develop distributed learning algorithms for Random Vector Functional-Link (RVF... more This paper aims to develop distributed learning algorithms for Random Vector Functional-Link (RVFL) networks, where training data is distributed under a decentralized information structure. Two algorithms are proposed by using Decentralized Average Consensus (DAC) and Alternating Direction Method of Multipliers (ADMM) strategies, respectively. These algorithms work in a fully distributed fashion and have no requirement on coordination from a central agent during the learning process. For distributed learning, the goal is to build a common learner model which optimizes the system performance over the whole set of local data. In this work, it is assumed that all stations know the initial weights of the input layer, the output weights of local RVFL networks can be shared through communication channels among neighboring nodes only, and local datasets are blocked strictly. The proposed learning algorithms are evaluated over five benchmark datasets. Experimental results with comparisons show that the DAC-based learning algorithm performs favorably in terms of effectiveness, efficiency and computational complexity, followed by the ADMM-based learning algorithm with promising accuracy but higher computational burden.
Neural Networks, 2016
The current big data deluge requires innovative solutions for performing efficient inference on l... more The current big data deluge requires innovative solutions for performing efficient inference on large, heterogeneous amounts of information. Apart from the known challenges deriving from high volume and velocity, real-world big data applications may impose additional technological constraints, including
Traditionally, two protein sequences are classified into the same class if their feature patterns... more Traditionally, two protein sequences are classified into the same class if their feature patterns have high homology. These feature patterns were originally extracted by sequence alignment algorithms, which measure similarity between an unseen protein sequence and identified protein sequences. Neural network approaches, while reasonably accurate at classification, give no information about the relationship between the unseen case and the classified items that is useful to biologist. In contrast, in this paper we use a generalized radial basis function (GRBF) neural network architecture that generates fuzzy classification rules that could be used for further knowledge discovery. Our proposed techniques were evaluated using protein sequences with ten classes of super-families downloaded from a public domain database, and the results compared favorably with other standard machine learning techniques.
IEEE Transactions on Systems, Man, and Cybernetics: Systems
Clinker free lime (f-CaO) content plays a crucial role in determining the quality of cement. Howe... more Clinker free lime (f-CaO) content plays a crucial role in determining the quality of cement. However, the existing methods are mainly based on laboratory analysis and with significant time delays, which makes the closed-loop control of f-CaO content impossible. In this paper, a multisource data ensemble learning-based soft sensor model is developed for online estimation of clinker f-CaO content. To build such a soft sensor model, input flame images, process variables, and the corresponding output f-CaO content data for a rotary cement kiln were collected from No. 2 rotary kiln at Jiuganghongda Cement Plant which produces 2000 tonnes of clinker per day. The raw data were preprocessed to distinguish the flame image regions of interest (ROI) and remove process variable outliers. Three types of flame image ROI features, i.e., color, global configuration, and local configuration features, were then extracted without segmentation. Further, a kernel partial least square technique was appli...
IEEE Transactions on Neural Networks and Learning Systems, 2016
SOM-based motif mining, despite being a promising approach for problem solving, mostly fails to o... more SOM-based motif mining, despite being a promising approach for problem solving, mostly fails to offer a consistent interpretation of clusters in respect to the mixed composition of signal and noise in the nodes. The main reason behind this shortcoming comes from the similarity metrics used in data assignment, specially designed with the biological interpretation for this domain, are not meant to consider the inevitable noise mixture in the clusters. This limits the explicability of the majority of clusters that are supposedly noise dominated, degrading the overall system clarity in motif discovery. This paper aims to improve the explicability aspect of learning process by introducing a Composite Similarity Function (CSF) that is specially designed for the k-mer-to-cluster similarity measure in respect to the degree of motif properties and embedded noise in the cluster. Our proposed motif finding algorithm in this paper is built on our previous work READ [1] and termed as READ csf , that performs slightly better than READ and shows some remarkable improvements over SOM-based SOMBRERO and SOMEA tools respectively in terms of F-measure on the testing datasets. A real dataset containing multiple motifs is used to explore the potential of the READ csf for more challenging biological data mining tasks. Visual comparisons with the verified logos extracted from JASPAR database demonstrate that our algorithm is promising to discover multiple motifs simultaneously.