François Laviolette - Academia.edu (original) (raw)
Papers by François Laviolette
Bookmarks Related papers MentionsView impact
PLoS computational biology, 2015
The discovery of peptides possessing high biological activity is very challenging due to the enor... more The discovery of peptides possessing high biological activity is very challenging due to the enormous diversity for which only a minority have the desired properties. To lower cost and reduce the time to obtain promising peptides, machine learning approaches can greatly assist in the process and even partly replace expensive laboratory experiments by learning a predictor with existing data or with a smaller amount of data generation. Unfortunately, once the model is learned, selecting peptides having the greatest predicted bioactivity often requires a prohibitive amount of computational time. For this combinatorial problem, heuristics and stochastic optimization methods are not guaranteed to find adequate solutions. We focused on recent advances in kernel methods and machine learning to learn a predictive model with proven success. For this type of model, we propose an efficient algorithm based on graph theory, that is guaranteed to find the peptides for which the model predicts max...
Bookmarks Related papers MentionsView impact
Bookmarks Related papers MentionsView impact
Principles and Practice of Constraint Programming, Sep 2015
In search theory, the goal of the Optimal Search Path (OSP) problem is to find a finite length pa... more In search theory, the goal of the Optimal Search Path (OSP) problem is to find a finite length path maximizing the probability that a searcher detects a lost wanderer on a graph. We propose to bound the probability of finding the wanderer in the remaining search time by relaxing the problem into a stochastic game of cop and robber from graph theory. We discuss the validity of this bound and demonstrate its effectiveness on a constraint programming model of the problem. Experimental results show how our novel bound compares favorably to the DMEAN bound from the literature, a state-of-the-art bound based on a relaxation of the OSP into a longest path problem.
Bookmarks Related papers MentionsView impact
The Set Covering Machine (SCM) is a greedy learning algorithm that produces sparse classifiers. W... more The Set Covering Machine (SCM) is a greedy learning algorithm that produces sparse classifiers. We extend the SCM for datasets that contain a huge number of features. The whole genetic material of living organisms is an example of such a case, where the number of feature exceeds 10^7. Three human pathogens were used to evaluate the performance of the SCM at predicting antimicrobial resistance. Our results show that the SCM compares favorably in terms of sparsity and accuracy against L1 and L2 regularized Support Vector Machines and CART decision trees. Moreover, the SCM was the only algorithm that could consider the full feature space. For all other algorithms, the latter had to be filtered as a preprocessing step.
Bookmarks Related papers MentionsView impact
BMC Bioinformatics, 2013
Bookmarks Related papers MentionsView impact
Journal of Immunological Methods, 2013
Bookmarks Related papers MentionsView impact
Bookmarks Related papers MentionsView impact
Bookmarks Related papers MentionsView impact
Principles and Practice of Constraint Programming, 2012
The optimal search path (OSP) problem is a single-sided detection search problem where the locati... more The optimal search path (OSP) problem is a single-sided detection search problem where the location and the detectability of a moving object are uncertain. A solution to this NP-hard problem is a path on a graph that maximizes the probability of finding an object that moves according to a known motion model. We developed constraint programming models to solve this probabilistic path planning problem for a single indivisible searcher. These models include a simple but powerful branching heuristic as well as strong filtering constraints. We present our experimentation and compare our results with existing results in the literature. The OSP problem is particularly interesting in that it generalizes to various probabilistic search problems such as intruder detection, malicious code identification, search and rescue, and surveillance.
Bookmarks Related papers MentionsView impact
We propose a new approach to verification of probabilistic processes for which the model may not ... more We propose a new approach to verification of probabilistic processes for which the model may not be available. We show how to use a technique from Reinforcement Learning to approximate how far apart two processes are by solving a Markov Decision Process. The key idea of the approach is to define the MDP out of the processes to be tested,
Bookmarks Related papers MentionsView impact
Discrete Mathematics, 2000
A (nite or innite) graph G is constructible if there exists a well-ordering 6 of its vertices suc... more A (nite or innite) graph G is constructible if there exists a well-ordering 6 of its vertices such that, for every vertex x which is not the smallest element, there is a vertex y<xwhich is adjacent to x and to every neighbor z of x with z<x. We prove that every Helly graph and every connected bridged graph are constructible.
Bookmarks Related papers MentionsView impact
Lecture Notes in Computer Science, 2008
Bookmarks Related papers MentionsView impact
Proceedings of the 22nd international conference on Machine learning - ICML '05, 2005
Bookmarks Related papers MentionsView impact
We present two alternative ways to apply PAC-Bayesian analysis to sequences of dependent random v... more We present two alternative ways to apply PAC-Bayesian analysis to sequences of dependent random variables. The first is based on a new lemma that enables to bound expectations of convex functions of certain dependent random variables by expectations of the same functions of independent Bernoulli random variables. This lemma provides an alternative tool to Hoeffding-Azuma inequality to bound concentration of martingale values. Our second approach is based on integration of Hoeffding-Azuma inequality with PAC-Bayesian analysis. We also introduce a way to apply PAC-Bayesian analysis in situation of limited feedback. We combine the new tools to derive PAC-Bayesian generalization and regret bounds for the multiarmed bandit problem. Although our regret bound is not yet as tight as state-of-the-art regret bounds based on other well-established techniques, our results significantly expand the range of potential applications of PAC-Bayesian analysis and introduce a new analysis tool to reinf...
Bookmarks Related papers MentionsView impact
We derive an instantaneous (per-round) data-dependent regret bound for stochas-tic multiarmed ban... more We derive an instantaneous (per-round) data-dependent regret bound for stochas-tic multiarmed bandits with side information (also known as contextual bandits). The scaling of our regret bound with the number of states (contexts) N goes as p NI ⇢t (S; A), where I ⇢t (S; A) is the mutual information between states and ac-tions (the side information) used by the algorithm at round t. If the algorithm uses all the side information, the regret bound scales as p N ln K, where K is the number of actions (arms). However, if the side information I ⇢t (S; A) is not fully used, the regret bound is significantly tighter. In the extreme case, when I ⇢t (S; A) = 0, the dependence on the number of states reduces from linear to logarithmic. Our analysis allows to provide the algorithm large amount of side information, let the algorithm to decide which side information is relevant for the task, and penalize the algorithm only for the side information that it is using de facto. We also present an alg...
Bookmarks Related papers MentionsView impact
We study the issue of domain adaptation: we want to adapt a model from a source distribution to a... more We study the issue of domain adaptation: we want to adapt a model from a source distribution to a target one. We focus on models expressed as a majority vote. Our main contribution is a novel theoretical analysis of the target risk that is formulated as an upper bound expressing a trade-off between only two terms: (i) the voters' joint errors on the source distribution, and (ii) the voters' disagreement on the target one; both easily estimable from samples. Hence, this new study is more precise than other analyses that usually rely on three terms (including a hardly controllable term). Moreover, we derive a PAC-Bayesian generalization bound, and specialize the result to linear classifiers to propose a learning algorithm.
Bookmarks Related papers MentionsView impact
We introduce a new representation learning approach for domain adaptation, in which data at train... more We introduce a new representation learning approach for domain adaptation, in which data at training and test time come from similar but different distributions. Our approach is directlyinspired by the theory on domain adaptation suggesting that, for effective domain transfer to be achieved, predictions must be made based on features that cannot discriminate between the training (source) and test (target) domains. The approach implements this idea in the context of neural network architectures that are trained on labeled data from the source domain and unlabeled data from the target domain (no labeled target-domain data is necessary). As the training progresses, the approach promotes the emergence of features that are (i) discriminative for the main learning task on the source domain and (ii) indiscriminate with respect to the shift between the domains. We show that this adaptation behaviour can be achieved in almost any feed-forward model by augmenting it with few standard layers a...
Bookmarks Related papers MentionsView impact
PAC-Bayesian theory provides generalization bounds for weighted majority vote. However, these bou... more PAC-Bayesian theory provides generalization bounds for weighted majority vote. However, these bounds do not directly focus on the risk of the majority vote, but on the risk of the Gibbs classifier. Indeed, it is well-known that the Gibbs classifier and the majority vote are related. To the best of our knowledge the tightest relation is the C-bound proposed by Lacasse et al. (2007); Laviolette et al. (2011) for binary classification. In this paper, we provide three generalizations of the C-bound to multiclass setting.
Bookmarks Related papers MentionsView impact
Lecture Notes in Computer Science, 2006
Bookmarks Related papers MentionsView impact
Bookmarks Related papers MentionsView impact
PLoS computational biology, 2015
The discovery of peptides possessing high biological activity is very challenging due to the enor... more The discovery of peptides possessing high biological activity is very challenging due to the enormous diversity for which only a minority have the desired properties. To lower cost and reduce the time to obtain promising peptides, machine learning approaches can greatly assist in the process and even partly replace expensive laboratory experiments by learning a predictor with existing data or with a smaller amount of data generation. Unfortunately, once the model is learned, selecting peptides having the greatest predicted bioactivity often requires a prohibitive amount of computational time. For this combinatorial problem, heuristics and stochastic optimization methods are not guaranteed to find adequate solutions. We focused on recent advances in kernel methods and machine learning to learn a predictive model with proven success. For this type of model, we propose an efficient algorithm based on graph theory, that is guaranteed to find the peptides for which the model predicts max...
Bookmarks Related papers MentionsView impact
Bookmarks Related papers MentionsView impact
Principles and Practice of Constraint Programming, Sep 2015
In search theory, the goal of the Optimal Search Path (OSP) problem is to find a finite length pa... more In search theory, the goal of the Optimal Search Path (OSP) problem is to find a finite length path maximizing the probability that a searcher detects a lost wanderer on a graph. We propose to bound the probability of finding the wanderer in the remaining search time by relaxing the problem into a stochastic game of cop and robber from graph theory. We discuss the validity of this bound and demonstrate its effectiveness on a constraint programming model of the problem. Experimental results show how our novel bound compares favorably to the DMEAN bound from the literature, a state-of-the-art bound based on a relaxation of the OSP into a longest path problem.
Bookmarks Related papers MentionsView impact
The Set Covering Machine (SCM) is a greedy learning algorithm that produces sparse classifiers. W... more The Set Covering Machine (SCM) is a greedy learning algorithm that produces sparse classifiers. We extend the SCM for datasets that contain a huge number of features. The whole genetic material of living organisms is an example of such a case, where the number of feature exceeds 10^7. Three human pathogens were used to evaluate the performance of the SCM at predicting antimicrobial resistance. Our results show that the SCM compares favorably in terms of sparsity and accuracy against L1 and L2 regularized Support Vector Machines and CART decision trees. Moreover, the SCM was the only algorithm that could consider the full feature space. For all other algorithms, the latter had to be filtered as a preprocessing step.
Bookmarks Related papers MentionsView impact
BMC Bioinformatics, 2013
Bookmarks Related papers MentionsView impact
Journal of Immunological Methods, 2013
Bookmarks Related papers MentionsView impact
Bookmarks Related papers MentionsView impact
Bookmarks Related papers MentionsView impact
Principles and Practice of Constraint Programming, 2012
The optimal search path (OSP) problem is a single-sided detection search problem where the locati... more The optimal search path (OSP) problem is a single-sided detection search problem where the location and the detectability of a moving object are uncertain. A solution to this NP-hard problem is a path on a graph that maximizes the probability of finding an object that moves according to a known motion model. We developed constraint programming models to solve this probabilistic path planning problem for a single indivisible searcher. These models include a simple but powerful branching heuristic as well as strong filtering constraints. We present our experimentation and compare our results with existing results in the literature. The OSP problem is particularly interesting in that it generalizes to various probabilistic search problems such as intruder detection, malicious code identification, search and rescue, and surveillance.
Bookmarks Related papers MentionsView impact
We propose a new approach to verification of probabilistic processes for which the model may not ... more We propose a new approach to verification of probabilistic processes for which the model may not be available. We show how to use a technique from Reinforcement Learning to approximate how far apart two processes are by solving a Markov Decision Process. The key idea of the approach is to define the MDP out of the processes to be tested,
Bookmarks Related papers MentionsView impact
Discrete Mathematics, 2000
A (nite or innite) graph G is constructible if there exists a well-ordering 6 of its vertices suc... more A (nite or innite) graph G is constructible if there exists a well-ordering 6 of its vertices such that, for every vertex x which is not the smallest element, there is a vertex y<xwhich is adjacent to x and to every neighbor z of x with z<x. We prove that every Helly graph and every connected bridged graph are constructible.
Bookmarks Related papers MentionsView impact
Lecture Notes in Computer Science, 2008
Bookmarks Related papers MentionsView impact
Proceedings of the 22nd international conference on Machine learning - ICML '05, 2005
Bookmarks Related papers MentionsView impact
We present two alternative ways to apply PAC-Bayesian analysis to sequences of dependent random v... more We present two alternative ways to apply PAC-Bayesian analysis to sequences of dependent random variables. The first is based on a new lemma that enables to bound expectations of convex functions of certain dependent random variables by expectations of the same functions of independent Bernoulli random variables. This lemma provides an alternative tool to Hoeffding-Azuma inequality to bound concentration of martingale values. Our second approach is based on integration of Hoeffding-Azuma inequality with PAC-Bayesian analysis. We also introduce a way to apply PAC-Bayesian analysis in situation of limited feedback. We combine the new tools to derive PAC-Bayesian generalization and regret bounds for the multiarmed bandit problem. Although our regret bound is not yet as tight as state-of-the-art regret bounds based on other well-established techniques, our results significantly expand the range of potential applications of PAC-Bayesian analysis and introduce a new analysis tool to reinf...
Bookmarks Related papers MentionsView impact
We derive an instantaneous (per-round) data-dependent regret bound for stochas-tic multiarmed ban... more We derive an instantaneous (per-round) data-dependent regret bound for stochas-tic multiarmed bandits with side information (also known as contextual bandits). The scaling of our regret bound with the number of states (contexts) N goes as p NI ⇢t (S; A), where I ⇢t (S; A) is the mutual information between states and ac-tions (the side information) used by the algorithm at round t. If the algorithm uses all the side information, the regret bound scales as p N ln K, where K is the number of actions (arms). However, if the side information I ⇢t (S; A) is not fully used, the regret bound is significantly tighter. In the extreme case, when I ⇢t (S; A) = 0, the dependence on the number of states reduces from linear to logarithmic. Our analysis allows to provide the algorithm large amount of side information, let the algorithm to decide which side information is relevant for the task, and penalize the algorithm only for the side information that it is using de facto. We also present an alg...
Bookmarks Related papers MentionsView impact
We study the issue of domain adaptation: we want to adapt a model from a source distribution to a... more We study the issue of domain adaptation: we want to adapt a model from a source distribution to a target one. We focus on models expressed as a majority vote. Our main contribution is a novel theoretical analysis of the target risk that is formulated as an upper bound expressing a trade-off between only two terms: (i) the voters' joint errors on the source distribution, and (ii) the voters' disagreement on the target one; both easily estimable from samples. Hence, this new study is more precise than other analyses that usually rely on three terms (including a hardly controllable term). Moreover, we derive a PAC-Bayesian generalization bound, and specialize the result to linear classifiers to propose a learning algorithm.
Bookmarks Related papers MentionsView impact
We introduce a new representation learning approach for domain adaptation, in which data at train... more We introduce a new representation learning approach for domain adaptation, in which data at training and test time come from similar but different distributions. Our approach is directlyinspired by the theory on domain adaptation suggesting that, for effective domain transfer to be achieved, predictions must be made based on features that cannot discriminate between the training (source) and test (target) domains. The approach implements this idea in the context of neural network architectures that are trained on labeled data from the source domain and unlabeled data from the target domain (no labeled target-domain data is necessary). As the training progresses, the approach promotes the emergence of features that are (i) discriminative for the main learning task on the source domain and (ii) indiscriminate with respect to the shift between the domains. We show that this adaptation behaviour can be achieved in almost any feed-forward model by augmenting it with few standard layers a...
Bookmarks Related papers MentionsView impact
PAC-Bayesian theory provides generalization bounds for weighted majority vote. However, these bou... more PAC-Bayesian theory provides generalization bounds for weighted majority vote. However, these bounds do not directly focus on the risk of the majority vote, but on the risk of the Gibbs classifier. Indeed, it is well-known that the Gibbs classifier and the majority vote are related. To the best of our knowledge the tightest relation is the C-bound proposed by Lacasse et al. (2007); Laviolette et al. (2011) for binary classification. In this paper, we provide three generalizations of the C-bound to multiclass setting.
Bookmarks Related papers MentionsView impact
Lecture Notes in Computer Science, 2006
Bookmarks Related papers MentionsView impact
Constraint Programming for Path Planning with Uncertainty: Solving the Optimal Search Path problem., 2012
The optimal search path (OSP) problem is a single-sided detection search problem from search theo... more The optimal search path (OSP) problem is a single-sided detection search problem from search theory where the location and the detectability of a moving object are uncertain. A solution to this NP-hard problem is a path on a graph that maximizes the probability of finding an object that moves according to a known motion model. We developed constraint programming models to solve this probabilistic path planning problem for a single indivisible searcher. These models include a simple but powerful branching heuristic as well as strong filtering constraints. The OSP problem is particularly interesting in that it generalizes to various probabilistic search problems such as intruder detection, malicious code identification, search and rescue, and surveillance.
Related paper:
M. Morin, A.P. Papillon, F. Laviolette, I. Abi-Zeid, and C.G. Quimper, “Constraint Programming for Path Planning with Uncertainty: Solving the Optimal Search Path problem,” in Proceedings of the 18th Conference on Principles and Practice of Constraint Programming, Québec, Qc, Canada, 2012, pp. 988-1003."
Bookmarks Related papers MentionsView impact