Change of representation and inductive bias (original) (raw)
Related papers
DATA-DRIVEN CONSTRUCTIVE INDUCTION: A Methodology and Its Applications
1998
The presented methodology concerns constructive induction, viewed generally as a process combining two intertwined searches: first for the "best" representation space, and second for the "best" hypothesis in that space. The first search employs a range of operators for improving the initial representation space, such as operators for generating new attributes, selecting best attributes among the given ones, and for abstracting attributes. In the methodology presented, these operators are chosen on the basis of the analysis of training data, hence the term data-driven. The second search employs an AQtype rule learning to the examples projected at each iteration to the newly modified representation space. The aim of the search is to determine a generalized description of examples that optimizes a task-oriented multicriterion evaluation function. The two searches are intertwined, as they are executed in a loop in which one feeds into another. Experimental applications of the methodology to text categorization and natural scene interpretation demonstrate a significant practical utility of the proposed methodology.
A Relevancy Filter for Constructive Induction
IEEE Expert / IEEE Intelligent Systems, 1998
rithms enable the learner to extend its vocabulary with new terms if, for a given a set of training examples, the learner's vocabulary is too restncted to solve the learning task. We propose a filter that selects potentially relevant terms from the set of constructed terms and eliminates terms that are irrelevant for the learning task. Restricting constructive induction (or predicate invention) to relevant terms allows a much larger explored space of constructed terms. The elimination of irrelevant terms is especially well-suited for learners of large time or space complexity, such as genetic algorithms and artificial neural networks.
The induction and transfer of declarative bias
People constantly apply acquired knowledge to new learning tasks, but machines almost never do. Research on transfer learning attempts to address this dissimilarity. Working within this area, we report on a procedure that learns and transfers constraints in the context of inductive process mod-eling, which we review. After discussing the role of constraints in model induction, we describe the learning method, MISC, and introduce our metrics for assessing the cost and benefit of transferred knowledge. The reported results suggest that cross-domain transfer is beneficial in the scenarios that we investigated, lending further evidence that this strategy is a broadly effective means for increasing the efficiency of learning systems. We conclude by discussing the aspects of inductive process modeling that encourage effective transfer, by reviewing related strategies, and by describing future research plans for constraint induction and transfer learning. Copyright {\textcopyright} 2010, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.
Data-Driven Constructive Induction
IEEE Intelligent Systems & Their Applications, 1998
The presented methodology concerns constructive induction, viewed generally as a process combining two intertwined searches: first for the "best" representation space, and second for the "best" hypothesis in that space. The first search employs a range of operators for improving the initial representation space, such as operators for generating new attributes, selecting best attributes among the given ones, and for abstracting attributes. In the methodology presented, these operators are chosen on the basis of the analysis of training data, hence the term data-driven. The second search employs an AQtype rule learning to the examples projected at each iteration to the newly modified representation space. The aim of the search is to determine a generalized description of examples that optimizes a task-oriented multicriterion evaluation function. The two searches are intertwined, as they are executed in a loop in which one feeds into another. Experimental applications of the methodology to text categorization and natural scene interpretation demonstrate a significant practical utility of the proposed methodology.
The accuracy of concepts learned from induction
Decision Support Systems, 1993
Inductive learning methods identify a concept from a training sample consisting of positive and negative examples of a target concept. Several studies have shown how such methods could be used to determine rules for expert systems. The question addressed in this paper is: how accurate is the induced concept when used to classify the original domain or another close domain? We derive results that can be used to determine the accuracy of an induced concept. Two previously published applications of inductive learning are used to illustrate our results.
Human Induction in Machine Learning: A Survey of the Nexus
ACM Computing Surveys, 2021
As our epistemic ambitions grow, the common and scientific endeavours are becoming increasingly dependent on Machine Learning (ML). The field rests on a single experimental paradigm, which consists of splitting the available data into a training and testing set and using the latter to measure how well the trained ML model generalises to unseen samples. If the model reaches acceptable accuracy, an a posteriori contract comes into effect between humans and the model, supposedly allowing its deployment to target environments. Yet the latter part of the contract depends on human inductive predictions or generalisations, which infer a uniformity between the trained ML model and the targets. The paper asks how we justify the contract between human and machine learning. It is argued that the justification becomes a pressing issue when we use ML to reach ‘elsewheres’ in space and time or deploy ML models in non-benign environments. The paper argues that the only viable version of the contract can be based on optimality (instead of on reliability which cannot be justified without circularity) and aligns this position with Schurz’s optimality justification. It is shown that when dealing with inaccessible/unstable ground-truths (‘elsewheres’ and non-benign targets), the optimality justification undergoes a slight change, which should reflect critically on our epistemic ambitions. Therefore, the study of ML robustness should involve not only heuristics that lead to acceptable accuracies on testing sets. The justification of human inductive predictions or generalisations about the uniformity between ML models and targets should be included as well. Without it, the assumptions about inductive risk minimisation in ML are not addressed in full.
INFERENCE TO THE BEST INDUCTIVE PRACTICES
Harman and Kulkarni (2007) provide a rigorous and informative discussion of reliable reasoning, drawing philosophical conclusions from the elegant formal results of statistical learning theory. They have presented a strong case that statistical learning theory is highly relevant to issues in philosophy and psychology concerning inductive inferences. Although I agree with their general thrust, I want to take issue with some of the philosophical and psychological conclusions they reach.
Logique Et Analyse, 2004
The aim of this paper is threefold. First, the sometimes slightly messy application of the conditional rule RC of the logic of inductive generalization is clarified by reducing this rule to a so-called basic schema BS. Next, some common truisms about inductive generalization are shown to be mistaken, but are also shown to be valid in special cases. Finally, and most importantly, it is shown that applications of the adaptive logic of inductive generalization to sets of data, possibly in the presence of background knowledge, invokes certain empirical tests and certain theoretically justified defeasible conjectures, which in a sensible way increase one's empirical and theoretical knowledge about a given domain.