Between Two Extremes: Examining Decompositions of the Ensemble Objective Function (original) (raw)
Related papers
Optimising Diversity in Classifier Ensembles
SN Computer Science, 2022
Ensembles of predictors have been generally found to have better performance than single predictors. Although diversity is widely thought to be an important factor in building successful ensembles, there have been contradictory results in the literature regarding the influence of diversity on the generalisation error. Fundamental to this may be the way diversity itself is defined. We present two new diversity measures, based on the idea of ambiguity, obtained from the bias-variance decomposition using the cross-entropy error or the hinge-loss. If random sampling is used to select patterns on which ensemble members are trained, we find that the generalisation error is negatively correlated with diversity at high sampling rates; conversely generalisation error is positively correlated with diversity when the sampling rate is low and the diversity high. We use evolutionary optimisers for small ensembles to select the subsets of patterns for predictor training by maximising these divers...
2008
Abstract This note presents a chronological review of the literature on ensemble learning which has accumulated over the past twenty years. The idea of ensemble learning is to employ multiple learners and combine their predictions. If we have a committee of M models with uncorrelated errors, simply by averaging them the average error of a model can be reduced by a factor of M.
Ensemble Learning in the Presence of Noise
2015
La disponibilidad de grandes cantidades de datos provenientes de diversas fuentes ampl a enormemente las posibilidades para una explotaci on inteligente de la informaci on. No obstante, la extracci on de conocimiento a partir de datos en bruto es una tarea compleja que requiere el desarrollo de m etodos de aprendizaje e cientes y robustos. Una de las principales di cultades en el aprendizaje autom atico es la presencia de ruido en los datos. En esta tesis, abordamos el problema del aprendizaje autom atico en presencia de ruido. Para este prop osito, nos centraremos en el uso de conjuntos de clasi cadores. Nuestro objetivo es crear colecciones de aprendices base cuyos resultados, al ser combinados, mejoren no solo la precisi on sino tambi en la robustez de las predicciones. Una primera contribuci on de esta tesis es aprovechar el ratio de submuestreo para construir conjuntos de clasi cadores basados en bootstrap (como bagging o random forests) precisos y robustos. La idea de utilizar...
Building Ensembles with Heterogeneous Models
2003
In the context of ensemble learning for regression problems, we study the effect of building ensembles from different model classes. Tests on real and simulated data sets show that this approach can improve model accuracy compared to ensembles from a single model class.