A Variation on the Theme of the Most Predictable and Reliable Criterion (original) (raw)
Related papers
On the Relationship Between Maximal Reliability and Maximal Validity of Linear Composites
Multivariate Behavioral Research, 2006
A linear combination of a set of measures is often sought as an overall score summarizing subject performance. The weights in this composite can be selected to maximize its reliability or to maximize its validity, and the optimal choice of weights is in general not the same for these two optimality criteria. We explore several relationships between the resulting reliability and validity estimates in different situations. Only in the case of congeneric tests are maximal reliability and maximal validity attained with the same weights, and a precise relationship between these two maximality concepts can be derived. A widely and readily applicable procedure for point and interval estimation of maximal validity is also outlined. Several inequalities are established for the case when the measures are not congeneric.
Journal of Geodesy
The reliability matrix, being an oblique projection operator, transforms correlated observations into the least squares residuals in Gauss-Markov models. It also allows to study model responses in individual observations to the assumed configurations of gross errors. The variability of the basic characteristics of the operator due to the increase in observation correlations is investigated by means of numerical tests and theoretical derivations. The characteristics such as diagonal elements and asymmetry indices have not that long ago been introduced as the response-based measures of internal reliability and subjected to the analysis. Here, additionally, the relationship between the asymmetry indices and the angles of non-orthogonality of projection is derived. The measures are compared in terms of the effect of observation correlations with the commonly used reliability measures obtained on the basis of statistical tests for detection and identification of outliers, such as generalized reliability numbers and minimal detectable biases. For the purposes of the present paper, the latter are termed the testingbased measures. The comparative analysis shows that both the types, when taken together, provide complete information about the behaviour of a GMM with correlated observations in the presence of a gross error in a particular observation and about its detectability. Hence, the conclusion is that the response-based measures can be a useful supplementation of the testing-based measures for the phase of network design.
A new approach for assessing the correlated risk
Purpose – The purpose of this paper is to explore the relationship between multivariate process capability indices and loss functions for both nominal-the-best and smaller-the-better cases, so the likelihood and consequences resulting from the nonconforming of a manufacturing process or an environmental system can be evaluated simultaneously. Design/methodology/approach – In this paper, the authors present a new approach of correlated risk assessment by linking the multiple process capability indices and loss functions, in which the multivariate process capability indices and multivariate loss functions describe the likelihood and consequences as a result of nonconformities in multivariate manufacturing or environmental system, respectively. Then, the associated relationship equations are developed using multivariate methods. Moreover, a step-by-step procedure is provided to facilitate the implementation of the correlated risk assessment. Findings – Given the multivariate process capability indices, the authors show that the expected loss can be estimated by developed relationship equations and two numerical examples are also given, to demonstrate how the correlated manufacturing and environmental risks can be properly assessed by linking the multivariate process capability indices and multivariate loss function. Practical implications – The risk information of likelihood and expected loss, classified in the four planning zones of a strategic planning matrix, provides practising managers and engineers with a decision-making tool for prioritizing their quality improvement projects when conducting risk assessment for any multivariate process or environmental system. Originality/value – Once the existing quality/environmental problems and their Key Performance Indicators are identified, one may conduct risk assessment by applying the relationship equations to evaluate the impact of correlated risk on manufacturing processes or multiple environmental emissions inside company; this can lead to the direction of continuous improvement for any industry.
Evaluation of prediction reliability in regression using the transduction principle
The IEEE Region 8 EUROCON 2003. Computer as a Tool., 2003
In machine learning community there are many efforts to improve overall reliability of predictors measured as an error on the testing set. But in contrast, very little research has been done concerning prediction reliability of a single answer. This article describes an algorithm that can be used for evaluation of prediction reliability in regression. The basic idea of the algorithm is based on construction of transductive predictors. Using them, the algorithm makes inference from the differences between initial and transductive predictions to the error on a single new case. The implementation of the algorithm with regression trees managed to significantly reduce the relative mean squared error on the majority of the tested domains.
On the Selection of Error Measures for Comparisons Among
1995
Clements and Hendry (1993) proposed the Generalized Forecast Error Second Moment(GFESM) as an improvement to the Mean Square Error in comparing forecasting performance across data series. They based their conclusion on the fact that rankings based on GFESM remain unaltered if the series are linearly transformed. In this paper, we argue that this evaluation ignores other important criteria. Also, their conclusions were illustrated by a simulation study whose relationship to real data was not obvious. Thirdly, prior empirical studies show that the mean square error is an inappropriate measure to serve as a basis for comparison. This undermines the claims made for the GFESM.
Journal of Multivariate Analysis
n. Such comparisons are called two-sample comparisons. It is shown that some of the results in this paper strengthen previous results in the literature. Some applications in reliability theory are described.
REGRESSION MODEL DIAGNOSTIC, TEST AND ROBUSTIFICATION IN THE PRESENCE OF MULTICOLLINEAR COVARIATES
Regression techniques has so many applications in the fields of science, social science and engineering. This article contributes to the theoretical and empirical literature related to the use of regression techniques and methods in experimental and non experimental subjects. We attempt to review the basic pedagogic concepts of classical linear regression model with respect to the necessary assumptions, optimality conditions, and its applications in quantitative analysis, environmental science and engineering. We address some mechanics of regression model robustificationwith an algebraic exposition of the ordinary least squares regression coefficients. Finally, we point out the peculiarity of the presence of multicollinear covariates as revealed in our empirical results using statistical software packages like STATA and SPSS.
Some new methods for the comparison of two linear regression models
Journal of Statistical Planning and Inference, 2007
The frequently used approach to the comparison of two linear regression models is to use the partial F test. It is pointed out in this paper that the partial F test has in fact a naturally associated two-sided simultaneous confidence band, which is much more informative than the test itself. But this confidence band is over the entire range of all the covariates. As regression models are true or of interest often only over a restricted region of the covariates, the part of this confidence band outside this region is therefore useless and to ensure 1 − simultaneous coverage probability is therefore wasteful of resources. It is proposed that a narrower and hence more efficient confidence band over a restricted region of the covariates should be used. The critical constant required in the construction of this confidence band can be calculated by Monte Carlo simulation. While this two-sided confidence band is suitable for two-sided comparisons of two linear regression models, a more efficient one-sided confidence band can be constructed in a similar way if one is only interested in assessing whether the mean response of one regression model is higher (or lower) than that of the other in the region. The methodologies are illustrated with two examples.