A parsimonious explanation of observed biases when forecasting one’s own performance (original) (raw)
Related papers
Pessimistic Bias in Predictions of Performance Results
Psychological Reports, 1995
Ben-Gurion U~tiuersity of the Negev, Israel Szimmary.-Reahsm in the pedormance predictions of 60 university students was investigated. After a practice trial on a task of creativity, one group of subjects were asked to state their expectations and the other group their hopes for their ance scores on the First and second test trials before each one. Both groups were unrealistically pessimistic about their performance: the firsc and second trial predictions of the expectation group as well as of the hope group were lower than their actual performance scores. In all cases (except the second-trial prediction of the hope group) the ddferences reached significance. Results are explained from the functional perspective. It is suggested that unreahsticdy low predictions may serve an affective function (feeling better).
Journal of Forecasting, 1994
A decomposition of the Brier skill score shows that the performance of judgmental forecasts depends on seven components: environmental predictability, fidelity of the information system, match between environment and forecaster, reliability of information acquisition, reliability of information processing, conditional bias, and unconditional bias. These components provide a framework for research on the forecasting process. Selected literature addressing each component is reviewed, and implications for improving judgmental forecasting are discussed. KEY WORDS Judgmental forecasting Brier skill score Lens model equation Bias Reliability In any field requiring judgmental forecasts, the performance of professional forecasters depends jointly on (1) the environment about which forecasts are made, (2) the information system that brings data about the environment to the forecaster, and (3) the cognitive system of the forecaster. For example, in weather forecasting the environment includes the atmosphere and the land, ocean and solar features that affect weather. The information system includes the instruments, observations, and algorithms that produce information about past and current weather and the communication and display systems that bring that information to the forecaster. The cognitive system consists of the perceptual and judgmental processes that the forecaster uses to acquire information, aggregate it, and produce the forecast. This paper describes how certain properties of these three systems combine to determine forecasting performance. A commonly used measure of skill (the 'skill score' based on the mean-square-error) is analyzed into seven components. Since each component describes a different aspect of forecast performance, the decomposition suggests a framework for research on judgmental forecasting.
Statistical correction of judgmental point forecasts and decisions
Omega, 1996
In many organizations point estimates labelled as 'forecasts' are produced by human judgment rather than statistical methods. However, when these estimates are subject to asymmetric loss they are, in fact, decisions because they involve the selection of a value with the objective ...
Improving Reliability of Judgmental Forecasts
International Series in Operations Research & Management Science, 2001
All judgmental forecasts will be affected by the inherent unreliability, or inconsistency, of the judgment process. Psychologists have studied this problem extensively, but forecasters rarely address it. Researchers and theorists describe two types of unreliability that can reduce the accuracy of judgmental forecasts: (1) unreliability of information acquisition, and (2) unreliability of information processing. Studies indicate that judgments are less reliable when the task is more complex; when the environment is more uncertain; when the acquisition of information relies on perception, pattern recognition, or memory; and when people use intuition instead of analysis. Five principles can improve reliability in judgmental forecasting: 1. Organize and present information in a form that clearly emphasizes relevant information. 2. Limit the amount of information used in judgmental forecasting. Use a small number of really important cues. 3. Use mechanical methods to process infomation. 4. Combine several forecasts. 5. Require justification of forecasts.
A Decomposition of the Correlation Coefficient and its Use in Analyzing Forecasting Skill
Weather and Forecasting, 1990
Estimates of several components of forecasting skill can be obtained by combining a skill-score decomposition developed by Allan Murphy with techniques for decomposing correlation coefficients that have been employed in research on human judgment. The decomposition of the correlation coefficient requires knowledge of the information or "cues" used by the forecaster. When the cues are known, it is possible to estimate the effects of uncertainty and the forecaster's consistency and use of the cues.
Judgmental forecasting: A review of progress over the last 25years
International Journal of Forecasting, 2006
The past 25 years has seen phenomenal growth of interest in judgemental approaches to forecasting and a significant change of attitude on the part of researchers to the role of judgement. While previously judgement was thought to be the enemy of accuracy, today judgement is recognised as an indispensable component of forecasting and much research attention has been directed at understanding and improving its use. Human judgement can be demonstrated to provide a significant benefit to forecasting accuracy but it can also be subject to many biases. Much of the research has been directed at understanding and managing these strengths and weaknesses. An indication of the explosion of research interest in this area can be gauged by the fact that over 200 studies are referenced in this review.
Accuracy, error, and bias in predictions for real versus hypothetical events
Journal of Personality and Social Psychology, 2006
Participants made predictions about performance on tasks that they did or did not expect to complete. In three experiments, participants in task-unexpected conditions were unrealistically optimistic: They overestimated how well they would perform, often by a large margin, and their predictions were not correlated with their performance. By contrast, participants assigned to task-expected conditions made predictions that were not only less optimistic but strikingly accurate. Consistent with predictions from construal level theory, data from a fourth experiment suggest that it is the uncertainty associated with hypothetical tasks, and not a lack of cognitive processing, that frees people to make optimistic prediction errors. Unrealistic optimism, when it occurs, may be truly unrealistic; however, it may be less ubiquitous than has been previously suggested.
Journal of Personality …, 1996
Although evidence for unrealistic optimism is considerable, there is reason to believe that individuals will abandon their optimism and may even become pessimistic in anticipation of self-relevant feedback. The authors propose and provide preliminary test of a model of the temporal transition from optimism to accuracy to pessimism in outcome predictions. In Study 1, college sophomores, juniors, and seniors estimated their likely salary at their first full-time job after graduation. Only seniors became less optimistic as graduation approached. In Study 2, students estimated their exam score a month before the exam, then again several times after completing the exam yet prior to receiving feedback. As the proximity of feedback neared, students abandoned their optimistic forecast in favor of a pessimistic forecast. Study 3 showed that, in anticipation of self-relevant feedback, participants with low self-esteem lowered their performance estimates more readily than did those with high selfesteem. People are constantly requested to make predictions about future events. Coaches, players, and fans alike are queried for their predictions regarding the outcome and likely score of sporting events. Physicians and other health specialists routinely must supply their opinion of the prognosis of a disease or illness. Meteorologists must predict the weather; auto mechanics must estimate the cost of repairs. When making predictions about the self, people can be quite optimistic in their predictions of future outcomes and risk, perceiving that good things are more likely and bad things are less likely to happen to them. For example, people tend to believe that they are more likely than others to have gifted children (Weinstein, 1980), to get a good first job (Weinstein, 1980), and to do well on future tasks (Crandall, Solomon, & Kelleway, 1955;Irwin, 1944, 1953;Marks, 1951). People also believe that they will be happier, more confident, more hardworking, and less lonely in the future than their peers (Perloff, 1987). A similar optimism pervades evaluations of negative events. People tend to believe that they will live longer than their cohorts (Snyder, 1978), that they will not be victims of auto accidents
The Psychology of Economic Forecasting
Is the imprecision of economic forecasts due to the judgments of ‘biased’ decision makers? This study explores decision-making among expert forecasters in Sweden using semi-structured interviews. The results indicate that forecasters’ decision processes are characterized by intuitive as well as calculating reasoning, gradually adopting mental models and conflicting goals. While forecasters make judgments that are non-optimal in terms of minimizing forecasting errors, these are not necessarily biased but can be described as ecologically rational decisions. The results indicate that behavioral forecasting research would benefit from taking into account the specific decision-making environment in which forecasters operate.