Combining Imprecise or Conflicting Probability Judgments: A Choice-Based Study (original) (raw)
Related papers
Subjective recalibration of advisors' probability estimates
Psychonomic bulletin & review, 2010
Are decision makers sensitive to the statistical properties (i.e., calibration) of probability estimates that they receive from advisors? After specifying the ideal use of such estimates, we derive the roughly ideal forecast consumer (RIFC) and generalize it to account for how humans might use the estimates. We report an experiment in which participants first experienced various advisors by seeing their probability estimates and the associated outcomes and then provided confidence judgments in the presence of the advisors' estimates. The generalized model described the data well and showed that the participants were appropriately sensitive to the statistical properties of the advisors. Models of the individuals were better calibrated than the participants themselves, but still inferior to the RIFC. A detailed description of our model-testing procedure can be found in an appendix to the article.
Decision Making and Confidence Given Uncertain Advice
Cognitive Science, 2006
We study human decision making in a simple forced-choice task that manipulates the frequency and accuracy of available information. Empirically, we find that people make decisions consistent with the advice provided, but that their subjective confidence in their decisions shows 2 interesting properties. First, people's confidence does not depend solely on the accuracy of the advice. Rather, confidence seems to be influenced by both the frequency and accuracy of the advice. Second, people are less confident in their guessed decisions when they have to make relatively more of them. Theoretically, we develop and evaluate a type of sequential sampling process model-known as a self-regulating accumulator-that accounts for both decision making and confidence. The model captures the regularities in people's behavior with interpretable parameter values, and we show its ability to fit the data is not due to excessive model complexity. Using the model, we draw conclusions about some properties of human reasoning under uncertainty.
Confidence in Beliefs and Rational Decision Making
Economics and Philosophy, 2018
The standard, Bayesian account of rational belief and decision is often argued to be unable to cope properly with severe uncertainty, of the sort ubiquitous in some areas of policy making. This paper tackles the question of what should replace it as a guide for rational decision making. It defends a recent proposal, which reserves a role for the decision maker's confidence in beliefs. Beyond being able to cope with severe uncertainty, the account has strong normative credentials on the main fronts typically evoked as relevant for rational belief and decision. It fares particularly well, we argue, in comparison to other prominent non-Bayesian models in the literature.
Maintaining credibility when communicating uncertainty: the role of directionality
Thinking & Reasoning, 2020
Risk communicators often need to communicate probabilistic predictions. On occasion, an event with 10% likelihood will occur, or one with 90% likelihood will not-a probabilistically unexpected outcome. Previous research manipulating communication format has found that communicators lose more credibility and are perceived as less correct if an 'unlikely' event occurs, than if a '10-30% likelihood' event occurs. We suggest 'directionality-outcome congruence' underlies the perception of predictions as 'erroneous'. For example, the negatively directional term 'unlikely' led to harsher ratings because the outcome was counter to the original focus of the prediction (on the event's nonoccurrence). In the context of both probabilistically unexpected (Experiment 1) and expected (Experiment 2) outcomes, we find that communicators are perceived as less credible and less correct given 'directionality-outcome incongruence'. Communicators should thus carefully consider the directionality implicit in their risk communications to maximise communication effectiveness.
Giving advice under uncertainty: What you do, what you should do, and what others think you do
Journal of Economic Psychology, 2008
The current paper examines whether advisors rely primarily on their personal beliefs and risk preferences or on their estimates of their clients' beliefs and risk preferences. The results of two studies show that risk preferences predicted for other individuals were more risk neutral than personal risk preferences under uncertainty, though no differences were found between personal beliefs and those attributed to other individuals. This suggests that advice that is based primarily on personal risk preferences may differ from advice which is based primarily on advisors' estimates of clients' risk preferences. The results of a third study show that advisors rely on their personal risk preferences when giving advice. Based on these findings we suggest that managers should be cautious in accepting advice concerning which alternative ought to be pursued, as advisors tend to instinctively rely on their own risk preferences, which may differ from the manager's actual risk preferences. Published by Elsevier B.V.
Coherence and Consistency of Investors' Probability Judgments
Management Science, 2007
This study investigates the quality of direct probability judgments and quantile estimates with a focus on calibration and consistency. The two response modes use different measures of miscalibration, so it is difficult to directly compare their relative (in)accuracy. We employed a more refined within-subject design in which decision makers (DMs) used both response modes to make judgments about a random sample of stocks accompanied by identical information to facilitate comparison between the two judgment methods. DMs judged the probabilities that the stocks will reach a certain threshold, provided lower and upper bounds of these forecasts, and estimated median, 50%, 70%, and 90% confidence intervals of their future prices. We found that the judgments were internally consistent and coherent, but in most cases they were slightly miscalibrated. We used several new methods of analysis that allow for more precise and reliable comparison between the two response modes. We inferred point ...
Intuitive evaluation of likelihood judgment producers: evidence for a confidence heuristic
Journal of Behavioral Decision Making, 2003
This research tests the hypothesis of Yates et al. (1996) that people prefer judgment producers who make extreme confidence judgments. In each of three experiments, college students evaluated two fictional financial advisors who judged the likelihood that each of several stocks would increase in value. One of the advisors (the moderate advi-sor) was reasonably well calibrated and the other (the extreme advisor) was overconfi-dent. In all three experiments, participants tended to prefer the extreme advisor. Experiments 2 and 3 showed that the advisors' confidence influenced participants' perception of their knowledge, and Experiment 3 showed that it influenced their perception of the number of categorically correct judgments they made. Both of these variables were, in turn, related to participants' preferences. Experiment 3 also suggested that need for cognition and right-wing authoritarianism are positively related to preference for the extreme advisor. A quantitative model is presented, which captures the basic pattern of results. This model includes the assumption that people use a confidence heuristic; they assume that a more confident advisor makes more categorically correct judgments and is more knowledgeable.
This paper discusses the conditions under which an agent is rationally permitted to leave some uncertain propositions relevant to her decision out of her deliberation. By relying on the view that belief involves a defeasible disposition to treat a proposition as true in one's reasoning, we examine the conditions under which such a disposition can be overridden and under which an agent should take into account her uncertainty as to a proposition she believes in the course of a particular deliberation. We argue that, in some contexts, an agent can be faced with the choice of either accepting or not accepting a proposition she believes in the course of her deliberation. We provide a description of such higher-order deliberations within the framework of expected utility theory and draw conclusions regarding the phenomenon of pragmatic encroachment on knowledge.
Games and Economic Behavior, 2013
Many real-life decisions have to be taken on the basis of probability judgements of which the decision maker is not entirely sure. This paper develops a decision rule for taking such decisions, which incorporates the decision maker's confidence in his probability judgements according to the following maxim: the larger the stakes involved in a decision, the more confidence is required in a probability judgement for it to play a role in the decision. A formal representation of the decision maker's confidence is proposed and used to formulate a family of decision models conforming to this maxim. A natural member of this family is studied in detail. It is structurally simpler than other recent models of decision under uncertainty, which may make it easier to apply to practical decisions, whilst being axiomatically sound, permitting the separation of beliefs and tastes, and allowing comparative statics analysis of attitudes to choosing in the absence of confidence.