A novel approach to reducing uncertainty: The group Delphi (original) (raw)
Related papers
Risk Analysis, 2007
Expert panels and averaging procedures are common means for coping with the uncertainty of effects of technology application in complex environments. We investigate the connection between confidence and the validity of expert judgment. Moreover, a formative consensus building procedure (FCB) is introduced that generates probability statements on the performance of technologies, and we compare different algorithms for the statistical aggregation of individual judgments. The case study refers to an expert panel of 10 environmental scientists assessing the performance of a soil cleanup technology that uses the capability of certain plants to accumulate heavy metals from the soil in the plant body (phytoremediation). The panel members first provided individual statements on the effectiveness of a phytoremediation. Such statements can support policymakers, answering the questions concerning the expected performance of the new technology in contaminated areas. The present study reviews (1) the steps of the FCB, (2) the constraints of technology application (contaminants, soil structure, etc.), (3) the measurement of expert knowledge, (4) the statistical averaging and the discursive agreement procedures, and (5) the boundaries of application for the FCB method. The quantitative statement oriented part of FCB generates terms such as: "The probability that the concentration of soil contamination will be reduced by at least 50% is 0.8." The data suggest that taking the median of the individual expert estimates provides the most accurate aggregated estimate. The discursive agreement procedure of FCB appears suitable for deriving politically relevant singular statements rather than for obtaining comprehensive information about uncertainties as represented by probability distributions.
A Novel Approach to Reducing Uncertainty
1991
A variation on the conventional Delphi was used to assemble an informational summary of expert opinion regarding the risks involved with the application of sewage sludge to farmland. The aim was to reduce uncertainties surrounding the associated health and environmental risks so agreement among citizens, farmers, and regulators could be reached. An expert panel was assembled for one day to take part in a structured communication process modeled after the Delphi. A two-part questionnaire using Likert scaling and open questions was iterated among rotating subgroups to build consensus and define disagreement. Plenary discussions were held between iterations to foster peer review. There was consensus about the risks of heavy metals, pathogens, and nutrients; but clear disagreement about the risks of organic toxins. Existing state regulations were deemed inadequate only for lead and some organic toxins. Expert quantitative ratings were found to differ radically for two hypothetical contexts: academic and public. Statement of the Problem: Decision Under Uncertainty Rarely do decision makers feel they have enough information available to make a decision about which they can be confident. Among regulators, politicians, managers, and public officials, considerable effort is expended in reducing uncertainty surrounding decisions. Uncertainty reflects incomplete knowledge. Depending on the type of knowledge, different techniques have proved effective at resolving uncertainties. Uncertain knowledge about factual evidence (random error, statistical error) can be resolved by scientific methodology and the peer review process, whereas uncertain knowledge about values (social priorities or preferences) can be reduced by political discourse or democratic voting procedures. But the domains of these two procedures to reduce uncertainty are not inclusive to all kinds of knowledge. Uncertainties about predicting future events and uncertainties
2000
Decision analysts are frequently called on to help inform decisionmakers in situations where there is considerable uncertainty. In such situations, expert elicitation of parameter values is frequently used to supplement more conventional research. This paper develops a formal protocol for expert elicitation with large, heterogeneous expert panels. We use formal survey methods to take advantage of variation in individual expert uncertainty and heterogeneity among experts as a means of quantifying and comparing sources of uncertainty about parameters of interest. We illustrate use of this protocol with an expert elicitation on the distribution of U.S. foodborne illness from each of 11 major foodborne pathogens to the consumption of one of 11 categories of food. Results show how multiple measures of uncertainty, made feasible by use of a large panel of experts, can help identify which of several types of risk management actions may be most appropriate.
Using Experts' Opinions Through Delphi Technique
The Delphi technique provides different opportunities to researchers than survey research. Essential components of the Delphi technique include the communication process, a group of experts, and essential feedback. This paper provides the foundations of the Delphi Technique, discusses its strengths and weaknesses, explains the use and stages followed, discusses panel selection, and explains how consensus among participants is reached.
Risk Analysis, 2002
This paper presents a protocol for a formal expert judgment process using a heterogeneous expert panel aimed at the quanti®cation of continuous variables. The emphasis is on the process's requirements related to the nature of expertise within the panel, in particular the heterogeneity of both substantive and normative expertise. The process provides the opportunity for interaction among the experts so that they fully understand and agree upon the problem at hand, including qualitative aspects relevant to the variables of interest, prior to the actual quanti®cation task. Individual experts' assessments on the variables of interest, cast in the form of subjective probability density functions, are elicited with a minimal demand for normative expertise. The individual experts' assessments are aggregated into a single probability density function per variable, thereby weighting the experts according to their expertise. Elicitation techniques proposed include the Delphi technique for the qualitative assessment task and the ELI method for the actual quantitative assessment task. Appropriately, the Classical model was used to weight the experts' assessments in order to construct a single distribution per variable. Applying this model, the experts' quality typically was based on their performance on seed variables. An application of the proposed protocol in the broad and multidisciplinary ®eld of animal health is presented. Results of this expert judgment process showed that the proposed protocol in combination with the proposed elicitation and analysis techniques resulted in valid data on the (continuous) variables of interest. In conclusion, the proposed protocol for a formal expert judgment process aimed at the elicitation of quantitative data from a heterogeneous expert panel provided satisfactory results. Hence, this protocol might be useful for expert judgment studies in other broad and/or multidisciplinary ®elds of interest.
Deliberative disjunction: Expert and public understanding of outcome uncertainty
2012
Many environmental and risk management decisions are made jointly by technical experts and members of the public. Frequently, their task is to select from among management alternatives whose outcomes are subject to varying degrees of uncertainty. Although it is recognized that how this uncertainty is interpreted can significantly affect decision-making processes and choices, little research has examined similarities and differences between expert and public understandings of uncertainty.
A group agreement-based approach for decision making in environmental issues
Environmental Modelling & Software, 2012
A decision-making process focusing on environmental issues is extremely complex because of the intricacy of the real-world systems. Such systems are subjected to many uncertain events, which make planning, modeling, and predicting performances and treatment inherently complicated. Typically, a decision-making process focusing on environmental problems is ill structured, uncertain, vague, and multidimensional and is often based on the opinions of experts with different viewpoints. A common problem is how to aggregate the opinions of experts, which might be diverse and sometimes even opposing. This paper presents a new method for aggregating experts' opinions and introduces a new aggregation operator MaxAgM, based on Shannon entropy, which maximizes the agreement of experts' opinions. Our method can be applied toward aggregating expert proposals that were expressed by crisp as well as fuzzy quantities to propose a binary solution or to estimate a numerical value of some parameter. A specialized software package MaxAgr was developed to optimize agreement drawn from experts' proposals. Application of the method and the software is illustrated in a case study on flood risk management.