New findings on unconscious versus conscious thought in decision making: additional empirical data and meta-analysis. | Judgment and Decision Making | Cambridge Core (original) (raw)

3.1.1 Study selection

The purpose of the meta-analysis was to compile all relevant study data that bear on the benefit of unconscious thought for normative decision making. The studies carried out by Dijksterhuis and colleagues show high methodological homogeneity and largely reflect the method used in the empirical study described here. The focus of the meta-analysis was restricted to those studies that directly compared unconscious and conscious thought with respect to a decision making task after the presentation of a large amount of stimulus material for different choice alternatives.

Studies on incubation, which arguably also deal with unconscious thought, were not included. Incubation studies usually present the participant with a problem solving task (e.g., Vul & Pashler, 2007) or judge the creative output after a period of incubation (e.g., Dijksterhuis & Meurs, Reference Dijksterhuis and Meurs2006). Both of these approaches are clearly different from the methodological selection criterion outlined and thus do not qualify for the meta-analysis.

Other studies that deal more directly with unconscious thought but also were not included are those that operationalise good decision making as post-choice satisfaction (e.g., Dijksterhuis & van Olden, Reference Dijksterhuis and van Olden2006). Judging the degree of content with an item after a period of time does not appear to be functionally equivalent to making a relatively instantaneous decision based on comparative judgment. Thus, including studies which use post-choice satisfaction as the dependent variable would have introduced further error variance into the analysis beyond the normal sampling variability and ultimately biased the overall effect size estimate.

In order to be included a study had to focus on normative decision making, compared unconscious and conscious thought conditions, operationalise unconscious thought as a distraction period following a standardized encoding period, present each piece of information for the same amount of time or at least make provisions for the participants to do so, make participants choose from or evaluate multiple, specified choice options, and finally operationalise choice options in terms of degree of good decision making rather than choosing the correct out of a number of incorrect options.

Studies were sourced using the psychinfo database (1806-present) and Google Scholar with the keywords “unconscious thought”, “decision making”, and “incubation” and results were narrowed down with combinations of these. Additionally, references cited in Dijksterhuis Reference Dijksterhuis(2004b), Dijksterhuis et al. Reference Dijksterhuis, Bos, Nordgren and van Baaren(2006), and Dijksterhuis et al. (in press) were checked for compliance with the selection criterion. Finally, studies citing either Dijksterhuis Reference Dijksterhuis(2004b) or Dijksterhuis et al. Reference Dijksterhuis, Bos, Nordgren and van Baaren(2006) were checked.

Two sets of data were identified to be of potential usefulness to this analysis but could not be obtained. These were one data set pertaining to decision quality after varying intervals of unconscious thought (Dijksterhuis, Reference Dijksterhuis2004a, as cited in Dijksterhuis and Nordgren, Reference Dijksterhuis and Nordgren2006) and a study that used the same stimulus material as the present study (Dijksterhuis, Bos, van Baaren, & van der Leij, in prep., as cited in Dijksterhuis et al., in press). However, instead of a contrast between unconscious thought and conscious thought, it focuses on the comparison between immediate decision making and unconscious thought.

Overall, only two published studies comprising six experiments were deemed to sufficiently fulfill the original selection criteria. These experiments were number 1 to 4 in Dijksterhuis Reference Dijksterhuis(2004b), Experiments 1 and 2 in Dijksterhuis et al. Reference Dijksterhuis, Bos, Nordgren and van Baaren(2006). All other studies cited in this meta-analysis have not yet been published and had either been cited in work by Dijksterhuis or came to my attention through word of mouth. This class of unpublished works included five additional studies incorporating 10 sets of relevant data. These data were extracted from Experiments 1 and 2 of the Ham, Bos and Doorn study (Ham, Bos, & Doorn, submitted), Experiments 1, 2 and 3 of Newell et al.’s (in press) study, Experiments 1 and 2 reported by Payne et al. Reference Payne, Samper, Bettman and Luce(2007), aggregate findings reported by Phillips et al. Reference Phillips, González-Vallejo, Bellezza, Chimeli, Harman, Lassiter and Lindberg(2007), as well as from a study by Lerouge (submitted).

Experiment 2 of the Dijksterhuis et al. Reference Dijksterhuis, Bos, Nordgren and van Baaren(2006) was the most comparable to the present empirical study. The main points of distinction were the difference in dependent variable scaling (50 point visual analogue versus 10 point rating) and the presentation time per item (8 versus 5 seconds). Experiment 1 (Dijksterhuis, Bos, Nordgren, & van Baaren, Reference Dijksterhuis, Bos, Nordgren and van Baaren2006) looked at proportional differences between participants in the unconscious and conscious thinking groups with respect to selecting the best choice alternative. In Dijsterhuis’ 2004 study, Experiments 1 and 2 were fairly similar to the present experiment, albeit the stimulus material was different. Experiment 3 (Reference DijksterhuisDijksterhuis, 2004b) was functionally similar to Experiment 2 but with yet again other stimulus materials. It also looked at the importance an individual places on the different information attributes that are presented and correlated this with the decision score. The present experiment had a similar index, however, in a more rudimentary form.

Experiment 4 (Dijksterhuis, Bos, Nordgren, & van Baaren, Reference Dijksterhuis, Bos, Nordgren and van Baaren2006) was not directly concerned with alternative selection but rather with the re-attribution of item content to its source. In comparison to the other experiments, the encoded items did not have to be integrated into global judgments but rather had to be remembered individually. The experiment also fostered response speed as a second dependent variable, but these results were not included here. The decision to include this experiment is debatable, but it seemed sufficiently suitable as the attribution still involved some decision making, although on an item-by-item level, after the same encoding procedure as in the other experiments.

Ham et al.’s experiments (Ham, Bos, & Doorn, submitted) were again functionally quite similar to the present experiment but focused on justice judgments rather than consumer choice. Newell et al. (in press) presented four experiments. The first three experiments were included in the meta-analysis, the fourth one was not. In all experiments Newell et al. collected data on a variety of dependent measures, such as a recall test for attributes, but only the data on choice preferences were used for the meta-analysis. Experiment 3 is virtually identical with the empirical study presented here in that it is also a direct replication of Dijksterhuis et al. Reference Dijksterhuis, Bos, Nordgren and van Baaren(2006) using the same materials. In contrast, Experiments 1 and 2 also used the same normative choice methodology but with original stimulus material. In Experiment 2 an additional experimental group is included: conscious thinkers that have access to the relevant information during the decision making period. Data from this group were not included in the meta-analysis but based on the results reported by Newell et al., their inclusion would not have much difference. Experiment 4 investigated the impact of primacy and recency effect on conscious and unconscious choices. It was unsuitable for inclusion to the meta-analysis as it only contained two choice alternatives but mainly because the two cars were both described by 10 positive attributes so that there was no normatively ‘rational’ choice.

Payne et al. Reference Payne, Samper, Bettman and Luce(2007) carried out two experiments that explored the boundary conditions of the unconscious thought effect. In addition to the contrast between usual conscious thought and unconscious thought, they included a further condition where the decision interval for conscious thought was self-paced. Only the conscious thought condition with a fixed time interval between presentation and decision was included in the meta-analysis. Both of Payne et al.’s experiments used a design whereby different numerical values were assigned to each piece of information and which resulted in different expected utilities for each choice option. Contrary to other studies, this method required successful and precise analytic integration of the values for each option, instead of choices based on the gist of all items.

The data set obtained from Phillips et al.’s study is part of a larger online study. Phillips et al. looked at, among other things, the effect of item presentation order, but the data presented here has been collapsed across all presentation order positions.

Finally Lerouge’s study (submitted) dealt explicitly with consumer choice but primarily investigated the impact of configural versus featural processing as a moderator of unconscious thought. Only Experiment 2 was included, since Experiment 1 generated data only for immediate and unconscious thought, but not for conscious thought. All experiments listed here aside from Dijksterhuis et al. Reference Dijksterhuis, Bos, Nordgren and van Baaren(2006) also included the immediate choice condition, which is of subordinate interest in the present study.

The selection of studies was homogeneous in terms of general methodological approach, albeit with much variation in the exact procedure. Table 5 provides an overview of some key aspects that lend themselves to meta-analytic investigation. A variety of decision processes and information conditions is represented. Decision making success is identified as choosing the best option and comparing the proportions across groups or by rating each choice option and then deriving some form of mean difference between groups. Ham et al.’s study presents an exception to these two patterns. Regarding the information presentation, most experiments presented the items one by one, but few presented multiple pieces of information simultaneously. Most studies included three conditions: immediate, conscious and unconscious decision making, but some (Newell et al., in press; Payne et al., Reference Payne, Samper, Bettman and Luce2007) had an additional condition to test specific predictions. The gender ratio also varied, although females predominated in most studies. There were minor variations with respect to the number of attributes for each choice option or the interval length after the presentation of the stimulus material (with the exception of Newell et al., Experiment 2).

Table 5:Overview of key features of the experiments included in the meta-analysis.

3.1.2 Meta analytic procedure

For each of the selected experiments, standardized effect sizes (g) were calculated following the guidelines presented in Grissom and Kim (2005) for mean differences and DeCoster Reference DeCoster(2004) for proportion differences. In line with recommendations of Schmidt, Oh and Hayes (in press) a random effects model was chosen for the analysis. Weights and the mean effect size were calculated using the procedure described in Borenstein, Hedges, Higgins, & Rothstein (Reference Borenstein, Hedges, Higgins and Rothstein2008). Gender ratio, presentation time per item, and decision (or better distraction) interval span were defined as moderators. Since the moderator variable investigated in Lerouge (submitted) had an appreciable effect on the results, the experiment was treated as two separate data sets for the meta-analysis in order to preserve this noteworthy contrast. This decision did not affect the final estimates of the population effect size and margin of error. The data as presented below overestimate the parameters by about 0.006, in contrast to estimates that treat Lerouge’s study as a single data set.