Abstract: "Opinion of geological experts is often formed despite a paucity of data, and is usually based on prior experience. In such situations humans employ heuristics (rules of thumb) to aid analysis and interpretation of data. As a result, future judgements are bootstrapped from, and hence biased by, both the heuristics employed and prior opinion.
This paper reviews the causes of bias and error inherent in prior information derived from the probabilistic judgements of people. Parallels are developed between the evolution of scientific opinion on one hand, and the limits of rational behaviour on the other. We show that the combination of data paucity and commonly employed heuristics can lead to herding behaviour within groups of experts. Elicitation theory mitigates the effects of such behaviour, but a method to estimate reliable uncertainties on expert judgements remains elusive.
We have also identified several key directions in which future research is likely to lead to methods that reduce such emergent group behaviour, thereby increasing the probability that the stock of common knowledge converges in a stable manner towards facts about the Earth as it really is. These include a) measuring the frequency with which different heuristics tend to be employed by experts within the geosciences; b) develop geoscience-specific methods to reduce biases originating from the use of such heuristics, c) create methods to detect scientific herding behaviour; and d) research how best to reconcile opinions from multiple experts to obtain the best probabilistic description of an unknown, objective reality (in cases where one exists)."
Baddeley, Curtis and Wood (2004)
"This reply clarifies what G. Gigerenzer’s (e.g., 1991, 1994; Gigerenzer & Murray, 1987 ) critique of the heuristics-and-biases approach to statistical reasoning is and is not about. At issue is the imposition of unnecessarily narrow norms of sound reasoning that are used to diagnose so-called cognitive illusions and the continuing reliance on vague heuristics that explain everything and nothing. D. Kahneman and A. Tversky (1996) incorrectly asserted that Gigerenzer simply claimed that frequency formats make all cognitive illusions disappear. In contrast, Gigerenzer has proposed and tested models that actually predict when frequency judgments are valid and when they are not. The issue is not whether or not, or how often, cognitive illusions disappear. The focus should be rather the construction of detailed models of cognitive processes that explain when and why they disappear. A postscript responds to Kahneman and Tversky’s (1996) postscript."
Gigerenzer (1996)
"Biases in probabilistic reasoning are affected by alterations in the presentation of judgment tasks. In our experiments, students made likelihood judgments that an event was produced by various causes. These judgments were made in terms of probability, relative frequency or absolute frequency on a full or a pruned list of causes. When they had little personal experience of the event (causes of death), the pruning bias was smaller with relative frequencies than with absolute frequencies or probabilities. When they had more personal experience of the event (missing a lecture), the bias was less with both types of frequency than with probability but still lowest with relative frequency. We suggest that likelihood information is usually stored as relative frequencies when it has been obtained from public sources but that it is based on event counts when it is derived from personal experience."
Harries and Harvey (2000)
"The study investigated the probabilistic misconceptions of Chinese students, and whether selected misconceptions could be overcome through a focused teaching intervention. A questionnaire was given to a 567 Chinese students from grades 6, 8 and 12 and two streams (advanced and ordinary). In addition 64 of the students were interviewed. Fourteen groups of misconceptions were identified. The SOLO taxonomy was used in this study to describe students� hierarchical understanding levels on the concept of probability. It was found that, generally there was no improvement in developmental level from grades 6 and 8, the two grades without any formal probability training. Grade 12 students have a better understanding than the younger students. The results of the activity-based short-term teaching programme with grade 8 students show that even a short intervention can help students overcome some of their misconceptions."
Jun and Pereira-Mendoza (2002)