Rarity, pseudodiagnosticity and Bayesian reasoning

Three experiments investigated the effect of rarity on people's selection and interpretation of data in a variant of the pseudodiagnosticity task. For familiar (Experiment 1) but not for arbitrary (Experiment 3) materials, participants were more likely to select evidence so as to complete a likelihood ratio when the initial evidence they received was a single likelihood concerning a rare feature. This rarity effect with familiar materials was replicated in Experiment 2 where it was shown that participants were relatively insensitive to explicit manipulations of the likely diagnosticity of rare evidence. In contrast to the effects for data selection, there was an effect of rarity on confidence ratings after receipt of a single likelihood for arbitrary but not for familiar materials. It is suggested that selecting diagnostic evidence necessitates explicit consideration of the alternative hypothesis and that consideration of the possible consequences of the evidence for the alternative weakens the rarity effect in confidence ratings. Paradoxically, although rarity effects in evidence selection and confidence ratings are in the spirit of Bayesian reasoning, the effect on confidence ratings appears to rely on participants thinking less about the alternative hypothesis.