| Summary: | Although the Analysis
of Competing Hypotheses method (ACH) is a structured analytic technique
promoted in several intelligence communities for improving the quality of
probabilistic hypothesis testing, it has received little empirical testing.
Whereas previous evaluations have used numerical evidence assumed to be
perfectly accurate, in the present experiment we tested the effectiveness of
ACH using a judgment task that presented participants with uncertain evidence
varying in source reliability and information credibility. Participants (N =
227) assigned probabilities to two alternative hypotheses across six cases that
systematically varied case features. Across multiple tests of coherence, the
ACH group showed no advantage over a no-technique control group. Both groups
showed evidence of subadditivity, unreliability, and overly conservative
non-Bayesian judgments. The ACH group also showed pseudo-diagnostic weighting
of evidence. The findings do not support the claim that ACH is effective at
improving probabilistic judgment.
|
|---|
