Drawing Inferences from Self-Selected Samples

During the course of the rhetoric surrounding the 1984 Presidential election campaign in the United States there were a variety of statements made that gave me pause. For example, I heard candidate Ferraro explain her poor showing in pre·election polls by saying. " I don't believe those polls. If you could see the enthusiasm for our candidacy 'out there' you wouldn't believe them either. " Obviously, trying to estimate one's popularity in the entire voting population from the enthusiasm of your supporters at political rallies is not likely to yield accurate results. I suspect that trying to statistically adjust the " rally estimate" through the use of the demographic characteris· tics of those who attend would not have belped enough to be usefuL A modest survey on a more randomly chosen sample would surely have been better. At about the same time, Secretary of Education Terrell Bell released a table entitled State Education Statistics. Among other bits of information, it contained the mean scores on the Scholastic Aptitude Test (the SAT) for 22 of the states. The College Board had previously released these mean scores for all states. At this point the mass media began carrying reports inter preting the differences. The Reagan White House pointed out that spending more money on education was not the way to improve educational outcomes. To support this they pointed to the mean SAT scores of Connecticut and New Hampshire. New Hampshire had modestly higher SAT scores but lower . . per pupil expenditure.