Restoration of Confidence: Polling’s Comeback from 1948

By Stephen Earl Bennett

 

Warren Mitofsky’s death late last summer was a blow to students of elections, public opinion, and polling and survey research. Mitofsky pioneered exit polls inside and outside of the United States, and, with the late Joseph Waksberg, he helped improve random digit dialing (RDD), the method most widely used to contact survey respondents today.

Those of us who utilize surveys and polls tend to forget how close this field came to coming to an end after the fiasco of 1948. That was the year pollsters such as George Gallup, Archibald Crossley, and Elmo Roper—pioneers of scientific polling—predicted Thomas Dewey would defeat Harry Truman in the presidential election. A funny thing happened on the way to Mr. Dewey’s inauguration: As Mitofsky himself once reported, Gallup’s and Crossley’s estimates were off by nine percentage points, while Roper missed the election’s outcome by nineteen.

In addition to prompting a loss of public confidence and a flurry of cancelled newspaper subscriptions, the pollsters’ failure to predict the outcome of the 1948 presidential election accurately led to a congressional investigation and at least two scholarly books dedicated to assessing what went wrong (one edited by Norman Meier and Harold Saunders, the other by Frederick Mosteller and colleagues), and proposing improvements.

 

As Amy Fried has recently reminded us, critics of polling—chief among them Lindsay Rogers—were quick to pounce on the mistakes made in 1948. In his 1949 book, The Pollsters, Rogers—a long-time professor of law at Columbia—not only faulted polling experts for failing to grasp American political history, institutions, and practice, but also for their misconception of the nature of public opinion, and even how to measure it. The fact that Rogers is little remembered today does not discount the damage done to polling and survey research at the time.

In the wake of the disaster, polling organizations made methodological and administrative changes. Quota polls were replaced by probability sampling. Equally important, perhaps, pollsters began polling right up the day of an election. (In 1948, a 1998 article by Mitofsky tells us, Roper finished polling in September, Crossley’s last poll was October 18, and Gallup stopped polling after October 28. Election day was November 2. Ending polls this early was problematic, for trends indicated Truman was closing the gap on Dewey.)

Nevertheless, the pollsters’ “comeback” took a long time. As Irving Crespi notes, it wasn’t until 1960—when pollsters predicted that, whether Kennedy or Nixon won, the margin would be very small—that public confidence in polling was restored.

 

Among others, Warren Mitofsky was instrumental in that restoration of confidence. He made many contributions, of course, both to that process and to the polling field; his stints as president of the American Association for Public Opinion Research and the National Council of Public Polls ought not be sloughed off. Still, it is more than sufficient here to consider his contributions to exit polling, for which is most recognized, and, especially, to the improvement of RDD sampling.  

In 1967, after doing survey work for the U.S. Census Bureau, Mitofsky began conducting exit polls for CBS News and became widely known as “the father” of exit polling. At CBS, Mitofsky developed the projection and analysis systems used to conduct national exit polls. Later, he headed the consortium of news organizations—under the umbrella organization, Voter Research and Surveys—that did exit polls in the 1990 and 1992 contests. In 1993, he founded Mitofsky International, which did exit polling in countries such as Russia, Mexico, the Philippines, and elsewhere. From 2003 until his death, Mitofsky International and Edison Media Research conducted exit polls in America for the Associated Press and the national TV networks.

Equally important were Mitofsky’s contributions to RDD, for this technological shift in the way surveys are done provides useful purchase on the ways the field is changing today. Prior to the advent of RDD in the 1970s, pollsters relied largely on professional interviewers who interviewed respondents in the respondents’ homes. If they did telephone interviewing, they used mostly local telephone directories for selecting respondents. Each method presented problems.

Using trained interviewers to conduct in-person interviews was expensive, time-consuming, and always a potential source of error. Although some thought that in-person interviews provided higher-quality data than telephone interviews, others cast doubts on that notion. Nevertheless, early methods of telephone interviews were problematic not only because they missed potential respondents with unlisted telephone numbers, but also because directories rather quickly got out of date.

Random digit dialing initially emerged in the 1960s as a means to overcome what James Frey calls “undercoverage in telephone directories.” By the 1970s, RDD had been adopted by several academic survey organizations, and was rapidly becoming pollsters’ “method of choice” for contacting respondents. Once Mitofsky and Waksberg improved the technique by developing a two-stage, cluster design to further reduce “unproductive dialings,” RDD became even more popular among pollsters and survey researchers. Combined with a CATI (Computer Assisted Telephone Interviewing) system, RDD made polling relatively inexpensive and very fast, and, assuming interviewers were well-trained and supervised, probably less susceptible to biases introduced by interviewer effects.

 

Public opinion research continues to evolve. New technologies—such as “caller ID” and, especially, cell phones—will very likely entail significant changes in telephone-based interviewing. Survey researchers are increasingly vexed by declining response rates; it is estimated that polls miss between 25 and 30 percent of the public, and, if these people are systematically different from those who agree to be interviewed, the resulting poll, no matter how otherwise technically sophisticated, is biased.

Even when not coping with technological and social changes, surveys may confront serious problems. Vastly uneven distributions of political information across the public, for example, may severely crimp their capacity to shed light on, as Scott Althaus puts it, “what the people really want.” If Robert Weissberg is right, the way survey organizations pose some queries introduces systematic, and serious, bias into their results.

 

Students of elections, public opinion, and survey methodology will have to cope with these—and a myriad of other problems—without Warren Mitofsky. It may be that survey researchers will “muddle through,” just as they have done after other pioneers of the field, such as George Gallup Sr., Archibald Crossley, and Elmo Roper, to name just three, have passed from the scene.

As we who rely on the polls face the future, something that Mitofsky wrote in 1999 ought to be our touchstone:

 

I do not want to abandon tested and proven survey methods that are based on a solid foundation of probability theory until we have a new theory-based methodology for doing so… A growing number of survey researchers are unfortunately being led to the rocks like Ulysses’ sailors following the Siren call of cheap, but worthless, data.

 

Mitofsky knew the field was facing changes, but he opposed change for change’s sake.

 

 

Stephen Earl Bennett is adjunct professor of political science at the University of Southern Indiana.

 

Additional Reading

 

Althaus, Scott L. 2003. Collective preferences in democratic politics: Opinion surveys and the will of the people. Cambridge: Cambridge University Press.

Asher, Herbert. 2004. Polling and the public: What every citizen should know. 6th ed. Washington, DC: C. Q. Press.

Backstrom, Charles H., and Gerald D. Hursch. 1963. Survey research. Evanston, IL: Northwestern University Press.

Bradburn, Norman M., and Seymour Sudman. 1988. Polls and Surveys: Understanding What They Tell Us. San Francisco: Jossey-Bass.

Brehm, John. 1993. The phantom respondents: Opinion surveys and political representation. Ann Arbor, MI: The University of Michigan Press.

CBS News. 2006. Exit poll pioneer Warren Mitofsky Dies, September 3.

Crespi, Irving. 1989. Public opinion, polls, and democracy. Boulder, CO: Westview Press.

Curtin, Richard, Stanley Presser, and Eleanor Singer. 2005. Changes in telephone survey nonresponse over the past quarter century. Public Opinion Quarterly 69:87-98.

Daves, Robert P., and Frank Newport. 2005. Pollsters under attack: 2004 Election incivility and its consequences. Public Opinion Quarterly 69:670-81.

Farnsworth, Stephen J., and S. Robert Lichter. 2005. The nightly news nightmare revisited: Network television’s coverage of the 2004 presidential election. Paper presented at the annual meeting of the American Political Science Association, Washington, DC.

Fowler, Floyd J. Jr. 1984. Survey research methods. Beverly Hills, CA: Sage.

Frankovic, Kathleen A. 1992. Technology and the changing landscape of media polls. In Media polls in American politics, ed. Thomas E. Mann and Gary R. Orren. Washington, DC: The Brookings Institution.

Frey, James H. 1983. Survey research by telephone. Beverly Hills, CA: Sage.

Fried, Amy. 2006. The forgotten Lindsay Rogers and the development of American political science. American Political Science Review 100:555-61.

Keeter, Scott. 2006. The impact of cell phone noncoverage bias on polling in the 2004 presidential election.” Public Opinion Quarterly 70:88-98.

Klecka, William R., and Alfred J. Tuchfarber. 1974. Random digit dialing as an efficient method for political polling. The Georgia Political Science Association Journal 2:133-51.

Klecka, William R., and Alfred J. Tuchfarber. 1978. Random digit dialing: A comparison to personal surveys. Public Opinion Quarterly 42:105-14.

Ladd, Everett Carll, and John Benson. 1992. The growth of news polls in American politics. In Media polls in American politics, ed. Thomas E. Mann and Gary R. Orren. Washington, DC: The Brookings Institution.

Lindeman, Mark, and Rick Brady. 2006. Behind the controversy: A primer on U.S. presidential exit polls. Publicopinionpros.com. January. 

Meier, Norman C., and Harold W. Saunders, eds. 1949. The polls and public opinion. New York: Henry Holt.

Mitoksky, Warren J. 1991. A short history of exit polls. In Polling and presidential election coverage, ed. Paul J. Lavrakas and Jack K. Holley. Newbury Park, CA: Sage.

Mitofsky, Warren J. 1998. The Polls. Review: Was 1996 a worse year for polls than 1948? Public Opinion Quarterly 62:230-49.

Mitofsky, Warren J. 1999. “Pollsters.com.” The Public Perspective 10, no. 4: 24-26.

Mosteller, Frederick, Herbert Hyman, Philip J. McCarthy, Eli S. Marks, and David B. Truman. 1949. The pre-election polls of 1948—Report to the Committee on Analysis of Pre-election Polls and Forecasts. New York: Social Science Research Council.

Public Opinion Quarterly. 2003. Symposium: Election night 2000 in perspective. Public Opinion Quarterly 67:2-58.

Rogers, Lindsay. 1949. The pollsters: Public opinion, politics, scholarship, and democracy. New York: Alfred A. Knopf.

Rogers, Theresa F. [1976] 1989. Interviews by telephone and in person: Quality of responses and field performance. In Survey research methods: A primer, ed. Eleanor Singer and Stanley Presser. Chicago: University of Chicago Press.

Smith, Tom W. [1983] 1989. The hidden 25 percent: An analysis of nonresponse on the 1980 General Social Survey. In Survey research methods: A primer, ed. Eleanor Singer and Stanley Presser. Chicago: University of Chicago Press.

Steeh, Charlotte G. [1981] 1989. Trends in non-response rates, 1952-1979. In Survey research methods: A primer, ed. Eleanor Singer and Stanley Presser. Chicago: University of Chicago Press.

Traugott, Michael W., and Paul J. Lavrakas. 2004. The voter’s guide to election polls. 3rd ed. Lanham, MD: Rowman & Littlefield.

Waksberg, Joseph. 1978. Sampling methods for random digit dialing. Journal of the American Statistical Association 73:40-46.

Weisberg, Herbert F., and Bruce Bowen. 1977. Introduction to survey research and data analysis. San Francisco: W. H. Freeman.

Weissberg, Robert. 2002. Polling, policy, and public opinion: The case against heeding the “voice of the people. New York: Palgrave Macmillan.