Public Opinion Poll

It has been suggested that this article or section be merged with Opinion poll. (Discuss)

A public opinion poll is a pseudoscientific survey of attitudes from a particular sample of the population at interest. Opinion polls supposedly represent the opinions of the total population by asking a sample of size N, where N is large enough for reliable results (usually expressed as a 95% confidence of +/- Z%). For most polls, N = 400 to 1000 people.

The sample had to be a valid representation of the population, which means that each person in the target population has a known chance of being sampled.[1] Unscientific public opinion polls, made by collecting information from a haphazard sample, date from the straw polls in American elections in the 1820s. As soon as modern statistical sampling theory was invented around 1930, George Gallup and other advertisers started created nationwide public opinion polls, using about 1500 cases. The American public became fascinated with polls and they spread to many democratic countries by 1940. Totalitarian countries did not allow them.

## Contents

* 1 History of Polling
* 2 Potential for inaccuracy
* 2.1 Sampling error
* 2.2 Nonresponse bias
* 2.3 Response bias
* 2.4 Wording of questions
* 2.5 Coverage bias
* 3 Polling organizations
* 4 Famous Failures
* 5 Exit polls
* 6 Influence
* 7 Political betting markets vs. political opinion polls
* 7.1 Arguments on why political betting markets are better than political polls
* 7.2 Track record of the website ElectionBettingOdds.com in terms of accuracy
* 7.3 How accurate is the website ElectionBettingOdds.com?
* 7.4 2019: Judgement and Decision Making Journal: Another approach
* 8 References
* 9 Primary sources
* 10 See also
* 11 External links

## History of Polling[edit]

The first known example of an opinion poll was a local straw vote conducted by a newspaper The Harrisburg Pennsylvanian in 1824; it showed Andrew Jackson leading John Quincy Adams by 335 votes to 169 in the contest for the presidency. Such straw votes--unweighted and unscientific-- gradually became more popular; but they remained local, usually citywide phenomena. In 1916, the large-circulation U.S. magazine Literary Digest embarked on a national survey (partly as a circulation-raising exercise) and correctly predicted Woodrow Wilson's election as president. Mailing out millions of postcards and simply counting the returns, the Digest correctly called the following four presidential elections.

In 1936, however, the Digest came unstuck. Its 2.3 million "voters" constituted a huge sample; however they were generally more affluent Americans who tended to have Republican sympathies. The Literary Digest saw the bias but did not know how to correct it. The week before election day, it reported that Alf Landon was far ahead of Franklin D. Roosevelt. At the same time, George Gallup conducted a far smaller, but more scientifically-based survey, in which he polled a demographically representative sample. Gallup correctly predicted Roosevelt's landslide victory. The Literary Digest went out of business soon afterwards, while the polling industry started to take off .

Gallup launched a subsidiary in Britain, where it correctly predicted Labour's victory in the 1945 general election, in contrast with virtually all other commentators, who expected the Conservative Party, led by Winston Churchill, to win easily.

By the 1950s, polling had spread to most democracies. Nowadays they reach virtually every country, although in more autocratic societies they tend to avoid sensitive political topics. In Iraq, surveys conducted soon after the 2003 war helped to measure the true feelings of Iraqi citizens to Saddam Hussein, post-war conditions and the presence of US forces.

For many years, opinion polls were conducted mainly face-to-face, either in the street or in people's homes. This method remains widely used, but in some countries it has been overtaken by telephone polls, which can be conducted faster and more cheaply. Because of the common practice of telemarketers to sell products under the guise of a telephone survey and due to the proliferation of residential call screening devices and use of cell phones, response rates for phone surveys have been plummeting. Mailed surveys have become the data collection method of choice among local governments that conduct a citizen survey to track service quality and manage resource allocation. In recent years, Internet and short message service (SMS, or text) surveys have become increasingly popular, but most of these draw on whomever wishes to participate rather than a scientific sample of the population, and are therefore not generally considered accurate.

## Potential for inaccuracy[edit]

### Sampling error[edit]

All polls based on samples are subject to sampling error which reflects the effects of chance in the sampling process. Too many of this group happen to fall in the sample, and too few of that group. This produces a bias that is inevitable when people are picked at random. The percentage error gets smaller as the sample size increases. The uncertainty is often expressed as a margin of error. A poll with a random sample of 1,000 people has margin of sampling error of 3% for the estimated percentage of the whole population. A 3% margin of error means that 95% of the time the procedure used would give an estimate within 3% of the percentage to be estimated. The margin of error can be reduced by using a larger sample, however if a pollster wishes to reduce the margin of error to 1% they would need a sample of around 10,000 people. In practice pollsters need to balance the cost of a large sample against the reduction in sampling error and a sample size of around 500-1,000 is a typical compromise for political polls. (Note that to get 500 complete responses it may be necessary to make thousands of phone calls.)[2]

The margin of error does not reflect other sources of error, such as poorly defined or ambiguous questions, or overlooking groups of people who might have different opinions. The Literary Digest sampled automobile and telephone owners in 1936 and thus got too many rich Republicans and too few poor Democrats. Today nearly all interviews are done by telephone or over the internet. Researchers today are studying whether people who only have cell phones are much different from people with land lines. (In general the differences are quites small.) Researchers have found that people who volunteer for a poll—say, on the internet—tend to be different; they are younger, better educated and more sure of their opinions.

### Nonresponse bias[edit]

Since some people do not answer calls from strangers, or refuse to answer the poll, poll samples may not be representative samples from a population. Because of this selection bias, the characteristics of those who agree to be interviewed may be markedly different from those who decline. That is, the actual sample is a biased version of the universe the pollster wants to analyze. In these cases, bias introduces new errors, one way or the other, that are in addition to errors caused by sample size. Error due to bias does not become smaller with larger sample sizes. If the people who refuse to answer, or are never reached, have the same characteristics as the people who do answer, the final results will be unbiased. If the people who do not answer have different opinions then there is bias in the results. In terms of election polls, studies suggest that bias effects are small, but each polling firm has its own formulas on how to adjust weights to minimize selection bias.[3]

### Response bias[edit]

Survey results may be affected by response bias, where the answers given by respondents do not reflect their true beliefs. This may be deliberately engineered by unscrupulous pollsters in order to generate a certain result or please their clients, but more often is a result of the detailed wording or ordering of questions (see below). Respondents may deliberately try to manipulate the outcome of a poll by e.g. advocating a more extreme position than they actually hold in order to boost their side of the argument or give rapid and ill-considered answers in order to hasten the end of their questioning. Respondents may also feel under social pressure not to give an unpopular answer. For example, respondents might be unwilling to admit to unpopular attitudes like racism or sexism, and thus polls might not reflect the true incidence of these attitudes in the population. If the results of surveys are widely publicized this effect may be magnified - the so-called "spiral of silence."

### Wording of questions[edit]

It is well established that the wording of the questions, the order in which they are asked and the number and form of alternative answers offered can influence results of polls. Thus comparisons between polls often boil down to the wording of the question. On some issues, question wording can result in quite pronounced differences between surveys. [4][5][6] This can also, however, be a result of legitimately conflicted feelings or evolving attitudes, rather than a poorly constructed survey.[7] One way in which pollsters attempt to minimize this effect is to ask the same set of questions over time, in order to track changes in opinion. Another common technique is to rotate the order in which questions are asked. Many pollsters also split-sample. This involves having two different versions of a question, with each version presented to half the respondents.

The most effective controls, used by attitude researchers, are:

* asking enough questions to allow all aspects of an issue to be covered and to control effects due to the form of the question (such as positive or negative wording), the adequacy of the number being established quantitatively with psychometric measures such as reliability coefficients, and

* analyzing the results with psychometric techniques which synthesize the answers into a few reliable scores and detect ineffective questions.

These controls are not widely used in the polling industry.

### Coverage bias[edit]

Another source of error is the use of samples that are not representative of the population as a consequence of the methodology used, as was the experience of the Literary Digest in 1936. For example, telephone sampling has a built-in error because in many times and places, those with telephones have generally been richer than those without. Several studies of mobile phone users by the Pew Research Center in the U.S. concluded that the absence of mobile users was not unduly skewing results, at least not yet. [8]

An oft-quoted example of opinion polls succumbing to errors was the British election of 1992. Despite the polling organizations using different methodologies virtually all the polls in the lead up to the vote (and exit polls taken on voting day) showed a lead for the opposition Labour party but the actual vote gave a clear victory to the ruling Conservative party.

In their deliberations after this embarrassment the pollsters advanced several ideas to account for their errors, including:

* Late swing. The Conservatives gained from people who switched to them at the last minute, so the error was not as great as it first appeared.

* Nonresponse bias. Conservative voters were less likely to participate in the survey than in the past and were thus underrepresented.

* The spiral of silence. The Conservatives had suffered a sustained period of unpopularity as a result of economic recession and a series of minor scandals. Some Conservative supporters felt under pressure to give a more popular answer.

The relative importance of these factors was, and remains, a matter of controversy, but since then the polling organizations have adjusted their methodologies and have achieved more accurate predictions in subsequent elections.

## Polling organizations[edit]

There are many polling organizations. The most famous remains the first one, the Gallup poll, created by George Gallup.

Other major polling organizations in the U.S. include:

* The Pew Research Center conducts polls concentrating on media and political beliefs.
* The Harris Poll.
* The Roper Poll.
* The World Public Opinion provides in-depth information and analysis on public opinion from around the world on international issues.
* Nielsen Ratings, virtually always for television.
* Rasmussen Reports (often accused of Republican bias- however, like FOX news, is only conservative by comparison to liberal groups)
* Garin Hart Yang (Democratic)
* Ayres, McHenry & Associates (Republican)
* Penn, Schoen & Berland Associates (Democratic)
* Moore Information (Republican)
* Frederick Polls (Democratic)
* OnMessage Inc. (Republican)
* Hickman-Maslin Research (Democratic)
* The Tarrance Group (Republican)
* Greenberg Quinlan Rosner (Democratic)
* Public Opinion Strategies (Republican)
* Quinnipiac Polls, run by Quinnipiac University in Hamden, Connecticut, and started as a student project.
* The National Opinion Research Center.
* Public Agenda, conducts research bridging the gap between what American leaders think and what the public really thinks.

In Britain the most notable "pollsters" are:

* MORI. This polling organization is notable for only selecting those who say that they are "likely" to vote. This has tended to favour the Conservative Party in recent years.
* YouGov, an online pollster.
* GfK NOP
* ICR/International Communications Research|ICR
* ICM (polling)|ICM
* Populus, official The Times pollster.

In Australia the most notable companies are:

* Newspoll - published in News Limited|News Limited's The Australian newspaper
* Roy Morgan Research - published in the Crikey email reporting service
* Galaxy Research|Galaxy Polling - published in News Limited|News Limited's tabloid papers
* ACNielsen|AC Nielsen Polling - published in Fairfax Media|Fairfax newspapers

In Canada the most notable companies are:

* Angus Reid Strategies
* Ipsos-Reid
* Environics
* Ekos
* Decima
* Leger
* CROP

In Nigeria the most notable polling organization is:

* NOI poll|NOI-Gallup poll

All the major television networks, alone or in conjunction with the largest newspapers or magazines, in virtually every democratic country, operate polling operations.

Several organizations monitor the behaviour of pollsters and the use of polling data, including PEW and, in Canada, the Laurier Institute for the Study of Public Opinion and Policy.[9]

## Famous Failures[edit]

The best-known failure of opinion polling to date in the U.S. was the prediction in 1948 that Thomas Dewey would defeat Harry S. Truman. Major polling organizations, including Gallup and Roper, indicated a landslide victory for Dewey.

In Britain, most polls failed to predict the Conservative election victories of 1970 and 1992, and Labour's victory in 1974. However, their figures at other elections have been generally accurate.

## Exit polls[edit]

Exit polls ask voters just after they leave the booth how they voted. The results are tabulated within minutes and allow news agencies to make a prediction of who will win (or if it is "too close to call".)

## Influence[edit]

By providing information about voting intentions, opinion polls can sometimes influence the behaviour of electors. The various theories about how this happens can be split up into two groups: bandwagon/underdog effects, and strategic ('tactical') voting.

A Bandwagon effect occurs when the poll prompts voters to back the candidate shown to be winning in the poll. The idea that voters are susceptible to such effects is old, stemming at least from 1884; Safire (1993: 43) reported that it was first used in a political cartoon in the magazine Puck in that year. It has also remained persistent in spite of a lack of empirical corroberation until the late 20th century. George Gallup spent much effort in vain trying to discredit this theory in his time by presenting empirical research. A recent meta-study of scientific research on this topic indicates that from the 1980s onward the Bandwagon effect is found more often by researchers (Irwin & van Holsteyn 2000).

The opposite of the bandwagon effect is the Underdog effect. It is often mentioned in the media. This occurs when people vote, out of sympathy, for the party perceived to be 'losing' the elections. There is less empirical evidence for the existence of this effect than there is for the existence of the Bandwagon effect (Irwin & van Holsteyn 2000).

The second category of theories on how polls directly affect voting is called strategic or tactical voting. This theory is based on the idea that voters view the act of voting as a means of selecting a government. Thus they will sometimes not choose the candidate they prefer on ground of ideology or sympathy, but another, less-preferred, candidate from strategic considerations. An example can be found in the general election of 1997. Then Cabinet Minister, Michael Portillo's constituency of Enfield was believed to be a safe seat but opinion polls showed the Labour candidate Stephen Twigg steadily gaining support, which may have prompted undecided voters or supporters of other parties to support Twigg in order to remove Portillo. Another example is the Boomerang effect where the likely supporters of the candidate shown to be winning feel that s/he is "home and dry" and that their vote is not required, thus allowing another candidate to win.

These effects only indicate how opinion polls directly affect political choices of the electorate. Other effect can be found on journalists, politicians, political parties, civil servants etc. in, among other things, the form of media framing and party ideology shifts.

## Political betting markets vs. political opinion polls[edit]

Political betting markets are markets where individuals wager on elections or on various politically related events to occur.

### Arguments on why political betting markets are better than political polls[edit]

* US midterms: why gambling markets often predict elections more accurately than polls, TheConversation.com, 2022
* Political betting: Is it a 'gambling den' or 'better than polls'?, Politico, 2022
* Betting on Elections Can Tell Us a Lot. Why Is It Mostly Illegal?, The New Yorker, 2022
* The Election Betting Markets Fell Short. They're Still the Most Flexible Predictor., Reason.com, 2022
* Prediction markets vs polls – an examination of accuracy for the 2008 and 2012 elections, Journal of Prediction Markets, Jan 8, 2015, DOI: https://doi.org/10.5750/jpm.v8i3.981

The history of political betting markets:

* The Long History of Political Betting Markets: An International Perspective, Oxford Academic
* Historical Presidential Betting Markets, American Economic Association (JOURNAL OF ECONOMIC PERSPECTIVES, VOL. 18, NO. 2, SPRING 2004, (pp. 127-141).

### Track record of the website ElectionBettingOdds.com in terms of accuracy[edit]

The website ElectionBettingOdds.com averages live odds that it pulls from FTX.com, Betfair.com, PredictIt.org, Smarkets.com, and Polymarket.com. The odds are then volume-weighted, meaning that the odds for each market are multiplied by that market's share of quarterly volume.[2]

* ElectionBettingOdds.com FAQ

### How accurate is the website ElectionBettingOdds.com?[edit]

The website ElectionBettingOdds.com has a respectable track record in terms of accuracy as can be seen at: Track Record - Election Betting Odds, 2022

Screen shot from the website ElectionBettingOdds.com.[1] The website is run by Maxim Lott and John Stossel.

### 2019: Judgement and Decision Making Journal: Another approach[edit]

According to the abstract for the 2019 journal article Are markets more accurate than polls? The surprising informational value of “just asking” published in the Judgment and Decision Making journal:

“  |  Psychologists typically measure beliefs and preferences using self-reports, whereas economists are much more likely to infer them from behavior. Prediction markets appear to be a victory for the economic approach, having yielded more accurate probability estimates than opinion polls or experts for a wide variety of events, all without ever asking for self-reported beliefs. We conduct the most direct comparison to date of prediction markets to simple self-reports using a within-subject design. Our participants traded on the likelihood of geopolitical events. Each time they placed a trade, they first had to report their belief that the event would occur on a 0–100 scale. When previously validated aggregation algorithms were applied to self-reported beliefs, they were at least as accurate as prediction-market prices in predicting a wide range of geopolitical events. Furthermore, the combination of approaches was significantly more accurate than prediction-market prices alone, indicating that self-reports contained information that the market did not efficiently aggregate. Combining measurement techniques across behavioral and social sciences may have greater benefits than previously thought."[3] |  ”

|
|

For the full Journal article, please see: Are markets more accurate than polls? The surprising informational value of “just asking” Judgment and Decision Making, Vol. 14, No. 2, March 2019, pp. 135-147

## References[edit]

1. ↑ In unweighted samples, each person has an equal chance. If people have a known but unequal chance, they are weighted to make the chances appear equal.
2. ↑ ElectionBettingOdds.com FAQ
3. ↑ Judgment and Decision Making, Vol. 14, No. 2, March 2019, pp. 135-147

* Asher, Herbert, 1998: Polling and the Public. What Every Citizen Should Know, fourth edition, Washington, D.C.: CQ Press.
* Bourdieu, Pierre. "Public Opinion does not exist" in Sociology in Question, London, Sage (1995)
* Bradburn, Norman M. and Seymour Sudman. Polls and Surveys: Understanding What They Tell Us (1988)
* Cantril, Hadley. Gauging Public Opinion (1944)
* Converse, Jean M. Survey Research in the United States: Roots and Emergence 1890-1960 (1987), the standard history
* Crespi, Irving. Public Opinion, Polls, and Democracy (1989)
* Gallup, George. Public Opinion in a Democracy (1939)
* Glynn, Carroll J., Susan Herbst, Garrett J. O'Keefe, and Robert Y. Shapiro. Public Opinion (1999) textbook
* Irwin, Galen A. and Joop J. M. Van Holsteyn. Bandwagons, Underdogs, the Titanic and the Red Cross: The Influence of Public Opinion Polls on Voters (2000).
* Lavrakas, Paul J. et al eds. Presidential Polls and the News Media (1995)
* Moore, David W. The Superpollsters: How They Measure and Manipulate Public Opinion in America (1995)
* Niemi, Richard G., John Mueller, Tom W. Smith, eds. Trends in Public Opinion: A Compendium of Survey Data (1989)
* Oskamp, Stuart and P. Wesley Schultz; Attitudes and Opinions (2004)
* Robinson, Claude E. Straw Votes (1932).
* Robinson, Matthew Mobocracy: How the Media's Obsession with Polling Twists the News, Alters Elections, and Undermines Democracy (2002)
* Rogers, Lindsay. The Pollsters: Public Opinion, Politics, and Democratic Leadership (1949)
* Traugott, Michael W. The Voter's Guide to Election Polls 3rd ed. (2004)
* James G. Webster, Patricia F. Phalen, Lawrence W. Lichty; Ratings Analysis: The Theory and Practice of Audience Research Lawrence Erlbaum Associates, 2000
* Young, Michael L. Dictionary of Polling: The Language of Contemporary Opinion Research (1992)

## Primary sources[edit]

* Cantril, Hadley and Mildred Strunk, eds. Public Opinion, 1935-1946 (1951), massive compilation of many public opinion polls from US, Britain, Canada, Australia, and elsewhere.
* Gallup, Alec M. ed. The Gallup Poll Cumulative Index: Public Opinion, 1935-1997 (1999) lists 10,000+ questions, but no results
* Gallup, George Horace, ed. The Gallup Poll; Public Opinion, 1935-1971 3 vol (1972) summarizes results of each poll.

## See also[edit]

* Gallup Poll

## External links[edit]

* Angus Reid Global Monitor - The world's largest free-access online public opinion database
* WorldPublicOpinion.org - Online publication covering worldwide opinion on international policy issues
* Public Agenda - Nonpartisan, nonprofit group that tracks public opinion data in the United States
* National Council on Public Polls - An association of polling organizations in the United States devoted to setting high professional standards for surveys
* Vizu Answers - Online Opinion Polling