Good professional practice imposes the obligation upon all public opinion researchers to include, in any report of research results, or to make available when that report is released, certain essential information about how the research was conducted.
–Section III, Code Of Professional Ethics And Practices, AAPOR
Public Opinion Polls
SoonerPoll conducts two types of public opinion polls: polls of likely voters and polls of Oklahoma residents.
SoonerPoll.com uses a combination of landline and cell phone random digit dial samples. This type of sample is known as a dual-frame sample.
In Oklahoma, only about 65% percent of the adult population can be reached via a landline phone, sampling both landlines and cell phones ensures that the data is representative of the entire adult population of the state of Oklahoma. SoonerPoll strives to conduct approximately 60% of the interviews via landline and 40% via cell phone.
Respondents in the landline sample are selected by randomly asking for the youngest adult male or female who is now at home. In the cell sample, the person who answers the phone, provided that person is an adult 18 years of age or older, is asked the survey questions.
Where Do SooonerPoll.com’s Samples Come From?
SoonerPoll.com purchases its samples from Survey Sampling International, the world’s leading provider of sampling, data collection and data analytic solutions for survey research.
SSI developed a variety of random digit methodologies to address landline representation issues. SSI has made more information about their random digit landline methodology available.
SSI ‘s wireless/mobile sample reduces the potential of coverage bias by providing a non-overlapping sampling frame that complements their Random Digit Dial telephone sample. SSI has made more information about their wireless methodology available.
Likey voters: how do we identify them?
When conducting research concerning public policy issues or political races, it is important to us likely voter models. These models are necessary because a large proportion of those eligible to vote do not end up voting in Oklahoma elections.
In the 2010 midterm election, less than 20 percent of Oklahomans age 18 or older cast ballots in the governor’s race. As we have talked about in detail elsewhere on this site, reporting voter preferences on the basis of all adults in Oklahoma, or even all Oklahomans registered to vote, will not give us an accurate estimate of what to expect on election day.
To ensure that participants are in fact likely voters, SoonerPoll selects a random sample of Oklahomans and subjects them to a comprehensive likely voter screen.
SoonerPoll.com’s likely voter screen mimics Gallup Poll’s screen, which asks poll respondents a series of questions about their interest in the coming election, their past voting behavior, and their current intention to vote in the election. Here are all of some the questions used to determine likely voters:
- Do you plan on voting in the General election on November 6TH?
- How much have you thought about the upcoming elections in November?
- Do you happen to know where people in your neighborhood go to vote?’
- Have you ever voted in your precinct or election district?
- How often would you say you vote?
- In the last presidential election did you vote for Barrack Obama or John McCain, or did things come up to keep you from voting?
- If ‘1’ represents someone who will definitely not vote and ’10’ represents someone who definitely will vote, where on this scale would you place yourself?
- Which of the following categories does your age fit into?
According to Gallup, it is important to ask so many questions because poll respondents often overstate their intention to vote in an election. Like Gallup, SoonerPoll.com scores respondents on a scale for their likelihood to vote and respondents with the highest scores are considered likely voters.
Depending on the prominence of the election, SoonerPoll.com may add questions or ask different questions to better determine a respondent’s likelihood to vote in that particular election. SoonerPoll may also decide to lower the threshold of respondents’ likelihood scores for more prominent elections and raise the threshold for more obscure elections.
This is very similar to how Pew Research conducts its likely voter screening as well.
Gallup’s likely voter model has proven to be accurate in their estimates of presidential and congressional midterm elections. SoonerPoll.com has only recently made the switch to this particular method of determining likely voters, but we intend to track our accuracy with this model and report our findings on the “How Well Did We Do?” page.
Has SoonerPoll always used a dual-frame sample?
On July 26, 2012, SoonerPoll.com announced that it would be changing its methodology to the dual-frame sample methodology described above. Prior to this, SoonerPoll used an internal database of registered voters, or a list-based methodology. For this previous methodology, SoonerPoll developed a statistical model that identified likely voters by evaluating past voting history and other factors to create a likely voter sample. Participants were then randomly selected.SoonerPoll.com had great success with this methodology for a number of years, but, as more and more families become cell phone only households, it became apparent that the original methodology would soon be unsustainable.To find out more information about SoonerPoll.com’s original methodology, visit our “Methodology Prior to July 2012″ page.
Every SoonerPoll public opinion poll completed by telephone is conducted within our on-site calling center in Oklahoma City using Computer-aided Telephone Interviewing (CATI) technology.
Our on-site call center allows us to manage every phase of the public opinion polling process and ensures that every Oklahoman we survey is contacted by an Oklahoman that is chosen, trained and supervised by the experts at SoonerPoll.
SoonerPoll’s data collection team includes a field director who is in charge of monitoring the sample, verifying interviews and providing feedback to interviewers. Each of our interviewers receives extensive training and practice interviewing sessions before working on any projects. Before they make that first call, every member of our data collection team is familiarized with the overall study objectives, interviewing techniques and the survey instrument.
Our Training Never Ends
While interviews are being conducted, the field director monitors interviewers and evaluates their performance. A monitoring team uses a standardized evaluation instrument to randomly assess individual interviewing effectiveness in areas including:
- appropriate feedback
- reading verbatim
- proper speech and pronunciation
- interviewing pace
- general rapport with respondents
Is all that really important?
Maintaining the highest standards throughout each interview is vital in order to maximize participation and accuracy. Other pollsters may be comfortable outsourcing the most sensitive phases of the data collection process, but at SoonerPoll, we believe accuracy is too important to leave up to chance.
Weighting the Sample
After all of the interviews are conducted, the combined landline and cell phone samples are weighted based on several key factors. The sample is weighted to ensure that it is representative of the population of Oklahoma in terms of gender, age, education, race, Hispanic origin, region, landline/cell status, population density. The sample is compared to the parameters of the most current U.S. Census data.
Weighting a sample does affect sampling errors and statistical tests of significance. SoonerPoll.com goes to great lengths to be as transparent as possible. A detailed methodology that discloses sampling errors and statistical tests of significance will be made available for every survey conducted by SoonerPoll.com. Among other data, SoonerPoll.com’s methodology reports will include sample sizes and disposition reports.
All polls come with a theoretical calculation of sampling error. Most of us are accustomed to hearing “plus or minus four percent” when we hear a newscaster citing the most recent poll results. Sampling error, however, isn’t the only error in polling.
Errors may result from a myriad of other factors and, unfortunately, most of these are impossible to quantify. Factors that can cause errors include:
- coverage error
- declined interviews
- interviewer bias
- question wording
- question order, and
- an imperfect likely voter screening process
How does SoonerPoll minimize these errors?
It’s simple and it’s something that sets us apart from our competitors. At SoonerPoll, we use our own technology, our own resources and our own employees to maximize the accuracy of every data collection project we conduct.
We don’t outsource any portion of a polling project – we conduct every aspect of every poll in our offices with employees chosen, trained and supervised by us.
In all of our public opinion polls, the professionals at SoonerPoll abide by the Code of Professional Ethics and Practices as prescribed by the American Association of Public Opinion Research (AAPOR).