Our Methodology at SoonerPoll

Good professional practice imposes the obligation upon all public opinion researchers to include, in any report of research results, or to make available when that report is released, certain essential information about how the research was conducted.

Section III, Code Of Professional Ethics And Practices, AAPOR

Public Opinion Polls

The vast majority of the live interview polling we perform is of likely voters in Oklahoma, using a tri-frame of landlines, cell phones, and an online panel.

Where Do SoonerPoll’s Samples Come From?

SoonerPoll.com purchases its cellphone samples from Survey Sampling International, the world’s leading provider of sampling, data collection and data analytic solutions for survey research.

SSI‘s wireless/mobile sample reduces the potential of coverage bias by providing a non-overlapping sampling frame that complements their Random Digit Dial telephone sample. SSI has made more information about their wireless methodology available.

To produce the landline sample, SoonerPoll uses its own internal database of registered voters and, using a variety of different data points, determines the likeliness of each voter voting in the next election, thereby producing a probability sample of likely voters.

Our online panel is utilized through Research Now, which maintains one of the largest political panels nationwide.

How Do We Identify Likely Voters?

When conducting research concerning public policy issues or political races, it is important to us likely voter models. These models are necessary because a large proportion of those eligible to vote do not end up voting in Oklahoma elections.

In the 2010 midterm election, less than 20 percent of Oklahomans age 18 or older cast ballots in the governor’s race. As we have talked about in detail elsewhere on this site, reporting voter preferences on the basis of all adults in Oklahoma, or even all Oklahomans registered to vote, will not give us an accurate estimate of what to expect on election day.

This was affirmed in July 2015, when the Tulsa World reported an in-depth analysis of voting behavior in Oklahoma.

For the cellphone sample and the online panel, SoonerPoll’s likely voter screen mimics Gallup Poll’s screen, which asks poll respondents a series of questions about their interest in the coming election, their past voting behavior, and their current intention to vote in the election.  Here are all of some the questions used to determine likely voters:

• Do you plan on voting in the General election on November 6th?
• How much have you thought about the upcoming elections in November?
• Do you happen to know where people in your neighborhood go to vote?’
• Have you ever voted in your precinct or election district?
• How often would you say you vote?
• In the last presidential election did you vote for Barack Obama or John McCain, or did things come up to keep you from voting?
• If ‘1’ represents someone who will definitely not vote and ’10’ represents someone who definitely will vote, where on this scale would you place yourself?
• Which of the following categories does your age fit into?

According to Gallup, it is important to ask so many questions because poll respondents often overstate their intention to vote in an election. Like Gallup, SoonerPoll.com scores respondents on a scale for their likelihood to vote and respondents with the highest scores are considered likely voters.

Depending on the prominence of the election, SoonerPoll.com may add questions or ask different questions to better determine a respondent’s likelihood to vote in that particular election. SoonerPoll may also decide to lower the threshold of respondents’ likelihood scores for more prominent elections and raise the threshold for more obscure elections.

This is very similar to how Pew Research conducts its likely voter screening as well.

Gallup’s likely voter model has proven to be accurate in their estimates of presidential and congressional midterm elections. SoonerPoll.com has only recently made the switch to this particular method of determining likely voters, but we intend to track our accuracy with this model and report our findings on the Track Record page.

Has SoonerPoll always used a dual-frame or tri-frame sample?

On July 26, 2012, SoonerPoll.com announced that it would be changing its methodology to the dual-frame sample methodology described above. Prior to this, SoonerPoll used an landline sample only, or a list-based methodology. For this previous methodology, SoonerPoll developed a statistical model that identified likely voters by evaluating past voting history and other factors to create a likely voter sample. Participants were then randomly selected. SoonerPoll.com had great success with this methodology for a number of years, but, as more and more families become cell phone only households, it became apparent that the original methodology would soon be unsustainable.

Starting in 2016, SoonerPoll began adding an online panel to its sample in a continued effort to lower any potential coverage bias.

Data Collection

Every SoonerPoll public opinion poll completed by telephone is conducted within our on-site calling center in Oklahoma City using Computer-aided Telephone Interviewing (CATI) technology.

Our on-site call center allows us to manage every phase of the public opinion polling process and ensures that every Oklahoman we survey is contacted by an Oklahoman that is chosen, trained and supervised by the experts at SoonerPoll.

SoonerPoll’s data collection team includes a field director who is in charge of monitoring the sample, verifying interviews and providing feedback to interviewers. Each of our interviewers receives extensive training and practice interviewing sessions before working on any projects. Before they make that first call, every member of our data collection team is familiarized with the overall study objectives, interviewing techniques and the survey instrument.

Because we randomly select voters from our likely voter database, we call and interview that particular voter so no household selection process is used or needed.  Cell phones are overwhelming individual driven as well, therefore no household selection is used or needed here as well.

Spanish-speaking interviewers are on staff and surveys are conducted in Spanish should a potential respondent needs or requests it.

Our Training Never Ends

While interviews are being conducted, the field director monitors interviewers and evaluates their performance. A monitoring team uses a standardized evaluation instrument to randomly assess individual interviewing effectiveness in areas including:

• appropriate feedback
• reading verbatim
• proper speech and pronunciation
• interviewing pace
• general rapport with respondents

Is all that really important?

Maintaining the highest standards throughout each interview is vital in order to maximize participation and accuracy. Other pollsters may be comfortable outsourcing the most sensitive phases of the data collection process, but at SoonerPoll, we believe accuracy is too important to leave up to chance.

Weighting the Sample

After all of the interviews are conducted, the combined landline and cell phone samples are weighted based on several key factors such as gender, age, education, race, congressional district and landline/cell status. The weights or statas are determined  using our internal database of likely voters.

This critical step would not be possible with our internal database of Oklahoma voters.  Other pollsters, without this type of internal resource, would be unable to accurately evaluate the demographical profile of Oklahoma likely voters.  This is how SoonerPoll is able to ensure that the sample reflects as closely as possible to the population of likely voters in the state.

The measurement of sampling error for any particular poll is typically not adjusted for any design effects due to sample weights and/or clustering unless otherwise noted in the release of poll results.

Polling Errors

All polls come with a theoretical calculation of sampling error. Most of us are accustomed to hearing “plus or minus four percent” when we hear a newscaster citing the most recent poll results. Sampling error, however, isn’t the only error in polling.

Errors may result from a myriad of other factors and, unfortunately, most of these are impossible to quantify. Factors that can cause errors include:

• Coverage error
• Declined interviews
• Interviewer bias
• Question wording
• Question order, and
• An imperfect likely voter screening process

How does SoonerPoll minimize these errors?

It’s simple and it’s something that sets us apart from our competitors. At SoonerPoll, we use our own technology, our own resources and our own employees to maximize the accuracy of every data collection project we conduct.

We don’t outsource any portion of a polling project – we conduct every aspect of every poll in our offices with employees chosen, trained and supervised by us.

In all of our public opinion polls, the professionals at SoonerPoll abide by the Code of Professional Ethics and Practices as prescribed by the American Association of Public Opinion Research (AAPOR).