Understanding Our Call Dispositions and Rate Calculations Report

In an effort to provide the most transparency possible with our polling methods, SoonerPoll.com produces a Call Dispositions and Rate Calculations Report with each poll fielded and publicly released.  As of this writing, no other pollster in Oklahoma, partisan or non-partisan, publicly releases this much information about their polling methods.

This sample report below provides insight into how this particular poll was fielded, using call dispositions and rate calculations as defined by the American Association for Public Opinion Research (AAPOR) in the Standard Definitions:  Final Dispositions of Case Codes and Outcome Rates for Surveys.

This webpage borrows liberally from AAPOR’s Standard Definitions Report. The citation is provided below:

The American Association for Public Opinion Research. 2015. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 8th edition. AAPOR.

Call Dispositions

The top half of the report is the aggregate of the final disposition of each call made, according to AAPOR's standardized dispositions. A detailed report of the final disposition status of all sampled cases in a survey is vital for documenting a survey’s performance and determining various outcome rates. Such a record is as important as detailed business ledgers are to a bank or business. In recognition of this premise, the reports on the final disposition of cases are often referred to as "accounting tables." They are as essential to a well-documented survey as they are to a well-organized business.

Average # of Attempts

In the middle of this report is the "Average # attempts." SoonerPoll also tracks the number of attempts made to each non-interview, of both known and unknown eligibility (category 2 and category 3). As presented here, for example, SoonerPoll made an average of 4.5 attempts to the non-interviews on this particular poll.

Rate Calculations

Response Rates: The number of complete interviews with reporting units divided by the number of eligible reporting units in the sample. The report provides FOUR definitions of response rates, ranging from the definition that yields the lowest rate to the definition that yields the highest rate, depending on how partial interviews are considered and how cases of unknown eligibility are handled.
‍
Cooperation Rates: The proportion of all cases interviewed of all eligible units ever contacted. The report provides FOUR definitions of cooperation rates, ranging from a minimum or lowest rate, to a maximum or highest rate.

Refusal Rates: The proportion of all cases in which a housing unit or the respondent refuses to be interviewed, or breaks-off an interview, of all potentially eligible cases. The report provides THREE definitions of refusal rates, which differ in the way they treat dispositions of cases of unknown eligibility.

Contact Rates: The proportion of all cases in which some responsible housing unit member was reached. The report provides THREE definitions of contact rates.

Why is this important?

Often it is assumed — correctly or incorrectly — that the lower the response rate, the more question there is about the validity of the sample. Although response rate information alone is not sufficient for determining how much non-response error exists in a survey, or even whether it exists, calculating the rates is a critical first step to understanding the presence of this component of potential survey error. By knowing the disposition of every element drawn in a survey sample, researchers can assess whether their sample might contain non-response error and the potential reasons for that error.