Back when dinosaurs roamed the earth and landlines were a common fixture in every household, phone surveys comprised the bulk of survey data collection. But with the light-speed evolution of the internet and the standardization of the smart phone, consumers have begun to engage more with products via their personal devices. Caller ID and increasing issues with spam calls have led individuals to be skeptical of answering unknown numbers, and suspicious of giving out personal information. Websites are more easily verifiable as legitimate, and web-based surveys allow participants to give thoughtful answers free from the anxiety and suspicion that often comes with interacting with a live stranger.
Several studies have shown that data quality from online methods is better than from phone surveys. A tracking study done by Pew Research reports that telephone survey cooperation rates are currently around 5%. Phone survey response rates decline again | Pew Research Center
To quote from that report: “Among the factors depressing participation in telephone polling may be the recent surge in automated telemarketing calls, particularly to cellphones. The volume of robocalls has skyrocketed in recent years, reaching an estimated 3.4 billion per month. Since public opinion polls typically appear as an unknown or unfamiliar number, they are easily mistaken for telemarketing appeals. In addition, new technologies sometimes erroneously flag survey calls – even those conducted for the Centers for Disease Control and Prevention – as ‘spam.’ Numerous cellphone operating systems, cellular carriers and third-party apps block incoming phone numbers or warn users that incoming numbers are from potential scammers, fraudsters or spammers.”
While telephone surveys may have their place in specific situations, researchers must bargain with the reality that the modern phone survey-taker is a very specific demographic. That raises the fundamental question of whether those who are willing to do a phone survey are at all representatives of the underlying population. Case in point is the wild inaccuracy of political polls that rely on phone surveying. After the 2016 election, where the major phone-based polling organizations all predicted a victory by Hillary Clinton, pollsters have largely shifted to digital data collection (or mixed mode, where they only use phones to reach rural and elderly demographics).
Here is a study of the trends in survey data collection from an academic journal: Response rates of online surveys in published research: A meta-analysis – ScienceDirect In 2017, for the first time, online surveys constituted the majority of all quantitative survey modes implemented worldwide (ESOMAR, 2018; Daikeler et al., 2020).
The best way to ensure comparability of the data is to look at the composition of the sample. If the gender, age, race, and geographic distribution is close to the same between the current and prior waves, then the data should be comparable. If one sample is skewed on any of these characteristics, sample weighting can be applied to match the samples more closely. At that point, the data collection methodology difference should not be as much of a factor.
While there may be some anecdotal situations where data quality remains consistent across both methodologies, phone surveys are increasingly a gamble. Response:AI is committed to helping you hone your goals and maximize your budget for the most reliable data at the most affordable price. So, whether you call or click today, we are standing by to assist all your research needs.