New Albany resident Philip Derrow is a regular Columbus Dispatch contributor.
The mid-term election season is roaring to life and with it come innumerable opinion polls telling us what – and who – our fellow citizens like and don’t like. Or so the pollsters and those who report the results claim. But it’s fair to wonder whether the entire enterprise is a true measure of public sentiment.
A recent poll conducted by Ohio Northern University illustrates the point. The headline: “Ohioans prioritize political civility despite future uncertainty.” A friend sent me the story because he knows of my family’s longstanding support of civil discourse initiatives.
I eagerly read the article and was heartened by the purported views of my fellow Buckeyes on the importance of political civility and their generally positive personal experiences sharing their views with family and friends. Then I clicked on the actual poll results and my opinion changed.
The 1,600 adults polled were randomly assigned to one of four groups that were asked different phrasings of the same questions about Ohio’s budget priorities. Of the 29 questions reported, the first 20 were those four variations. That means each respondent answered a set of five of those questions.
This can be a useful methodology to tease out whether the way a question is asked substantively alters the responses. So far so good.
Then I read the questions. Regular readers will recall that my most recent column highlighted the importance of recognizing policy tradeoffs. Budgetary choices are a perfect example of that reality.
The cake you can’t eat
The survey questions completely ignored it.
Instead, they start with, “Imagine that the state of Ohio needs to change its budget.” Then, depending on the group, the questions were about whether we should spend more or less on social welfare, education, health care, law enforcement and infrastructure. Not more or less relative to current tax revenue, that taxpayers would pay more, or that other categories of spending would be cut.
Predictably, across all categories, people wanted more spending. Cue the campaign ads for candidates promising more government support for – everything!
Because the survey says that’s what people want. Conversely, candidates who promise spending cuts would be bashed for disregarding voter preferences.
The proverb that “You can’t have your cake and eat it too” has been around for nearly 500 years. Asking what people want without asking what they’d sacrifice for it is a surefire way to tee up impossible expectations.
The uncounted margins
After that unhelpful exercise, the survey purports to gauge political civility and pessimism/optimism about the future. I’d like to believe Americans are comfortable sharing their political beliefs without being ostracized by family or friends because of them.
Alas, that doesn’t seem likely since a recent study shows that 63% privately acknowledge they self-silence their beliefs. And therein lies one of the many great challenges of divining public opinion, particularly in this internet, cell phone and social media era.
While poll results often imply scientific-sounding certainty by reporting a margin of error, that figure only captures sampling error, not other important sources of error such as methodology problems, question wording effects (as with the ONU poll), coverage problems or non-response bias.
Pollsters have always had to manage a long list of response biases, where respondents purposely or subconsciously manipulate their answers in ways that render the results suspect. Spam blocking software for cell phones and email make it harder for firms to connect with representative demographic samples.
Larger polling firms employ teams of social science experts and statisticians to design surveys and outreach efforts and analyze the results. AI might help even smaller firms break through the noise. But the fundamental challenges of a fractured, distrustful and technologically aloof population will be difficult to overcome.
This all plays out at a time when Ohioans are weighing multiple ballot initiatives, which makes getting an accurate read of public sentiment more important than ever.
Whether we’re considering government budget priorities or other pressing questions, we ought to think about polling the same as virtually everything else these days: There’s always more to the story.
Worse yet, we’re told constantly what voters want but rarely how those opinions were elicited or manipulated. Reading the methodology before believing or sharing the headline is a small act, but it’s necessary for honest civic engagement.
New Albany resident Philip Derrow is a retired business owner. He was a two-term member of the New Albany-Plain Local Board of Education. He is a regular Columbus Dispatch contributor. Reach him at philderrowdispatch@gmail.com.
This article originally appeared on The Columbus Dispatch: Pollsters want their cake and to eat it too. You can’t trust them | Opinion
Reporting by Philip Derrow, Columnist / The Columbus Dispatch
USA TODAY Network via Reuters Connect
