The 2016 Election could have been called the War of the Polls. Even without subsequent misreading by traditional media and social media outlets, poll results were wildly inaccurate, hotly debated, and/or misunderstood.
Polls are conducted by news media, campaigns, academics, and others involved in the business of running elections. In addition to getting information to the candidates about how to sway voters, they are also conducted to create buzz and make news for the pollsters and their sponsors. Polls are neither more nor less than a snapshot in time of how people say they plan to vote, perhaps with some add-on, multiple choice questions.
The main pitfall is over- focus on single-source, quantitative survey data. In business, we rarely rely on one type of information. Best practices would call for us to look at internal data, secondary analysis, and behavioral data. Further, we would use qualitative research to understand the motivations underneath the behavior, attitudes, and opinions we can measure.
There has been some discussion about other quantitative measures that could have been used to round out the evidence from the poll data. For example, the size of the crowds at campaign events, social media activity, donations size and number, and others could have indicated a different result than many of the polls. These measures of what potential voters’ actions could also help overcome another flaw inherent in polls: reliance on human beings’ ability to predict future behavior. Research has shown time and time again that we are spectacularly poor prognosticators of our own behavior!
I would agree with adding more data sources, but would also recommend more attention to what could be learned from qualitative information. Some of the ways that qualitative information and research could have changed interpretation of the polling data include:
- Strength of emotion: Qualitative information can clue us in on the strength of emotion or passion, anger, and concern underlying the attitudes about the candidates. Especially when quantitative data is describing very similar levels of agreement, the difference in the passion behind supporters’ beliefs can be critical, and can then be measured quantitatively.
- Other question topics: Qualitative research can inform the questions used in a quantitative survey by identifying, in advance, relevant topics for additional questioning not included in the boilerplate poll question set. Perhaps we don’t understand all the issues driving voter preference; there may be other questions we need to ask.
- See the bigger picture: Qualitative, because it is not restricted to a static question set, can help us understand when quantitative results may not be giving us the whole picture. Topics raised independently by qualitative respondents are probably very salient to those respondents – whether they are in the polls or not. Additional questions can then be pursued to clarify respondents’ positions.
- What motivates voters? In the 2016 election, there seems to have been motivational forces at work that were not identified or researched properly. Many supporters of one candidate or the other held very strong positions that were different from some of their candidate’s positions. This sort of dichotomy can only be explored and reconciled qualitatively – and the fact this “irrational” behavior existed should have been a red flag to encourage additional qualitative exploration.
Perhaps it is time for the pendulum to swing away from this over-reliance on quantitative data to include a qualitative understanding of voter motivations. As we are seeing with the Brexit vote, as well as other national elections around the globe, polling’s inability to accurately predict results is causing public distrust with polling and may accelerate this shift.
Quantitative data has its place and will not and should not disappear from the campaign landscape. But it may be time for pollsters to become “researchers” to make more appropriate use of all the tools in the marketing research toolbox, especially looking at qualitative techniques to help voters, as well as candidates, understand what the numbers are telling us!