The Myth of Data-Driven Objectivity
~ 6 minute read
Photo credit: Alexander Sinn
This is the second in a series of quick myth-busting posts on data and assessment. See also the first on the myth of bias-free data.
Relatively early in my graduate school experience, I began to hear people use the term “Data-Driven Decision-Making,” or DDDM for short. I chalked it up to one of those many terms people use, like “best practice” and “evidence-based,” that are meant to give an air of legitimacy to their claims. Saying your decision was “data-driven,” was implicitly saying this was obviously the right decision.
Who could argue with “the data?”
This week’s data myth is less explicit, and more implicit. There is a tendency to think of decisions based on “the data” as inherently effective, fair, and accurate. In a word, better.
But decisions driven by data are not inherently better. They inherit the biases of the data that drive them and the ones making decisions.
So, how do leaders, teams, and organizations avoid falling prey to implicit biases? The answer starts by making the implicit, explicit.
A Data-Driven Case Study
Irmela is the executive director of a small nonprofit that has contracted a firm to survey a sample of active community members. She is feeling pressure to release a public statement on recent legislative action, though she herself feels the issue is outside of the nonprofit’s lane and worries that a statement could incur unwanted attention.
The results of the survey validate her concerns.
Over 70% of those surveyed felt that a public statement “. . .could attract blowback from opposition groups.”
The firm Irmela is working with also examined social media data, analyzing the public response to regional nonprofits that responded to similar issues over the past 5 years. Sure enough, nonprofits that spoke out on these issues had 2.3 times more vitriolic comments on their social media pages than those that did not speak out.
With this, Irmela is confident in her data-driven decision not to make a public statement. She is confident that the decision is not grounded in her own biases but in community input and historical data. She is confident that this is, objectively, the right decision.
Data-Driven, Values-Hidden
Diversity of data helps to reduce bias. The social media analysis helped address potential biases that existed among community members. The community survey helped mitigate potential negative bias among those simply following social media. However, diversity in data does not eliminate bias, and it does not make decisions value-free.
Values are at the root of all human decisions. It’s just that some decisions clearly state what those values are, while others shroud them behind the data.
Irmela’s confidence that her decision is objectively right because it aligns with the data she shared is misplaced. By casting her decision in this light, she risks alienating the membership, who may be able to see the bias that Irmela cannot.
Perhaps Irmela secretly agrees with the legislative action. Or perhaps she feels attention will distract from the nonprofit’s central work. Or maybe she is thinking of the political conflicts that could ensue with a large funder.
All of this may have driven both the survey and social media review, which focused heavily on potential negative optics, rather than the positive response, to a statement.
If Irmela had made her actual values explicit at the start, she might have been able to avoid some of the bias in the data collection efforts.
Values-Driven, Data-Informed
Trying to get accurate data with diverse methods without being aware of your biases and values is like wearing a blindfold, being spun around, and flinging dart after dart at a bull’s eye.
You’re likelier to get a hit than if you only threw a single dart, but your chances are slim, and some people could get hurt.
When you and others are values-aware, it becomes easier to see when those values are biasing your decisions and to pivot. Irmela could have asked the firm to reword the survey questions. A member might have noticed issues in the social media analysis and asked that Irmela look for supportive comments.
Being open about one’s values and biases invites short-term vulnerability, in exchange for longer-term trust and adaptability.
Take a look at the graphic below, which positions decision-making strategies on a spectrum from data-driven, values-hidden to values-driven, data-informed.
Moving from data-driven, values-hidden to values-driven, data-informed decision-making
Data-driven, values-hidden strategies make claims of objectivity or false claims about the values behind their actions. It may lead to short-term support, but over the long term, it builds resistance and mistrust among team members and a sense of decisional rigidity.
Values-driven, data-informed strategies clarify the values that explain decisions while providing data that supports and sometimes complicates those decisions. Such strategies may lead to more receptivity and trust among team members and more adaptability in decisions in the future.
Questions to Guide Values-Driven Decisions
I’ve heard the term “data-driven” less than I used to, but I continue to see folks using the language of data objectivity.
For those who want to be more explicitly values-driven in their decisions, ask yourself these three questions to guide your path ahead:
What hopes and fears do I have about the decisions ahead?
How much do my concerns affect the way I’m seeking more information?
Would sharing my answers to the above change how others view my decision?
Truly values-explicit work leaves leaders vulnerable. This is a challenging step for anyone to take. But the payoff is a decision that is more clearly grounded in values that matter and leaves leaders better prepared for change.
If you’re looking for support in your values-driven, data-informed journey or to build capacity in data fluency, let us know. You can also review our services page for more info.
