The Myth of Bias-Free Data
~ 4 minute read
Photo credit: UX Indonesia
This is the first in a series of quick myth-busting posts on data and assessment.
Last week, someone asked me about the diversity in data part of my systems thinking piece. So this week, let’s quickly clarify why more diverse data and methods typically mitigate bias, and why bias-free data is a myth.
First, a definition. Bias is a tendency towards a particular outcome. But most of the time, people use it to describe a tendency towards inaccurate or unfair outcomes.
The Dennis Dilemma: Diversity Helps
Ariel is a team leader and wants everyone’s input on purchasing a ChatGPT team subscription. Unfortunately, Dennis is the only one speaking up in meetings to share his take. The meeting is a biased method of gathering data since requests for input turn into the Dennis show. Ariel’s chart of the data, “How Much the Team Supports Getting ChatGPT,” is biased since it inaccurately describes the data. It should be titled, “How Much Dennis Supports Getting ChatGPT.”
We’re all glad that Dennis feels free to share his thoughts, but there are many other team members who have opinions.
Ariel creates an anonymous survey asking for their team’s input. After a week, they have nearly all team members’ responses. But in a one-on-one, Dennis admits that he didn’t take the survey. He had trouble reading the options and feels the survey wouldn’t have captured his thoughts about AI anyway. A chart of this data might accurately be titled, “How Much Everyone Except Dennis Supports the Project.”
If Ariel uses both the team meeting and the survey to gather responses, they might be able to get everyone’s input.
The Ariel Dilemma: Power Matters
Diversity in data methods is usually good practice. All things being equal, it will get us more representative data.
But all things are never equal.
Ariel has more power in this situation. If the data doesn’t align with what Ariel wants, they can always decide not to release the results. But even if Ariel is open about the findings, bias can still creep in.
Ariel decided on the methods to gather input. They facilitated the meeting structure. They decided on the survey questions.
A third-party focus group without Ariel might have led to more input than Dennis’ statements alone. People love and respect Ariel, and they know how excited they are by AI. The team may not have wanted to share their misgivings with Ariel directly.
The survey questions asked how much more “productive” and “effective” the team’s work would be. Ariel’s excitement led to them forgetting to ask about how “unethical” AI use might be for their team’s work. Many team members would have expressed more hesitance with AI if asked.
In truth, unequal power dynamics will always lead to some bias. And since power dynamics are omnipresent, bias-free data is a myth.
Reduce Bias Where You Can. Be Aware of Where You Cannot.
Photo credit: Patrick Tomasso
Societal and organizational structures are unlikely to change before you need to get your team’s input. Bias-free methods and data do not exist, but bias-reduced and bias-aware ones do.
Diverse methods of data collection are helpful. Power-sharing and participatory assessment methods can also address some bias—though they may introduce other forms.
The AI example above is intentional. Organizations of all kinds are considering the use of AI in their teams. You might be doing the same. The idea that more and more data will somehow create bias-free AI is, very simply, a myth. Do the ones gathering the data, building the infrastructure, and developing the AI have more power than the ones who are not? If so, then not only does bias exist, but as the imbalance of power grows, so will the bias.
If you’re looking for third-party assessments or to build capacity in data fluency, let us know. You can also review our services for more info.
