Eliminating Bias in User Research
“Don’t find customers for your products, find products for your customers.” – Seth Godin
Some of the phrases we hear with some regularity from our clients are, “we want to expand and diversify our audiences,” and “we want to deliver a personalized experience.” And that’s great; you should make every effort to expand your supporter base and cultivate them in meaningful ways that make an impact.
When we tell our clients that the best way to achieve this goal is to start with user research, it’s often met with hesitation, and we get it. We’re selling a strategy that is meant to challenge assumptions of stakeholders and team members who think they already know their audiences. Or maybe you think you’ve already done enough research.
But what if that research was biased or had a narrow audience?
What if you asked questions in a way that yielded the answers you wanted to hear?
What if your respondents weren’t diverse and were only recruited based on your established circles of influence?
User research, when done well, provides a deeper understanding of users’ needs. It surfaces new solutions and opportunities, and can often save you from investing money in the wrong solutions or features for your web project.
“If you have money to do a thing, you have money to make sure it’s the right thing.” – Erika Hall, Let’s All Do Design Research Right!
Incorporating user research into your digital projects ensures that you will invest in the best decisions– all while centering the needs of your audience throughout the process.
There are countless combinations of methodologies for how you can conduct user research, and our team is ready to help you get the most out of it. Even if you’re just asking three people three questions, it’s better than not asking anyone at all. It’s all in an effort to help you better understand who your audiences (supporters, advocates, organizers) are. This research also informs your personas, which lay the foundation for your marketing, communications, and content strategy.
Personas represent your research
Personas are meant to create realistic representations of your users to enable stakeholders, designers, and developers – everyone in the project – to empathize with the actual humans that will be engaging with your digital products.
You may be familiar with personas, and they’ve probably looked a little like this:
While this is a nicely designed template, there’s a lot here that isn’t helpful in building your digital product. And there are other things that (depending on who you’re presenting to) could lead to more unproductive assumptions. Now I’ll be real, I’ve produced persona templates that are not far off from this – but it’s problematic. Let’s talk about why.
Don’t focus on demographics
Personas focused on demographics are not useful for design. I don’t need to know Chad’s salary or what kind of car he likes (unless I’m trying to sell him a car, which I’m not). In fact, the name Chad already makes me feel a certain *way*. Now, I’ve chosen a particularly bad example above to prove a point, and while you might roll your eyes at the details in the persona template, these are the sorts of things that will distract stakeholders from the important, useful information in your research. Here are some considerations to take into account when creating your personas:
Names
- Keep them gender-neutral
- Avoid fictional, stereotypical, or famous references
- Strike a balance between generic and inspirational
Our recommendation is that you name your persona based on their dominant behavior or need. For example, instead of “Kelly, 5th grade teacher in [rural American town]” try “Educator in low digital infrastructure environment.”
Photos
Kim Goodwin said it best, “Photos are badly done at least half the time…but the right set of photos can help the (often very male and very white) team remember that the whole user base doesn’t look like what they expect.”
We’ve created persona deliverables that include photos when the goal of our research was to highlight and address the needs of a marginalized audience. But we’ve also created deliverables with zero photos, and it did not negatively impact the reception of the information. Both can be effective, it just depends on the goals of your research and the needs of your audience.
Accessibility
If you’ve been following ThinkShout for a while, you know our commitment to web accessibility. In this instance I’m referring to the inclusion and representation of individuals with a range of abilities in addition to those with limited access to power and privilege and how that impacts their experience.
Does your audience only access the internet on a mobile phone? You should call that out, especially if in prototyping and testing your team defaults to using laptops. What about internet access more broadly? Do you need to address ways for people to print materials at a library (or other location) so they can then consume it at home where they don’t have wifi? How does your content look then? Accessibility means equal access to all.
Age, Gender, Race and Ethnicity
If I have to read one more judgemental thinkpiece about millennials while enjoying my avocado toast, I’m gonna lose my mind. Some of us millennials are rounding 40 pretty soon. Don’t describe age using a number or generational label; instead, state how long the typical member of your audience has been doing X. For example, “So and so has been a software engineer for 5 years.” This person could be new to the industry but be any age, therefore removing any unfair assumptions we might make about them based on how old they are. The exception to this rule is if you are specifically seeking out the needs and motivations of a specific age group for your study.
When it comes to gender, race and ethnicity, take a minute to examine why you might want to highlight it. Ask yourself if anything you’ve included in the description will reinforce stereotypes or unhelpful assumptions about this population. Again, Kim Goodwin has some helpful guidelines for why you may want to highlight these attributes:
- If the reasons for including it have roots in culture, affecting thinking in the context of your study. Not if it’s as a shallow, head-nod to diversity.
- The team needs reminding that everyone doesn’t come from the same guiding principles and reasoning as themselves.
- There’s a guideline in the protocol at your organization that requires it.
As it applies to highlighting race and ethnicity, if you do call it out, it’s important to be abundantly clear that your insights are based on your research, and not representative of ALL people in said population. It’s especially important to note this if your research was regionally based and acknowledge that just because someone is of a certain race or ethnicity does not make them the representative for all folks of that race or ethnicity. None of us are a monolith.
Narrow down the context: User Behavior Segments
Instead of demographics, focus on the behaviors of your users, because behaviors will drive the end product.
What do I mean by that?
Let’s use an example of me traveling: my needs when traveling for business are probably 80% different than when I’m traveling for vacation. So if you’re creating a persona for a traveler and trying to figure out which one I fall into, that’s going to vary depending on the purpose of my trip. I am most certainly not the same person when I’m flying to DC for business as I am flying to Playa del Carmen.
Or let’s make this more applicable and say we’re making personas for educators. One year a classroom may have six ESL (English as a Second Language) students, requiring a teacher to shift much of how they prep and deliver their lessons. The next year, they may have zero ESL students, enabling them to scale back their prep time slightly.
That’s why it’s important to narrow down the context to a certain situation and focus the scope on people’s thinking as they move toward a particular intent or purpose.
When you start designing for behaviors instead of demographics, you’re creating better products that are focused on those user intentions and motivations, rather than unhelpful assumptions or stereotypes. Here’s an example from a recent project we executed on behalf of a think tank:
User Behavior Segment 1: Priority Audiences
User Behavior Segment 2
What’s important to note here is a user can shift from one segment to another, depending on the context of their situation and their resulting needs.
Where to start?
So you want to get a fresh take on your organization’s audiences and take a stab at building personas without bias – great! This empathy mapping exercise is a great place to start with your team:
By David Gray, XPLANE
From here you can start to map out what users are going through prior to and after they discover your organization. If you’re short on time/staff support, you can very well leave your research at this point and call it good. Or, based on these outcomes you can start to form questions that you may want to ask in a survey or in interviews to confirm/deny your assumptions.
The biggest take away from all of this is: never assume you know everything about your audience. People are complex. It’s in your organization’s best interest to understand your audience so that you can create digital products and experiences that they want. Not so you can build what you think they want and then try to find an audience to fit.
A bonus? If your research is presented in a way to eliminate bias, you’ll have an easier time getting the buy-in you need, because you’ll be focused on the behaviors that matter, instead of the fact that Chad’s hobbies are yachting.
If all this feels like a lot to tackle, let us know and we can help you figure out an approach that is manageable, or we can partner on it. We’d also love to know what you think – let us know what you have questions about, or if you have other resources for eliminating bias in research that we should point to here!