Skip to content

Tag: participants

Posted
September 20, 2021
Category
Advice, Methods
Tags
, , , , ,
Comments
0 Responses

Even More Difficult Usability Testing Participants

Four years ago, in 2017, I published an article in UXmatters giving advice about how to handle ten types of difficult usability testing participants, Wrangling Difficult Usability Testing Participants. The ten types of difficult participants were:

  • Bad Fits to the User Profile
  • Untalkative Participants
  • Overly Talkative Participants
  • Participants Who Ramble Off Topic
  • Inarticulate Participants
  • Participants Who Struggle to Think Aloud
  • Participants Who Have No Opinions
  • Uncritical Participants
  • Participants Who Blame Themselves
  • Uncooperative Participants

Four years later, I felt I had encountered enough new types of participants to write a part two, Wrangling Difficult Usability Testing Participants, Part 2. These include:

  • Happy Clickers
  • Talkers, Not Doers
  • Givers of Facts, Not Opinions
  • Representatives of the Business
  • Participants Who Take Prototypes Too Literally
  • Professional Research Participants
  • Uncomfortable, Nervous Participants
  • Participants Who Are Too Relaxed
  • Harassers

Of course, most participants are just regular people who are trying to do their best in the unusual situation of participating in a usability test. It’s up to you as the researcher to help them understand what you need them to do.

Image by Rinaldo Wurglitsch under Creative Commons License

Posted
December 2, 2019
Category
Advice
Tags
, , , ,
Comments
0 Responses

Should You Ask Your Clients to Help Recruit Participants?

Danger keep out sign

For some projects, your clients can be the best source of user research participants. When you’re looking for their employees, their members, or their customers, your clients are the best source for lists of potential participants. Also they often have a relationship with these potential participants. They may know them personally, or at the very least they are associated with a company with which the potential participants have a relationship. When people get a request to participate in a user research study, they are more likely to pay attention to it, and seriously consider it, if it comes from someone they know or at least if it comes from a person in a company they do business with.

However, there are some perils of asking your clients recruit participants, including:

  • They may not have the time or organizational skills it takes to recruit and schedule participants.
  • They probably won’t describe the research correctly.
  • They probably won’t know the right types of people to recruit.
  • They may not schedule the sessions logically and effectively.
  • They may give participants the wrong ideas about what they’ll be participating in.

 

In my latest UXmatters article, I describe these perils and provide tips to avoid them: The Perils of Client Recruiting.

Image credit: George Hodan

Posted
January 9, 2017
Category
Advice, Methods, User Experiences
Tags
, , , ,
Comments
0 Responses

Questions People Have a Hard Time Answering

Question marks

Over the years, I’ve made my share of mistakes and learned about the types of questions and topics that participants have a hard time answering accurately in user research. Most people do try to answer your questions, but they may not be able to easily and accurately answer these types of questions:

  • Remembering details about the past
  • Predicting what they might do in the future
  • Accurately answering a hypothetical question
  • Discussing the details of their tasks out of context
  • Telling you what they really need
  • Imagining how something might work
  • Envisioning an improved design
  • Distinguishing between minuscule design differences
  • Explaining the reasons for their behavior

I discuss these types of difficult questions, and better ways to get that information from participants, in my latest article on UXmatters:
Avoiding Hard-to-Answer Questions in User Interviews.

Image credit: Véronique Debord-Lazaro on Flickr

Posted
March 10, 2015
Category
Advice, Methods
Tags
, ,
Comments
1 Response

Suspicious Minds

office scene

In previous research projects, there have been several times when participants were suspicious of our motives. This tends to happen when you’re doing research with a group of employees, trying to understand their work processes. These are the times that you’re trying to observe what they do in their jobs more than studying an existing system.

When there’s a direct connection to an application that they use, people tend to feel less suspicious. They can see that you’re trying to understand how well the application works and where it can be improved. This especially happens when it’s in a company that’s had poor previous experiences with reorganizations, layoffs, and offshoring. People tend to see us as another group of consultants coming in to study how they do their work to see what can be improved or who can be eliminated.

So how do you reassure people in these difficult situations about your true purpose? I wrote a recent article about this at UXmatters – Winning Over Wary Participants. Check it out, and if you have additional tips to make people feel more comfortable in these situations, feel free to leave a comment.

 

Image: Jake Sutton

Posted
June 10, 2014
Category
Methods, User Experiences
Tags
,
Comments
0 Responses

What’s the Formula for Success in Recruiting User Research Participants?

Formula

If you believe that designing an effective user experience requires involving users throughout the design process, then you must admit that finding and recruiting participants for user research and usability testing is the most crucial step. After all, if you can’t get participants, you can’t do research or usability testing. A recent, difficult recruiting effort left me ruminating about what it is that makes recruiting either easy or difficult. Is there something like an equation that can account for the ease or difficulty of recruiting participants? Yes, there is: RE = ((K + S) x RM) x ((C+A) x PM) Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation) Okay, I just kind of made that up, but I think it’s a good way to visualize the various factors that influence how easy or difficult it will be to recruit participants. Let’s look at the components of the formula in more detail.

The Recruiter

The first part of the equation involves attributes of the recruiter: Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation) By recruiter, I mean whoever is doing the recruiting. It could be you, a recruiting company, your client, a salesperson, or someone else. Ideally, the researcher would also recruit the participants, because that person knows the most about who to recruit. But sometimes you have to rely on other people to do the recruiting, usually because they have a better connection to participants.

Knowledge

Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation) The first factor is the recruiter’s knowledge of the research and the type of people to recruit.

  • Does the recruiter know the characteristics of the people that are desired? Can the recruiter identify people who match those characteristics?
  • How well can the recruiter describe the research to the participants?

Without this knowledge, recruiters may select the wrong types of people or give them the wrong impression about the research. If you’re not doing the recruiting, it’s important to give the recruiter a clear definition of the types of people you’re looking for and a simple description of the research that they can use when contacting participants.

Skills

Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation)  Effective recruiting also depends on the recruiter’s interpersonal, communication, and organizational skills.

  • Is the recruiter personable and persuasive enough to convince people to participate?
  • Can the recruiter clearly and concisely communicate with the participants?
  • Is the researcher organized and coordinated enough to contact, schedule, and keep track of many participants?

Ensure that the person doing the recruiting has these skills. If not, have them delegate this task to someone who does. Administrative assistants often excel at recruiting because they are already well versed in communicating and coordinating schedules.

Recruiter Motivation

Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation) When finding participants becomes difficult, recruiter motivation is the most important criteria.

  • How much does the recruiter care about finding participants?
  • Will there be positive or negative consequences for the recruiter if participants are found or not found?

If you’re doing the research and the recruiting, you’re usually highly motivated to find participants, but when you have to rely on someone else, try to make sure that person has a stake in the recruiting. The negative consequences of not finding participants and derailing the project are highly motivating. Once a project starts and dates are set, there’s a ticking clock counting down the days to the research sessions. Delays in finding participants will delay the project.

With deadlines looming and money on the line, there’s a lot of pressure for whoever’s responsible to find participants. You can bet that most of the time, it gets done. Professional recruiting companies have a financial motivation to find participants, since they get paid a certain amount for each participant they recruit. However, if the recruiting turns out to be more difficult than they expected, they will begin to lose money as they spend more time finding each participant. When that happens, they’ll usually try to get you to loosen up the recruiting requirements to make it easier to find people, or they may say that they’ve exhausted their efforts and give up.

The Participants

The second part of the equation involves the recruiter’s relationship to the participants: Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation)

Connection

Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation) When people have a connection with the recruiter, they’re more likely to agree to participate. The strength of a connection can take various forms:

  • The participant may know the recruiter personally.
  • The participant may share a common affiliation with the recruiter. For example, they may be fellow employees or members of the same organization.
  • The participant may have an affiliation with an organization that the recruiter represents. For example, they may be a customer of the company the recruiter represents.

An email or phone call to participate in a study, often raises the questions about whether it’s legitimate, whether it’s worth considering, and whether it’s worth the time to participate. The closer the connection the person has with the recruiter, the more likely they will take the time to consider the request. Unless you already have this relationship, you might have someone else, who the potential participants know, do the recruiting. They’re more likely to listen to, consider, and respond to someone they know rather than a complete stranger.

Access

Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation) When you have access to lists of potential participants (such as employees, organization members, or customers) it’s much easier to recruit participants, especially when those lists contain characteristics that you can use to filter and narrow down those lists to the right types of people. It’s very difficult when you and your client have no access to the types of people you need. That’s when it makes sense to use a recruiting company.

Participant Motivation

Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation) Once they’ve been contacted and have considered the pitch, motivation determines whether a person will volunteer to participate. We can look at participant motivation as its own sub-equation: PM = A – D Participant Motivation = Advantages of participating – Disadvantages of participating To make it worth participating, the advantages have to be greater than the disadvantages. A > D = Participate A < D = Decline The advantages of participating can include things like:

  • Incentives – A $150 check, $50 Starbucks gift card, free software, etc.
  • Personal benefit from the improvements that come out of research – They’re going to improve this horrible software that I have to use every day at work!
  • Feelings of altruism
  • Feeling good about having your ideas heard and being able to improve something
  • Novelty of doing something different and interesting
  • Getting praise for participating and avoiding getting in trouble for not helping out

The disadvantages of participating can include things like:

  • The difficulty and effort it takes to participate
  • The time it takes to participate
  • Discomfort, anxiety, and fear of the unknown

Add it Up

Actually, something this complex can’t be distilled into an easy formula, but I think this equation provides a good visualization of the important factors. The quality and motivation of the recruiter, combined with the ease of finding participants, and the way people perceive the benefits they’ll receive versus the hassles they’ll have to go through, determines how easy or difficult it will be to recruit participants.

Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation)

 

Mathematics image courtesy of Tom Brown

Posted
May 31, 2012
Category
Methods
Tags
, ,
Comments
0 Responses

Tattle Tale Participants

Have you ever come across a tattle-tale participant? See if this sounds familiar.

You conduct a user research session (interview, contextual inquiry, focus group, etc.), asking simple questions to get the participants to speak about the subject at a basic level of understanding. Although you may know some of the answers to the questions you’re asking, you ask the questions anyway to hear the answers from the participants’ perspective. Misunderstanding your intent, the participant is alarmed that you don’t know what you’re doing and tattles on you to the client, “It was clear from his questions that he didn’t know anything about our process/system/technology.”

For example, a few years ago I was doing research on a company’s use of SAP. We had a group of interns show us the HR tasks that they do in SAP. Our questions led them to conclude that we didn’t know anything about SAP (which was mostly correct). They tattled to their manager who contacted our main client, alarmed that we weren’t SAP experts. Fortunately, he reassured her that we were user experience experts, and we weren’t supposed to be SAP experts.

What turns a participant into a tattle tale? Usually, it comes from a misunderstanding of our role. As user experience consultants working on many different projects, we constantly have to learn about new organizations, new systems, new processes, new technologies, and new types of people. We interview business stakeholders and conduct user research to learn about these things, but our goal isn’t to become experts. In fact, it’s often better to not be an expert. We have the advantage of seeing a group, system, process, or set of tasks from an outside perspective. Because we don’t already have the same insider knowledge as the business stakeholders and participants,  we can get them to explain things to us as outsiders. To do this, we often need to ask basic questions and sometimes even act “dumb” to get participants to fully explain things that they would otherwise forget to explain or gloss over at a high level.

The worst thing about tattling is that it can make us afraid to ask questions for fear that they might expose our “ignorance.” So make it clear who the experts are – the users and the business stakeholders, and where your expertise lies – user experience. Combining the expertise of business stakeholders, users, and user experience professionals is the key to a successful project.