Skip to content

The Most Difficult User Research Method

User research participant at desk

What do these three things have in common – playing in a one-man band, juggling chainsaws, and babysitting 10 three-year-olds? When you try to do all of these things at the same time, it’s only slightly more difficult than conducting field studies.

Of course, I’m just kidding, but not by much. In my opinion, field studies are the most difficult user research technique for three reasons: unpredictability, the need to learn about unfamiliar domains, and the need to deal with competing demands. There’s not much you can do about unpredictability or the need to learn a new domain, but there are things that you can do to better handle the competing demands of field studies.

In my latest article on UXmatters, I discuss these competing demands and how to best handle them:

  • Observing and listening
  • Understanding
  • Determining whether and when to ask questions
  • Formulating questions
  • Assessing answers
  • Managing the session
  • Assessing the session
  • Keeping track of the time
  • Managing observers
  • Capturing the session
  • Maintaining a good rapport with the participant

Read more in my latest article, Handling the Competing Demands of Field Studies.

Image credit: Highways England on Flickr

More UX Analogies

I just published an article on UXmatters today, “Why So Many UX Analogies?” It’s an investigation into why there are so many articles that compare UX to other things and whether these UX analogies have any value. I mentioned some examples in my article, but here are links to many more UX Analogies. Trust me, this doesn’t even scratch the surface.

Professions


Other Fields


Pop Culture

 

Hobbies/Activities

 

History

 

Miscellaneous

UX Career Advice

Today I published a new article in UXmatters, “What to Consider When Choosing a UX Job.” It details the questions you should ask and consider when contemplating a new job in the UX field.

I’ve written several UX career advice articles over the years. I link to them on my Publications page, but here they are compiled for easy reference. I don’t claim to be a career expert, but these are just some of the things I’ve learned (sometimes the hard way) in my 15 years in UX.

What to Consider When Choosing a UX Job
UXmatters – January 4, 2016
Lately, it seems like there are more jobs in User Experience than ever before. Deciding whether to accept a particular position is always an important decision, but in a hot job market like this, with so many opportunities, choosing the right company to work for is more important than ever. As with any other job opportunity, there are typical criteria to consider such as salary, benefits, company culture, and the commute. But, in this article, I’ll focus on the special considerations when you’re contemplating a new UX job.

UX Generalists or Specialists?
UXmatters – September 7, 2015
This is a question that every UX professional faces at some point: is it better to be a UX generalist, or is it better to specialize? Companies often question whether a team of UX generalists or a mix of specialists is best. In this column, I’ll discuss the advantages and disadvantages of generalization and specialization for UX professionals and the companies that hire them.

Better UX Internships
UXmatters – March 10, 2014
An internship is a great way to get into the field of user experience, but internships are often failures—for both the intern and the hiring company. Why? The hiring companies often don’t have a plan for how to use their interns, and interns often don’t know how they can contribute or where they fit in. In this column, I’ll discuss what interns and companies can do to ensure a better internship experience.

Career Advice for User Researchers
UXmatters – December 5, 2011
Eleven years seems like a good point at which to reflect back on the things I’ve learned over my career and pass on some advice to those who are just getting started in the field of user research.

Publishing and Presenting, Part 1: Yes, You Can!
UXmatters – November 12, 2012
In Part 1, I’ll discuss the benefits of publishing and presenting, as well as explore the excuses that prevent people from doing either. Publishing and presenting is a lot of work, especially when you’re already a busy UX professional. So why bother? Here are a few reasons you should.

Publishing and Presenting, Part 2: Publishing
UXmatters – December 10, 2012
Perhaps Part 1 of this series convinced you of the benefits of publishing, dispelled your fears, and defeated the excuses that have prevented you from publishing in the past. But how do you get started writing, and how do you get your writing published? These are the questions I’ll answer in Part 2.

Publishing and Presenting, Part 3: Presenting
UXmatters – January 7, 2013
In part 3 of this of this series, I’ll discuss how to generate ideas for conference topics, find the right conference at which to present, submit a proposal, and create a presentation, and what to do during a conference where you’re presenting.

User Experience in Bulgaria

Me presenting at UXify

I’m happy to report that user experience is alive and well in Bulgaria. I just got back a week ago from attending and presenting at UXify 2015 in Sofia, Bulgaria. A very enthusiastic audience of over 300 people attended a day of presentations, followed by a day of in-depth workshops. UXify was the end of a month of user experience events – UX Month Sofia.

On Friday, June 19th, I presented User Research in the Wild, a presentation about visiting people in their natural environments to observe their tasks. In addition to being a how-to guide, I covered common problems you face in conducting field studies and how to solve them, and how to overcome obstacles to field research.

On Saturday, June 20th, I gave a workshop on Paper Prototyping to about 20 attendees. After an introduction to creating paper prototypes and how to test them, the participants divided into groups to create their own paper prototypes. We only had a limited time to create them (about 40 minutes), so I was very surprised by how detailed and creative their prototypes turned out in such a short period of time. Each group of two people then joined with another group to take turns conducting usability testing on their paper prototypes. Each group did four rounds of tests, switching roles, so that each person was able to experience the role of the facilitator, “the computer,” and the participant twice. The attendees really seemed to have a good time, with a lot of laughing and joking around.

Me presenting at the workshop

In between my own presentations, I attended several interesting presentations about dashboard design, gamification, socially responsible design, and ecommerce user experience.

It was a fun trip, and it was nice to meet a lot of similar people, in another culture, who are similarly committed and enthusiastic about user experience.

Another Feedback Loop

driving

I recently wrote an article in UXmatters about how the information displays in my Prius changed my driving behavior. Seeing the miles per gallon (MPG) I was getting in real time, acted like a feedback loop, causing me to make a connection between my driving actions and how they affected my gas mileage.

Something I didn’t write about, however, was another type of feedback that you get as a Prius driver – the drivers behind you. When someone drives up behind you and tailgates, that’s feedback to go faster or get out of the way. However, that feedback conflicts with the feedback from your Prius to drive economically.

To drive economically and get good gas mileage requires a steady pace and slow acceleration. That doesn’t necessarily mean slow driving. I drive 65 to 75 miles-per-hour to work, while getting about 60 miles per gallon. But it does mean that you need to avoid quickly accelerating. So being a Prius driver sometimes puts your gas-saving goals in conflict with the drive-fast goals of other drivers. There’s a reason some drivers get annoyed by Prius drivers.

Although I always drive in the right hand or middle lanes when I’m driving economically, I’ve found that the best way to drive economically is to ride along behind someone else. If there’s no one in front of you, and another driver comes up from behind, you feel pressure to go faster. If you have another driver in front of you, you have an excuse to drive economically. It’s that guy’s fault, you’re just driving behind him at a reasonable distance. So you don’t feel the pressure to speed up.

I’ve gotten my highest MPG records from driving (at a safe distance) behind a truck that was going a steady, reasonable speed (maybe 70 MPH). That allows you to ignore the feedback from drivers behind you and pay attention to the MPG display.

Sounds like I’ve put too much thought into this gas mileage thing with my Prius. Maybe, but I save money and it’s fun.

What’s the Formula for Success in Recruiting User Research Participants?

Formula

If you believe that designing an effective user experience requires involving users throughout the design process, then you must admit that finding and recruiting participants for user research and usability testing is the most crucial step. After all, if you can’t get participants, you can’t do research or usability testing. A recent, difficult recruiting effort left me ruminating about what it is that makes recruiting either easy or difficult. Is there something like an equation that can account for the ease or difficulty of recruiting participants? Yes, there is: RE = ((K + S) x RM) x ((C+A) x PM) Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation) Okay, I just kind of made that up, but I think it’s a good way to visualize the various factors that influence how easy or difficult it will be to recruit participants. Let’s look at the components of the formula in more detail.

The Recruiter

The first part of the equation involves attributes of the recruiter: Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation) By recruiter, I mean whoever is doing the recruiting. It could be you, a recruiting company, your client, a salesperson, or someone else. Ideally, the researcher would also recruit the participants, because that person knows the most about who to recruit. But sometimes you have to rely on other people to do the recruiting, usually because they have a better connection to participants.

Knowledge

Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation) The first factor is the recruiter’s knowledge of the research and the type of people to recruit.

  • Does the recruiter know the characteristics of the people that are desired? Can the recruiter identify people who match those characteristics?
  • How well can the recruiter describe the research to the participants?

Without this knowledge, recruiters may select the wrong types of people or give them the wrong impression about the research. If you’re not doing the recruiting, it’s important to give the recruiter a clear definition of the types of people you’re looking for and a simple description of the research that they can use when contacting participants.

Skills

Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation)  Effective recruiting also depends on the recruiter’s interpersonal, communication, and organizational skills.

  • Is the recruiter personable and persuasive enough to convince people to participate?
  • Can the recruiter clearly and concisely communicate with the participants?
  • Is the researcher organized and coordinated enough to contact, schedule, and keep track of many participants?

Ensure that the person doing the recruiting has these skills. If not, have them delegate this task to someone who does. Administrative assistants often excel at recruiting because they are already well versed in communicating and coordinating schedules.

Recruiter Motivation

Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation) When finding participants becomes difficult, recruiter motivation is the most important criteria.

  • How much does the recruiter care about finding participants?
  • Will there be positive or negative consequences for the recruiter if participants are found or not found?

If you’re doing the research and the recruiting, you’re usually highly motivated to find participants, but when you have to rely on someone else, try to make sure that person has a stake in the recruiting. The negative consequences of not finding participants and derailing the project are highly motivating. Once a project starts and dates are set, there’s a ticking clock counting down the days to the research sessions. Delays in finding participants will delay the project.

With deadlines looming and money on the line, there’s a lot of pressure for whoever’s responsible to find participants. You can bet that most of the time, it gets done. Professional recruiting companies have a financial motivation to find participants, since they get paid a certain amount for each participant they recruit. However, if the recruiting turns out to be more difficult than they expected, they will begin to lose money as they spend more time finding each participant. When that happens, they’ll usually try to get you to loosen up the recruiting requirements to make it easier to find people, or they may say that they’ve exhausted their efforts and give up.

The Participants

The second part of the equation involves the recruiter’s relationship to the participants: Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation)

Connection

Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation) When people have a connection with the recruiter, they’re more likely to agree to participate. The strength of a connection can take various forms:

  • The participant may know the recruiter personally.
  • The participant may share a common affiliation with the recruiter. For example, they may be fellow employees or members of the same organization.
  • The participant may have an affiliation with an organization that the recruiter represents. For example, they may be a customer of the company the recruiter represents.

An email or phone call to participate in a study, often raises the questions about whether it’s legitimate, whether it’s worth considering, and whether it’s worth the time to participate. The closer the connection the person has with the recruiter, the more likely they will take the time to consider the request. Unless you already have this relationship, you might have someone else, who the potential participants know, do the recruiting. They’re more likely to listen to, consider, and respond to someone they know rather than a complete stranger.

Access

Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation) When you have access to lists of potential participants (such as employees, organization members, or customers) it’s much easier to recruit participants, especially when those lists contain characteristics that you can use to filter and narrow down those lists to the right types of people. It’s very difficult when you and your client have no access to the types of people you need. That’s when it makes sense to use a recruiting company.

Participant Motivation

Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation) Once they’ve been contacted and have considered the pitch, motivation determines whether a person will volunteer to participate. We can look at participant motivation as its own sub-equation: PM = A – D Participant Motivation = Advantages of participating – Disadvantages of participating To make it worth participating, the advantages have to be greater than the disadvantages. A > D = Participate A < D = Decline The advantages of participating can include things like:

  • Incentives – A $150 check, $50 Starbucks gift card, free software, etc.
  • Personal benefit from the improvements that come out of research – They’re going to improve this horrible software that I have to use every day at work!
  • Feelings of altruism
  • Feeling good about having your ideas heard and being able to improve something
  • Novelty of doing something different and interesting
  • Getting praise for participating and avoiding getting in trouble for not helping out

The disadvantages of participating can include things like:

  • The difficulty and effort it takes to participate
  • The time it takes to participate
  • Discomfort, anxiety, and fear of the unknown

Add it Up

Actually, something this complex can’t be distilled into an easy formula, but I think this equation provides a good visualization of the important factors. The quality and motivation of the recruiter, combined with the ease of finding participants, and the way people perceive the benefits they’ll receive versus the hassles they’ll have to go through, determines how easy or difficult it will be to recruit participants.

Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation)

 

Mathematics image courtesy of Tom Brown