Skip to content

Is Any User Research Better Than None?

Donkey looking at camera

Any user research is better than doing no user research, right? If you can’t reach your target users, you can do research with your company’s employees, because they’re kind of similar right? If you can’t visit people in person to see them perform their tasks, maybe you can do phone interviews or send out a survey. That’s better than nothing, right?

The truth is that it’s sometimes better not to do any user research than to do half-assed user research. I’m not saying that you always have to the perfect user research conditions or its not worth doing. In reality, we rarely have all the time we need and the perfect circumstances to conduct extensive user research. So it’s understandable that we sometimes have to cut corners and make do with what we’re able to get. However, there’s a fine line between discount user research and half-assed user research.

The danger is when you always cut corners, you can become an enabler. Your shortcuts become the norm, allowing your company to check off the user research checkbox, allowing them to say, “Yes, we do user research.” If you can’t eventually convince them to devote more time and effort to user research, sometimes it’s better to practice tough love and let them fail by not doing any user research, rather than allowing them to rely on poor quality research.

In my latest UXmatters article, I provide advice about how to know when you’re practicing half-assed user research and how to improve. Check it out: Avoiding Half-Assed User Research

Image by Spider.Dog

What Personal Qualities Do You Need As a User Researcher?

""

Have you ever wondered what qualities you need to succeed in user research? I just published an article on UXmatters, Qualities of Effective User Researchers, which lists the following qualities that lead to a successful career in user research:

  • Curiosity
  • Idealism
  • Pragmatism
  • Persuasiveness
  • Open-Mindedness
  • Ability to Learn Quickly
  • Organizational Skills and Attention to Detail
  • Time Management Skills
  • Collaboration
  • Empathy
  • Friendliness
  • Neutrality
  • Perceptiveness
  • Patience
  • Mental Agility
  • Flexibility and Adaptability
  • Good Memory
  • Effective Notetaking
  • Analytical Skills
  • Problem Solving
  • Design Skills
  • Writing Skills
  • Communication Skills

This may sound like an intimidating list, but you don’t have to be perfect in all of these areas. Check out the full article on UXmatters – Qualities of Effective User Researchers.

Cow image by FFCU (Free for Commercial Use) by Creative Commons License

Paper Prototyping: Is it still worth it?

In my latest UXmatters article, I compare the latest prototyping tools to paper prototyping. Paper has long had the advantage in allowing designers to quickly and easily create early prototypes, that look unfinished, and encourage users to honestly provide criticism. However, the latest prototyping tools have caught up to, and in some cases surpassed, paper in making it very easy and quick to create prototypes without any coding.

So, do the advantages of paper prototypes still beat these new prototyping tools? That’s what I explore in my latest article, Prototyping: Paper Versus Digital.

Image credit: Samuel Mann

Questions People Have a Hard Time Answering

Question marks

Over the years, I’ve made my share of mistakes and learned about the types of questions and topics that participants have a hard time answering accurately in user research. Most people do try to answer your questions, but they may not be able to easily and accurately answer these types of questions:

  • Remembering details about the past
  • Predicting what they might do in the future
  • Accurately answering a hypothetical question
  • Discussing the details of their tasks out of context
  • Telling you what they really need
  • Imagining how something might work
  • Envisioning an improved design
  • Distinguishing between minuscule design differences
  • Explaining the reasons for their behavior

I discuss these types of difficult questions, and better ways to get that information from participants, in my latest article on UXmatters:
Avoiding Hard-to-Answer Questions in User Interviews.

Image credit: Véronique Debord-Lazaro on Flickr

The Most Difficult User Research Method

User research participant at desk

What do these three things have in common – playing in a one-man band, juggling chainsaws, and babysitting 10 three-year-olds? When you try to do all of these things at the same time, it’s only slightly more difficult than conducting field studies.

Of course, I’m just kidding, but not by much. In my opinion, field studies are the most difficult user research technique for three reasons: unpredictability, the need to learn about unfamiliar domains, and the need to deal with competing demands. There’s not much you can do about unpredictability or the need to learn a new domain, but there are things that you can do to better handle the competing demands of field studies.

In my latest article on UXmatters, I discuss these competing demands and how to best handle them:

  • Observing and listening
  • Understanding
  • Determining whether and when to ask questions
  • Formulating questions
  • Assessing answers
  • Managing the session
  • Assessing the session
  • Keeping track of the time
  • Managing observers
  • Capturing the session
  • Maintaining a good rapport with the participant

Read more in my latest article, Handling the Competing Demands of Field Studies.

Image credit: Highways England on Flickr

This One Goes to 11

I just published an article on UXmatters, 10 User Research Myths and Misconceptions. It addresses common misunderstandings about user research that I’ve encountered over the years.

Here’s a bonus outtake from the article, Myth 11…

Myth 11: Field Research Is Better Than Usability Testing

On the other end of the spectrum from those who don’t understand the difference between user research and usability testing, are the user research elitists who think up-front, generative user research methods are far superior to usability testing. In this view, field studies take researchers out of the lab to observe people in their natural environments performing their usual activities, while usability testing takes place in the sterile, artificial environment of a usability lab and asks people to perform a limited set of artificial tasks. Instead of learning about people and what they really do, usability testing provides the limited value of learning whether people can perform your artificial tasks.

The Truth: Both Field Research and Usability Testing Have Their Places

Field studies and usability testing are two different methods used for different, but equally important, purposes. Field studies provide information to inform design, while usability testing evaluates a design. You have to make interpretations and conclusions from the user research and apply that to a design. Even after very thorough user research, you’re never completely sure that what you’ve designed will work well for the users. Usability testing is the evaluation that either confirms your decisions or points you to refinements. Both user research and usability testing are important and necessary. There’s no reason we can’t appreciate the value of both methods.

Analysis Is Cool

Affinity diagram

Analyzing the data is the most interesting part of user research. That’s where you see the trends, spot insights, and make conclusions. It’s where all the work comes together and you get the answers to your questions.

Why, then, did I publish an article in UXmatters – Analysis Isn’t Cool? All too often I’ve realized that clients, management, and project stakeholders underestimate the analysis phase and just want to get to the answers. People like to say that they did user research, but they don’t like to spend the time to analyze the data. They like the deliverables, whether they read them or not, but they don’t want to spend a lot of time on the analysis to produce those deliverables.

In this article, I discuss what analysis involves, methods for individual and group analysis, and ways to speed up the analysis process.

 

Photo by Josh Evnin on Flickr

Better Group Research Sessions

Group working together

Many user experience professionals cringe when clients want to do focus groups. In UX we extol the value of user research with individuals, either in usability testing or field studies, where we observe and interview individuals in the context of their tasks. Focus groups seem like a flaky, unreliable marketing activity. Unfortunately, because of our disdain for focus groups we sometimes extend that disdain to any research activity that involves a group of people.

Why Don’t We Like Focus Groups?

In UX we highly value observing the behavior of individuals in their natural context. Focus groups provide the exact opposite. They reveal what people say in a group out of context of their typical environment.

What People Say

Focus groups usually gather feelings, attitudes, opinions, and preferences; but they can’t tell us about behavior. We can’t rely much on what people say they do, because so much of what we do is done automatically without much conscious thought. So it’s difficult for people to accurately talk about what they do, out of context. That’s why we like to observe people in the context of performing their tasks. It’s more accurate, and it’s easier for people to show us what they do.

 In a Group

Being in a group has its limitations:

  • Group dynamics often influence individuals to go along with the group instead of speaking their own minds. They often say what they think will sound good to the group, and they may not feel comfortable speaking honestly about certain topics.
  • A few outspoken people often dominate the discussion, while others remain more silent.
  • You get much less information from each person than you would in individual sessions.
  • Group sessions are not useful for evaluation, because it’s hard for people to evaluate a design or prototype unless they can try it out themselves, which usually isn’t possible in a group.

 Out of Context

Focus groups bring people out of their usual context into an unnatural environment. We don’t get to see their usual behavior, the tasks they perform, their tools and technology, their environment, or the people they typically interact with.

Group Sessions Can be Useful

Because of these disadvantages with focus groups, we tend to overlook the value of any type of group research session. However, group research sessions can be valuable. Even when we are specifically requested to conduct focus groups, we don’t have to conduct traditional, marketing focus groups. We can modify them to make them more useful.

Why Conduct Group Research?

Group sessions provide some advantages over individual sessions:

  • You save time and money by meeting with many participants at once.
  • Group sessions are a useful way to quickly gather initial information about a topic, before going out to do individual user research sessions. For example, you could learn about a process at a high-level by having a group of employees each talk about their role in the process.
  • Interaction between participants can spark discussions that might never be revealed in individual interviews. For example, a group of employees from the same company may have a more productive discussion about their work than if you held individual interviews with each of them separately. Because they all know each other and have things in common, they may talk about things together that they would never bring up to you, as an outsider.
  • In group sessions, you can have participants work together in small groups and listen to their decision making and reasoning. For example, you can do group card sorting to hear how participants make decisions about how to group the cards.

Research Formats

In addition to group discussion, you can break up a group into smaller groups or give them individual activities.

The Entire Group

Discussing topics as a group takes advantage of the group dynamic to hear different perspectives, to get consensus, to debate, or to hear about different parts of a process.

Smaller Groups

Breaking into several smaller groups allows individuals to work together on activities that would be unwieldy to do with the entire group. As they work together, you can listen to and observe their decision making. You can then have each group present their results to the entire group to generate further discussion.

Individual Activities

Another option is to have the participants each perform an activity on their own and then present it to the group. This is effective with activities that require thought and are easier to do alone. For example, you could give the participants time to draw diagrams of their process, and then you could ask them to present their drawing to the entire group. You get the advantage of getting each person’s individual perspective but also with the advantage of group discussion.

Group Size

Any group activity becomes difficult to manage with too many participants. More than ten participants are usually too much. If there are more than ten people that need to be involved, it’s better to schedule multiple sessions.

Dyads and Triads

Market research uses terms like dyads and triads to describe group sessions with two and three participants. Groups of these sizes have different dynamics than a large group. They combine some of the dynamics of individual interviews with those of group sessions. In user experience, dyads (sessions with two participants) or triads (sessions with three participants) make sense when there’s a natural relationship between those people. For example, if you’re researching the car buying process, it might make sense to interview a husband and wife together. Or if three employees perform different parts of a related process, it may make sense to talk with them together to understand the process and how their roles relate.

Alternative Group Research Activities

In addition to group discussions, there are many creative and unique activities that you can use to gather information about the users and their tasks, and to get user input into the design. Instead of describing these techniques in detail, each of which could be its own article, I’ve provided links to read more about each technique.

Group Card Sorting

If organization of information and gathering common terminology is your goal, group card sorting is a quick way to gather that information. You can break up the group into smaller sub-groups of two or three participants each. They then work together to sort pieces of information on index cards or post-it notes into categories and sub-categories, and then name each category. For more information, see: http://www.usability.gov/how-to-and-tools/methods/card-sorting.html

Affinity Diagrams

Similar to card sorting, you can have sub-groups work together to group related items on post-it notes and then label those groups to create affinity diagrams. Afterwards, the entire group can discuss the themes that emerge. For more information, see: http://www.usabilitybok.org/affinity-diagram

Drawing or Modeling

Breaking the group into small groups or individuals and having them illustrate or build a model of their ideas, concepts, workflows, or processes is a great way to get insight into their thought processes and mental models. After each person presents their drawing or model to the group, discussion is facilitated by having these concrete items to talk about. For more information, see: http://uxmag.com/articles/creativity-based-research-the-process-of-co-designing-with-users and http://johnnyholland.org/2011/10/storyboarding-ux-part-3-storyboarding-as-a-workshop-activity/

Collaging

Individuals or groups can create collages from a set of provided images to express their feelings about a particular issue. The collages then can be described and discussed. For more information, see: http://www.smashingmagazine.com/2012/02/06/collaging-getting-answers-questions-you-dont-know-ask/

Priming Activities

You can give the participants assignments to do before the group session, such as writing about or taking photos of an experience, creating drawings, or creating collages. Because they do this outside the group session, it allows more time for presenting their work and discussing it as a group. It also brings in a little bit of each person’s real experience outside of the conference room. For more information, see: http://johnnyholland.org/2010/05/not-to-prime-is-a-crime/

Design Games

Several sources provide creative group activities to provide input to design problems. Gamestorming, a book by Dave Gray, Sunni Brown, and James Macanufo provides more than 80 group games to encourage creativity and generate new ideas. The site Design Games also provides a number of games with complete directions about how to play them.

Group Sessions as a First Step

Group sessions with users or stakeholders are an efficient way to get high-level information about subject matter, users, tasks, and processes. With participatory design and design games, you can get user input into the design. Ideally, group sessions shouldn’t replace individual observations and interviews with users, but they are a good first step to gather the initial information to understand the basics before going out to conduct field research.

 

 

Group photo courtesy of Kennisland on Flickr under Creative Commons license

Two Rarely Used Research Methods

Observing in a public areaMy two most recent articles in UXmatters have been about two research techniques that are very common to anthropology and the social sciences but are rarely used in design research. Covert naturalistic observation and participant observation certainly require more work and time than we usually have in UX projects, but it’s worth taking a look at these two methods to see what we can adapt to design research.

Covert Naturalistic Observation
This type of study is known in psychology, anthropology, and other social sciences as covert naturalistic observation. It is the opposite of the techniques we typically use, which are forms of overt naturalistic observation. Being covert means observing behaviors in their natural contexts without any intervention or influence by the researcher and without participants knowing that they’re being observed.

Participatory Observation
Anthropologists and sociologists often practice participant observation, in which they join a group as a participating member to get a first-hand perspective of the group and their activities. Instead of observing as an outsider, they play two roles at once—objective observer and subjective participant…

A UX Researcher’s New Year’s Resolutions

New Year

Flickr: RLHyde

Try new things

In 2013, I’ll try new techniques instead of relying on the same routine research activities. For each project, I’ll step back and think about what research activities make the most sense based on the situation. Trying new things and inventing new techniques keeps things interesting.

Work faster

This year, I’ll do some things faster, to be more agile and lean where possible so that research continues to be included in projects. We’ve done a good job of selling clients and project team members on the value of including user research in projects. The remaining hurdle is that it often takes longer than they would like. There definitely are areas that can be sped up.

Work slower

In 2013, I’m going to use the time I save on working faster to spend more time on the activities that provide the most value. Some things shouldn’t be rushed. Analysis of research data, for example, is the most important, but least understood part of user research. No one ever seems to understand what analysis involves, how long it can take, and how important it is. In 2013, I’m going to fight for the time needed for analysis and do a better job explaining what it involves, why it takes so long, and why it’s so important to give it the time needed.

Get better participants

In 2013, instead of aiming high and settling for what we’re able to recruit, I’m going to create better screeners and spend more time making sure that the people who have been recruited, match the type of people we want to get. I’m going to be especially careful when clients are doing the recruiting of their own customers, employees, or members. I’ll give them better instructions in findings and recruiting people, and I’ll evaluate the types of people that they’ve scheduled.

Publish and present

In 2013, I’ll continue to publish articles in UXmatters and elsewhere. I’ll try to present at a conference. Publishing and presenting are great ways to share your knowledge with others in the field. For more tips on publishing and presenting, see my series of articles on Publishing and Presenting in UXmatters.

Attend more UX events

In 2013, I’ll attend more local UX events. In Philadelphia, we have a very active CHI group, PhillyCHI, but I always find it difficult to get motivated to go out after a long work day and attend their events. Every time I do attend, however, I find that it’s a great way to meet others in the field, and I always learn something useful.

Read more

In 2013, I’d like to read more UX-related books and articles. That’s easier said than done when you’re really busy at work. And after a day practicing UX, it’s difficult to get motivated to read UX in your spare time. Usually, I want to read anything else. Fortunately, a great trend in UX books is towards shorter, more practical books that can be read quickly (for example, the Rosenfeld Media books).

Be thankful for what I have

It can feel good to complain and think about what could be better about your job, but I find that I often don’t think about how good I have it. I’m doing a job I enjoy, that’s challenging, and usually interesting. In 2013, I’d like to focus more on the positive and appreciate what I have. If I find that I have only complaints and nothing to be thankful for, I’ll know that it’s up to me to change things.

Those are my resolutions. I hope I can achieve most, if not all of them, in 2013.