Today I published an article in UXmatters, Testing Your Own Designs. It’s often been said that you shouldn’t conduct usability testing on your own designs, because you may be too biased, defensive, or too close to the design to be an impartial facilitator. Although that may be the ideal, often UX designers don’t have a choice. They may be the only person available to test the design, so if they don’t test it, no one will. So in this article I provide advice for those times when you have to test your own design, and I also provide advice for when someone else tests your design.
I was hesitant to write this article, because it’s been a topic that many others have written about, but I felt that as someone who has been on all sides of the issue, I had something additional to add. Here are some other good articles about this topic:
Testing Your Own Designs: Bad Idea? and Testing Your Own Designs Redux by Paul Sherman
Should Designers and Developers Do Usability? by Jakob Nielsen
BECAUSE NOBODY’S BABY IS UGLY … SHOULD DESIGNERS TEST THEIR OWN STUFF? by Cathy Carr at Bunnyfoot
I just published an article on UXmatters, 10 User Research Myths and Misconceptions. It addresses common misunderstandings about user research that I’ve encountered over the years.
Here’s a bonus outtake from the article, Myth 11…
Myth 11: Field Research Is Better Than Usability Testing
On the other end of the spectrum from those who don’t understand the difference between user research and usability testing, are the user research elitists who think up-front, generative user research methods are far superior to usability testing. In this view, field studies take researchers out of the lab to observe people in their natural environments performing their usual activities, while usability testing takes place in the sterile, artificial environment of a usability lab and asks people to perform a limited set of artificial tasks. Instead of learning about people and what they really do, usability testing provides the limited value of learning whether people can perform your artificial tasks.
The Truth: Both Field Research and Usability Testing Have Their Places
Field studies and usability testing are two different methods used for different, but equally important, purposes. Field studies provide information to inform design, while usability testing evaluates a design. You have to make interpretations and conclusions from the user research and apply that to a design. Even after very thorough user research, you’re never completely sure that what you’ve designed will work well for the users. Usability testing is the evaluation that either confirms your decisions or points you to refinements. Both user research and usability testing are important and necessary. There’s no reason we can’t appreciate the value of both methods.
Analyzing the data is the most interesting part of user research. That’s where you see the trends, spot insights, and make conclusions. It’s where all the work comes together and you get the answers to your questions.
Why, then, did I publish an article in UXmatters – Analysis Isn’t Cool? All too often I’ve realized that clients, management, and project stakeholders underestimate the analysis phase and just want to get to the answers. People like to say that they did user research, but they don’t like to spend the time to analyze the data. They like the deliverables, whether they read them or not, but they don’t want to spend a lot of time on the analysis to produce those deliverables.
In this article, I discuss what analysis involves, methods for individual and group analysis, and ways to speed up the analysis process.
Photo by Josh Evnin on Flickr
In previous research projects, there have been several times when participants were suspicious of our motives. This tends to happen when you’re doing research with a group of employees, trying to understand their work processes. These are the times that you’re trying to observe what they do in their jobs more than studying an existing system.
When there’s a direct connection to an application that they use, people tend to feel less suspicious. They can see that you’re trying to understand how well the application works and where it can be improved. This especially happens when it’s in a company that’s had poor previous experiences with reorganizations, layoffs, and offshoring. People tend to see us as another group of consultants coming in to study how they do their work to see what can be improved or who can be eliminated.
So how do you reassure people in these difficult situations about your true purpose? I wrote a recent article about this at UXmatters – Winning Over Wary Participants. Check it out, and if you have additional tips to make people feel more comfortable in these situations, feel free to leave a comment.
Image: Jake Sutton
Are consent forms always necessary? We’re told that consent forms are an indispensable part of ethical user research. Consent forms are the vehicle to give and get informed consent – they inform the participants of what the study will entail and they allow the participant to indicate consent – with a signature and date.
Yet consent forms can conflict with the informal, friendly rapport that we try to establish with participants. Anything you present for people to sign immediately looks like a legal document or liability waiver. It puts them on guard.
That’s ironic because consent forms are the opposite of legal waivers. Legal documents are created to protect the interests of the company that creates them, while consent forms are created to protect the rights of the people signing them. Yet most participants assume they are signing a typical legal waiver.
Consent forms seem acceptable in more formal user research situations, such as usability testing and focus groups, but they seem odd and even off-putting when used in more informal situations. I’ve found them to be especially awkward when doing field studies at people’s offices. You strive to set up an informal situation, such as asking someone to show you how they create reports or asking them to try out a new design for an expense report application. But when you show up with a consent form for them to sign, it shatters the informal, comfortable rapport you tried so hard to establish. I’ve had people react to consent forms in this kind of situation with, “Hey! I thought we were just talking here.” How many times in the course of your work-life have you had someone show up to a meeting with a legal document for you to sign?
So I say use your judgment. When a consent form feels like it would be overly formal, don’t use it (unless your legal department requires it). Instead, get informed consent informally by email. “Inform” with your email describing what will take place, and get “consent” from their reply email agreeing to participate. At the start of the session, you can inform them again with a summary of what you’ll be doing. They will then give consent by continuing to participate in the session.
A good guideline is how comfortable or uncomfortable you feel when giving participants the consent form. If you feel uncomfortable, you’re probably breaking a group norm. So you should find a more acceptable way of getting informed consent.
I presented at UXPA 2013 today on Effectively Communicating User Research Findings.
This is the reason I’ve been way too busy to blog lately. I’ve been working non-stop on this presentation and also my UXmatters article published this week on Creating Better UX Research Videos: http://www.uxmatters.com/mt/archives/2013/07/creating-better-ux-research-videos.php
So check either of these out.
Old Man Usability
Okay, now wait just a goddamn minute! UX testing? U-X testing?!! Now that’s just going too far!
You think you’re all better than me and don’t need “usability” anymore? “User experience” is a more inclusive and descriptive term about the aspects we’re interested in these days. Yeah yeah, fine. It’s more than just usability. Okay, I get it.
But keep your damn UX hands off my usability testing!!! That’s my signature method. I invented that! Talk about kicking a man when he’s down.
What am I talking about, you say? I’ve begun to notice this disturbing trend of you UX creeps stealing my method and calling it “UX” testing. Just look at this recent article from those fancy-pants, “digital marketers” over at eConsultancy: A Case for UX Testing and Agile. And then I noticed this article from last year: UX Testing and Cultural Preferences. Even User Zoom has gotten into the act with this article: 17 Questions Answered About UX Testing and Agile. And it doesn’t stop there. I just Googled “ux testing” and got 28,300 results!
Usability testing has been providing more than just usability for a long time. So in some ways I see your point that perhaps the word “usability” only describes part of what this method provides insight into.
But usability testing is the one method that’s still primarily about usability. Put participants in a lab (or test them remotely), give them tasks to perform, observe their behavior, and ask them to tell you what they’re thinking – that’s usability testing. In addition to assessing usability, it can provide information about satisfaction, emotions, and opinions, but it doesn’t give you a true depiction of the user experience. Other UX research methods give you a better picture of the user experience by observing people in their natural contexts of use. You can test usability, but you can’t really test the user experience.
And what are these people who are doing “UX testing” really doing? You guessed it! Usability testing. It’s nothing different. Just a name change.
So, I agree that user experience makes sense, but that doesn’t mean you should do a global find and replace, turning every instance of “usability” into “user experience.”
So keep your damn hands off my usability testing! It will always be “usability testing” no matter what you want to call it.
By the way, Googling “usability testing” brings up 2,110,000 results. So there!
– Old Man Usability