Skip to content

Paper Prototyping: Is it still worth it?

In my latest UXmatters article, I compare the latest prototyping tools to paper prototyping. Paper has long had the advantage in allowing designers to quickly and easily create early prototypes, that look unfinished, and encourage users to honestly provide criticism. However, the latest prototyping tools have caught up to, and in some cases surpassed, paper in making it very easy and quick to create prototypes without any coding.

So, do the advantages of paper prototypes still beat these new prototyping tools? That’s what I explore in my latest article, Prototyping: Paper Versus Digital.

Image credit: Samuel Mann

Testing Your Own Designs

Usability testing session

Today I published an article in UXmatters, Testing Your Own Designs. It’s often been said that you shouldn’t conduct usability testing on your own designs, because you may be too biased, defensive, or too close to the design to be an impartial facilitator. Although that may be the ideal, often UX designers don’t have a choice. They may be the only person available to test the design, so if they don’t test it, no one will. So in this article I provide advice for those times when you have to test your own design, and I also provide advice for when someone else tests your design.

I was hesitant to write this article, because it’s been a topic that many others have written about, but I felt that as someone who has been on all sides of the issue, I had something additional to add. Here are some other good articles about this topic:

Testing Your Own Designs: Bad Idea? and Testing Your Own Designs Redux by Paul Sherman

Should Designers and Developers Do Usability? by Jakob Nielsen

BECAUSE NOBODY’S BABY IS UGLY … SHOULD DESIGNERS TEST THEIR OWN STUFF? by Cathy Carr at Bunnyfoot

This One Goes to 11

I just published an article on UXmatters, 10 User Research Myths and Misconceptions. It addresses common misunderstandings about user research that I’ve encountered over the years.

Here’s a bonus outtake from the article, Myth 11…

Myth 11: Field Research Is Better Than Usability Testing

On the other end of the spectrum from those who don’t understand the difference between user research and usability testing, are the user research elitists who think up-front, generative user research methods are far superior to usability testing. In this view, field studies take researchers out of the lab to observe people in their natural environments performing their usual activities, while usability testing takes place in the sterile, artificial environment of a usability lab and asks people to perform a limited set of artificial tasks. Instead of learning about people and what they really do, usability testing provides the limited value of learning whether people can perform your artificial tasks.

The Truth: Both Field Research and Usability Testing Have Their Places

Field studies and usability testing are two different methods used for different, but equally important, purposes. Field studies provide information to inform design, while usability testing evaluates a design. You have to make interpretations and conclusions from the user research and apply that to a design. Even after very thorough user research, you’re never completely sure that what you’ve designed will work well for the users. Usability testing is the evaluation that either confirms your decisions or points you to refinements. Both user research and usability testing are important and necessary. There’s no reason we can’t appreciate the value of both methods.

UX Testing?!!

Old Man Usability

Old Man Usability

Okay, now wait just a goddamn minute! UX testing? U-X testing?!! Now that’s just going too far!

You think you’re all better than me and don’t need “usability” anymore? “User experience” is a more inclusive and descriptive term about the aspects we’re interested in these days. Yeah yeah, fine. It’s more than just usability. Okay, I get it.

But keep your damn UX hands off my usability testing!!! That’s my signature method. I invented that! Talk about kicking a man when he’s down.

What am I talking about, you say? I’ve begun to notice this disturbing trend of you UX creeps stealing my method and calling it “UX” testing. Just look at this recent article from those fancy-pants, “digital marketers” over at eConsultancy: A Case for UX Testing and Agile. And then I noticed this article from last year: UX Testing and Cultural Preferences. Even User Zoom has gotten into the act with this article: 17 Questions Answered About UX Testing and Agile. And it doesn’t stop there. I just Googled “ux testing” and got 28,300 results!

Usability testing has been providing more than just usability for a long time. So in some ways I see your point that perhaps the word “usability” only describes part of what this method provides insight into.

But usability testing is the one method that’s still primarily about usability. Put participants in a lab (or test them remotely), give them tasks to perform, observe their behavior, and ask them to tell you what they’re thinking – that’s usability testing. In addition to assessing usability, it can provide information about satisfaction, emotions, and opinions, but it doesn’t give you a true depiction of the user experience. Other UX research methods give you a better picture of the user experience by observing people in their natural contexts of use. You can test usability, but you can’t really test the user experience.

And what are these people who are doing “UX testing” really doing? You guessed it! Usability testing. It’s nothing different. Just a name change.

So, I agree that user experience makes sense, but that doesn’t mean you should do a global find and replace, turning every instance of “usability” into “user experience.”

So keep your damn hands off my usability testing! It will always be “usability testing” no matter what you want to call it.

By the way, Googling “usability testing” brings up 2,110,000 results. So there!

– Old Man Usability

How is usability testing like beer?

Glass of beer

Flickr: HeadCRasher

My manager asked me recently, “Do you think usability testing has become a commodity?” He was referring to the fact that in the last year or two we’ve seen clients go with cheaper usability testing companies. He was questioning whether clients have decided that there’s no difference between usability testing companies except price. Quality isn’t a differentiator to them anymore (if it ever was).

It does seem that there are companies out there doing usability testing at increasingly lower prices. How they cut corners to get their costs low enough to still make a profit amazes me, and it makes me wonder what kind of quality the clients receive.

Then an analogy dawned on me – usability testing, as a consulting service, is like beer. There are many people out there who are perfectly happy drinking cheap beer. It’s cheap, it’s bland, but it does the trick in the end – it gives you a buzz. But that doesn’t mean everyone is satisfied with cheap beer. There are still those out there who appreciate and will pay more for craft beer with quality, taste, and a better buzz.

If you want cheap usability testing, you can get it. It won’t taste good or be the best quality, but if all you’re looking for is a cheap buzz, it will do the trick in the end. On the other hand, if you have a sophisticated enough palette, you’ll be able to tell the difference between cheap usability testing and craft usability testing, and you’ll be more satisfied in the end. The bottom line is: you get what you pay for. There’s usability testing out there for all tastes and budgets.

Too Cool for Your Usability Test

CoolNo matter how well you recruit representative participants for a usability test and no matter how well you plan the testing, there are times when you’ll ask participants to perform a task that they might not normally perform themselves. It’s rare that every task you ask people to perform matches exactly what they would do. When this happens, most participants are agreeable enough to just “play along” for the purposes of the test.

Sure, it’s good to know what a participant would normally do instead of your planned task, but that’s more useful to learn during field studies. During a usability test, you usually just want to observe how well people can perform tasks.

During a recent usability test of an intranet design, I asked participants to browse the Blogs section to test out the usability of the filtering and searching functions common across the various sections of the intranet. I used the Blogs section as an example because that was the section we had built out in our prototype. Unfortunately, I came across two participants who were “too cool” to read blogs. In fact, they were too cool to even play along with my ridiculous and demeaning scenario.

It went something like this:

Me: Show me where you’d go if you wanted to see all the blogs in the company.

Joe Cool: Oh, I wouldn’t do that.

Me: Why?

Joe Cool: I don’t read the blogs.

Me: Why is that?

Joe Cool: Who cares about blogs? I don’t have time to read blogs.

Me: Okay, but if you did want to see all the blogs, where would you find them?

Joe Cool: I really wouldn’t do that. People here don’t really pay attention to the blogs. Who has time for that? We have enough to do with…

[Two minutes later]

Me: Okay, well that’s good to know, but just for the sake of this session, let’s say that you did want to read the blogs, where would you go to do that?

Joe Cool: [Sigh] Well, I guess I’d go here, and – here it is. But you see the problem with blogs is that…

[One minute rant later]

Me: Okay, what would you do if you wanted this to show you the most popular blog posts in the company?

Joe Cool: I don’t really care about what other people think is popular, especially from people who read blogs.

Me: Okay, but if someone else wanted to see the most popular blog posts in the company, what should they do here?

Joe Cool: Maybe they should ask someone else who reads blogs a lot? Or they should get a life and do something more productive.

Me: Okay, let’s move on to the next part…

Luckily, the next task was cool enough for him. Sometimes that’s all you can hope for.

Caring for Your User Researcher

Affectionate dog

Affectionate dog (Photo credit: Wikipedia)

Congratulations! You are now the proud owner of a user researcher. Treat him or her well and you’ll have years of effectively designed products. This guide will help you in the care and nurturing of your user researcher.

Feed your researcher well

  • Research days are often very busy and unpredictable. The most important person to consider on these days is not the clients or other observers; it’s the person who has to remain sharpest – the researcher.
  • Make sure your researcher is well fed and has enough time to eat meals. This will provide the energy needed to concentrate on the research session.
  • You may remember the meals, but don’t forget the snacks. If everyone else gets cookies, make sure you save some for the researcher.

Avoid burnout

  • Research is often very mentally challenging. It’s easy for your researcher to get burned out, and when that happens, he or she can’t operate at top form, which could mean missing things or not asking the right questions.
  • Don’t schedule too many sessions in a single day. Four to five usability testing sessions and three or four field studies are about the maximum before burnout sets in.
  • Provide enough breaks between sessions, and allow your researcher to relax during those times.
  • Don’t fill the break times with meetings or discussions with clients. Provide time to rest the mind. Your researcher may need to step away and have some alone time.

Provide variety

  • Most researchers don’t want to do the same things all the time. If you don’t provide enough variety, eventually he or she will seek variety by going to another company.
  • Provide variety in the following aspects: clients, platforms to work on (websites, intranets, mobile devices, software, products, service design, etc.), research activities (usability testing, unmoderated studies, field studies, etc.), and types of people to work with.

Let your researcher run free

  • Don’t micromanage your user researcher. Trust his or her judgment.
  • Provide input on what you want to learn from the research, review the research plan, but give your researcher the independence to make the final decisions on how to accomplish the research goals. Remember you selected your researcher for his or her expertise. Listen to it.

Provide enough time to do quality work

  • Ensure that your researcher has enough time to plan the research and analyze the results.
  • Understand that your researcher will often get very interested and involved in the results and will want to produce a thorough deliverable. If there is time, allow for this.

Give praise and recognition

  • Like most people, your researcher will appreciate praise, recognition, and rewards for a job well done.
  • Your researcher will appreciate it when people take the user research seriously and appreciate the findings.

Let your researcher out to play with others

  • Your researcher will often interact with users, but otherwise, research is often a solitary activity that can get somewhat lonely.
  • Encourage your researcher work with others on the project team, such as designers and developers, to provide a more well-rounded perspective of their work and to give them a first-hand insight into the research.
  • Don’t exclude your researcher as soon as the research part of the project is completed. Keep your researcher around to use his or her valuable knowledge in the design and development phases.

Encourage your researcher’s development

  • Encourage your researcher to keep up with developments in the field by reading books, reading blogs and other Web resources, and attending events.
  • Encourage him or her to publish, present, and attend conferences and other industry events.

By following these guidelines, you should have a long and healthy working relationship with your user researcher. Good luck and have fun!