Skip to content

Posts from the ‘User Experiences’ Category

Another Feedback Loop

I recently wrote an article in UXmatters about how the information displays in my Prius changed my driving behavior. Seeing the miles per gallon (MPG) I was getting in real time, acted like a feedback loop, causing me to make a connection between my driving actions and how they affected my gas mileage.

Something I didn’t write about, however, was another type of feedback that you get as a Prius driver – the drivers behind you. When someone drives up behind you and tailgates, that’s feedback to go faster or get out of the way. However, that feedback conflicts with the feedback from your Prius to drive economically.

To drive economically and get good gas mileage requires a steady pace and slow acceleration. That doesn’t necessarily mean slow driving. I drive 65 to 75 miles-per-hour to work, while getting about 60 miles per gallon. But it does mean that you need to avoid quickly accelerating. So being a Prius driver sometimes puts your gas-saving goals in conflict with the drive-fast goals of other drivers. There’s a reason some drivers get annoyed by Prius drivers.

Although I always drive in the right hand or middle lanes when I’m driving economically, I’ve found that the best way to drive economically is to ride along behind someone else. If there’s no one in front of you, and another driver comes up from behind, you feel pressure to go faster. If you have another driver in front of you, you have an excuse to drive economically. It’s that guy’s fault, you’re just driving behind him at a reasonable distance. So you don’t feel the pressure to speed up.

I’ve gotten my highest MPG records from driving (at a safe distance) behind a truck that was going a steady, reasonable speed (maybe 70 MPH). That allows you to ignore the feedback from drivers behind you and pay attention to the MPG display.

Sounds like I’ve put too much thought into this gas mileage thing with my Prius. Maybe, but I save money and it’s fun.

What’s the Formula for Success in Recruiting User Research Participants?

If you believe that designing an effective user experience requires involving users throughout the design process, then you must admit that finding and recruiting participants for user research and usability testing is the most crucial step. After all, if you can’t get participants, you can’t do research or usability testing. A recent, difficult recruiting effort left me ruminating about what it is that makes recruiting either easy or difficult. Is there something like an equation that can account for the ease or difficulty of recruiting participants? Yes, there is: RE = ((K + S) x RM) x ((C+A) x PM) Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation) Okay, I just kind of made that up, but I think it’s a good way to visualize the various factors that influence how easy or difficult it will be to recruit participants. Let’s look at the components of the formula in more detail.

The Recruiter

The first part of the equation involves attributes of the recruiter: Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation) By recruiter, I mean whoever is doing the recruiting. It could be you, a recruiting company, your client, a salesperson, or someone else. Ideally, the researcher would also recruit the participants, because that person knows the most about who to recruit. But sometimes you have to rely on other people to do the recruiting, usually because they have a better connection to participants.

Knowledge

Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation) The first factor is the recruiter’s knowledge of the research and the type of people to recruit.

  • Does the recruiter know the characteristics of the people that are desired? Can the recruiter identify people who match those characteristics?
  • How well can the recruiter describe the research to the participants?

Without this knowledge, recruiters may select the wrong types of people or give them the wrong impression about the research. If you’re not doing the recruiting, it’s important to give the recruiter a clear definition of the types of people you’re looking for and a simple description of the research that they can use when contacting participants.

Skills

Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation)  Effective recruiting also depends on the recruiter’s interpersonal, communication, and organizational skills.

  • Is the recruiter personable and persuasive enough to convince people to participate?
  • Can the recruiter clearly and concisely communicate with the participants?
  • Is the researcher organized and coordinated enough to contact, schedule, and keep track of many participants?

Ensure that the person doing the recruiting has these skills. If not, have them delegate this task to someone who does. Administrative assistants often excel at recruiting because they are already well versed in communicating and coordinating schedules.

Recruiter Motivation

Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation) When finding participants becomes difficult, recruiter motivation is the most important criteria.

  • How much does the recruiter care about finding participants?
  • Will there be positive or negative consequences for the recruiter if participants are found or not found?

If you’re doing the research and the recruiting, you’re usually highly motivated to find participants, but when you have to rely on someone else, try to make sure that person has a stake in the recruiting. The negative consequences of not finding participants and derailing the project are highly motivating. Once a project starts and dates are set, there’s a ticking clock counting down the days to the research sessions. Delays in finding participants will delay the project.

With deadlines looming and money on the line, there’s a lot of pressure for whoever’s responsible to find participants. You can bet that most of the time, it gets done. Professional recruiting companies have a financial motivation to find participants, since they get paid a certain amount for each participant they recruit. However, if the recruiting turns out to be more difficult than they expected, they will begin to lose money as they spend more time finding each participant. When that happens, they’ll usually try to get you to loosen up the recruiting requirements to make it easier to find people, or they may say that they’ve exhausted their efforts and give up.

The Participants

The second part of the equation involves the recruiter’s relationship to the participants: Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation)

Connection

Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation) When people have a connection with the recruiter, they’re more likely to agree to participate. The strength of a connection can take various forms:

  • The participant may know the recruiter personally.
  • The participant may share a common affiliation with the recruiter. For example, they may be fellow employees or members of the same organization.
  • The participant may have an affiliation with an organization that the recruiter represents. For example, they may be a customer of the company the recruiter represents.

An email or phone call to participate in a study, often raises the questions about whether it’s legitimate, whether it’s worth considering, and whether it’s worth the time to participate. The closer the connection the person has with the recruiter, the more likely they will take the time to consider the request. Unless you already have this relationship, you might have someone else, who the potential participants know, do the recruiting. They’re more likely to listen to, consider, and respond to someone they know rather than a complete stranger.

Access

Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation) When you have access to lists of potential participants (such as employees, organization members, or customers) it’s much easier to recruit participants, especially when those lists contain characteristics that you can use to filter and narrow down those lists to the right types of people. It’s very difficult when you and your client have no access to the types of people you need. That’s when it makes sense to use a recruiting company.

Participant Motivation

Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation) Once they’ve been contacted and have considered the pitch, motivation determines whether a person will volunteer to participate. We can look at participant motivation as its own sub-equation: PM = A – D Participant Motivation = Advantages of participating – Disadvantages of participating To make it worth participating, the advantages have to be greater than the disadvantages. A > D = Participate A < D = Decline The advantages of participating can include things like:

  • Incentives – A $150 check, $50 Starbucks gift card, free software, etc.
  • Personal benefit from the improvements that come out of research – They’re going to improve this horrible software that I have to use every day at work!
  • Feelings of altruism
  • Feeling good about having your ideas heard and being able to improve something
  • Novelty of doing something different and interesting
  • Getting praise for participating and avoiding getting in trouble for not helping out

The disadvantages of participating can include things like:

  • The difficulty and effort it takes to participate
  • The time it takes to participate
  • Discomfort, anxiety, and fear of the unknown

Add it Up

Actually, something this complex can’t be distilled into an easy formula, but I think this equation provides a good visualization of the important factors. The quality and motivation of the recruiter, combined with the ease of finding participants, and the way people perceive the benefits they’ll receive versus the hassles they’ll have to go through, determines how easy or difficult it will be to recruit participants.

Recruiting Ease = ((Knowledge + Skills) x Recruiter Motivation) x ((Connection + Access) x Participant Motivation)

 

Mathematics image courtesy of Tom Brown

Are Consent Forms Always Necessary?

Are consent forms always necessary? We’re told that consent forms are an indispensable part of ethical user research. Consent forms are the vehicle to give and get informed consent – they inform the participants of what the study will entail and they allow the participant to indicate consent – with a signature and date.A consent form

Yet consent forms can conflict with the informal, friendly rapport that we try to establish with participants. Anything you present for people to sign immediately looks like a legal document or liability waiver. It puts them on guard.

That’s ironic because consent forms are the opposite of legal waivers. Legal documents are created to protect the interests of the company that creates them, while consent forms are created to protect the rights of the people signing them. Yet most participants assume they are signing a typical legal waiver.

Consent forms seem acceptable in more formal user research situations, such as usability testing and focus groups, but they seem odd and even off-putting when used in more informal situations. I’ve found them to be especially awkward when doing field studies at people’s offices. You strive to set up an informal situation, such as asking someone to show you how they create reports or asking them to try out a new design for an expense report application. But when you show up with a consent form for them to sign, it shatters the informal, comfortable rapport you tried so hard to establish. I’ve had people react to consent forms in this kind of situation with, “Hey! I thought we were just talking here.” How many times in the course of your work-life have you had someone show up to a meeting with a legal document for you to sign?

So I say use your judgment. When a consent form feels like it would be overly formal, don’t use it (unless your legal department requires it). Instead, get informed consent informally by email. “Inform” with your email describing what will take place, and get “consent” from their reply email agreeing to participate. At the start of the session, you can inform them again with a summary of what you’ll be doing. They will then give consent by continuing to participate in the session.

A good guideline is how comfortable or uncomfortable you feel when giving participants the consent form. If you feel uncomfortable, you’re probably breaking a group norm. So you should find a more acceptable way of getting informed consent.

User Experience to be Thankful For

The First Thanksgiving, painting by Jean Louis...

The First Thanksgiving, painting by Jean Louis Gerome Ferris (Photo credit: Wikipedia)

This Thanksgiving, before you carve the turkey, you may take a moment to think about the things you’re thankful for. But what about the things you’re thankful for in your work? After all, work takes up a large part of your life. If you’re a user experience geek like me, you may want to think about all the things that you’re thankful for in user experience. So let’s count our UX blessings:

The user experience of most products, websites, and applications is better than ever.

People are having a better experience than ever with most products, websites, and applications. Things are far from perfect, but we’ve made a big difference in improving the overall user experience over the last 20 years.

UX professionals are needed more than ever.

Although the overall user experience has definitely improved, it hasn’t improved enough that we’re not needed. There will always be a need for someone to focus on user research and designing the user experience. As we move from software and websites to mobile devices, wearable computing, and ubiquitous computing, there are many interesting challenges ahead.

We don’t have to do as much educating and selling UX as we used to.

More people than ever have a general idea of what user experience is and believe in its importance. They may not be able to define user experience, but they are primed to understand what it means with a little explanation. More people than ever know what usability (or at least “ease of us”) is. Even if they don’t know it by the term “user experience,” they can already sense it and value it. It’s easier to explain to people and we are less often challenged to prove its value.

There are more user experience jobs than ever before.

I don’t know this for sure, but it does seem like there are more user experience jobs than ever before, and it seems like that will only increase. There’s room in this field for generalists and specialists.

User experience is a very inclusive field.

User experience is still a very multidisciplined field. We welcome many different specialties and perspectives.

We don’t fight as much as we used to.

Compared to a few years ago, there doesn’t seem to be as much fighting about job titles, definitions, and which field or organization really represents the user experience.

User experience professionals are more connected than ever before.

We have more opportunities to connect with other UX professionals, either in-person or virtually, than ever before. The list of conferences and local events seems to continue to grow with more networking opportunities than ever before. With Twitter and LinkedIn, we can communicate and connect with people that we would never have met or heard from just five years ago.

There are more UX resources to learn from than ever before.

If you want to keep up to date and continue learning, there are more opportunities than ever before. There is a lot of great UX-related content out there, from web magazines, blogs, publications from UX organizations and conferences, and books. UX books have become shorter, more specialized, and aimed at experienced professionals.

We’ve never had better tools.

We’ve never had better tools for user research, prototyping, and design. Tools for remote usability testing, unmoderated testing, card sorting, tree testing, and eyetracking have expanded the type of research we can do. Prototyping and design tools make it easier than ever to create prototypes that we can evaluate with clients and test with users.

We do good work. We are part of a noble calling.

We’re lucky to be in a line of work where we solve interesting problems and help make people’s lives a little better. It may sound corny, but our work is a noble calling. We create better experiences for people. That may be as minor in the grand scheme of things as creating an easy, pleasant online shopping experience; or it may be as serious as preventing major safety errors that could endanger lives. Most of the time it’s somewhere in between, but it’s nice to know that to some degree we’re making the world a better place.

Effectively Communicating User Research Findings

I presented at UXPA 2013 today on Effectively Communicating User Research Findings.

This is the reason I’ve been way too busy to blog lately. I’ve been working non-stop on this presentation and also my UXmatters article published this week on Creating Better UX Research Videos: http://www.uxmatters.com/mt/archives/2013/07/creating-better-ux-research-videos.php

So check either of these out.

Quicken Extortion

Apparently Intuit has found a good way to get Quicken customers to upgrade to the latest version – extortion. Bloating the software with new features is no longer enough to get customers to upgrade, so instead they effectively disable the software by taking away critical features, such as the ability to download transactions and balances from your financial institutions.

I recently received the email shown below that my Quicken 2010 software was expiring and “critical services” will be discontinued. I can understand canceling support and help for an older piece of software, but downloading your financial information is a major part of using Quicken. It’s like a car company trying to get you to buy a new car by taking away your power steering after three years.

Planned obsolescence is pretty common, but it’s usually achieved these days by putting out a better product that makes the product you own pale in comparison. Quicken 2010 works just fine, and the 2013 version doesn’t seem to offer anything new and worthwhile. On Amazon, it gets an average of 2.8 stars out of 5, with most reviewers giving it one star.

If feature bloat no longer does the trick, how do they get people to upgrade? Extortion and monopolizing the personal finance software market. Maybe it’s time to use something else instead. Microsoft Money? Oh yeah, that’s gone now. Mint.com maybe? Sounds good, but guess who owns Mint now? Intuit.

Oh well, I guess all I can do is give in and upgrade to 2013. At least I’ll get a $20 off coupon and free shipping. How considerate!

Quicken2013

A Researcher Observing User Research

I had an interesting experience recently observing another user researcher conducting user research. I was overseeing a project with a more junior researcher who was actually doing the work. She was conducting contextual inquiries, and I was the secondary person on the team, simply observing her lead the session. The experience made me realize several things.

What you notice as an observer

It’s difficult to let someone else lead

Sitting back and simply observing instead of leading the session was a very odd experience for me. I’m used to taking the active role and leading the session. It was difficult to restrain myself from asking questions and taking over the session. It took me a while to relax and just let the session unfold. Once I did that, I realized that I noticed different things.

You notice different things

When you’re simply observing and you don’t have to facilitate, you’re able to observe more. Since you don’t have to focus on taking notes, assessing the information that you’re taking in, thinking of the next question to ask, and maintaining a positive rapport with the participant; you can focus more on the context in which the tasks are performed. Since I knew my colleague was recording the session and taking detailed notes on what the participant was doing and saying, I found myself taking more notes on the work environment, the participant’s desk, the things posted on the cubicle wall, and tools that the participant used.

You notice how well the research works

As an observer, you notice much more about how well the primary researcher facilitates the session, you see how the participant reacts to the situation, and you pick up tips for your own facilitating. You can sit back and focus on the interaction between the researcher and participant with a critical eye.

What is the role of a secondary researcher?

What can we learn from this, and what’s the role of a secondary researcher on field research? The secondary researcher:

  • Should stay out of the way and let the primary researcher lead the session
  • Can note additional questions and ask them at appropriate intervals, without throwing off the direction of the session
  • Can focus on the environment and contextual cues, taking detailed notes and asking specific questions about these elements
  • Can be the note-taker who takes detailed notes, when there won’t be enough time to listen to the recordings later
  • Can evaluate how well the primary researcher facilitates the session, using that knowledge to provide constructive feedback later and also to improve his/her own skills in facilitation

Two is the magic number of people who should attend field studies. One is the primary researcher. The other can be another researcher, but it’s often better to include a designer to allow him/her to see the research first hand. So in most cases, a second researcher is not the best person to attend research. Although when a second researcher does attend, use the techniques I’ve discussed in this post to make the most of the experience.

Online Ticket Ordering Pressure!

Login problems make online ticket ordering difficultThere’s probably no higher pressure experience for a consumer than ordering tickets for a popular event online. Once the tickets go on sale, you’re competing against thousands of other people for the best seats. Time is of the essence! Once you enter the number of tickets you want and the search results come back, those tickets are reserved for five minutes. A countdown clock starts ticking down the time to complete your order. If you don’t finish within that time, your tickets are released and you have to start over again with much worse seats.

I recently ordered tickets for a Foo Fighters concert on Comcast Tix, the vendor for the Wells Fargo Center in Philadelphia. They went on sale at 10am on a Monday, and I was one of the first into the site at 10am. I was excited to see that I had floor seats. I’ve never been so close to the stage before in a major concert.

As the clock started ticking down, the first screen asked me to enter my user name and password. I had ordered tickets from Comcast Tix before, but it was a long time ago. I don’t usually remember or take the time to write down passwords for sites that I use infrequently. I tried my standard password that I usually use in situations like this, but it didn’t work. I tried another variation, but that didn’t work either. Glancing at the countdown clock, I saw the time ticking away and started to sweat.

Maybe I had the wrong email address entered as the user name? I tried another email address. That didn’t work. I thought about going through a password reset, but by the time I got that, I’d lose the tickets.

Okay, maybe I’ll just try the option to complete the purchase with a new account. I filled out the form, but I got an error message saying that my email address was already in use as an existing account. Now I was really getting nervous. I only had two minutes left.

Finally, I hit upon the solution. I’d create a new account using my work email address. That worked! Now I had less than two minutes to race through the forms to complete my purchase or I’d lose the tickets. Glancing at the countdown clock as I raced to complete the forms, I felt like I was defusing a bomb. Finally I completed the transaction and got the tickets. Whew!

If ever there was a situation that needed foolproof usability, it’s the online ticket buying experience. I recognize the need to only hold tickets for a limited period of time, but the checkout process has to be extremely simple. An easy solution would be to give people the option to checkout without creating or logging into an account.

Especially when they charge you a $5.00 “convenience charge” and $2.75 “delivery charge” to print the tickets yourself online, they should offer a much better experience. That, or give you a $20.00 “poor usability” discount.

Netflix: It’s About the Format

Okay, it’s also about the format. In my last post, I wrote about how the choice between Netflix streaming and DVD was about the content. Netflix’s streaming content is mostly poor quality and quantity, while you can get almost anything you want by DVD. But format also makes a difference. If the content was the same in both formats, I think streaming would have a definite advantage.

The ability to instantly watch a movie from any internet-connected device is a great advantage of streaming, although many people don’t have a wired internet connection near their TVs and setting up a wireless network is beyond them.

Streaming provides the freedom to choose what you want to watch, when you want to watch. Although sometimes that freedom of choice can be paralyzing. When you have DVDs, you feel a commitment to watch them. They came in the mail, and even if you turn them off and send them back, it will be a while before you receive more. With streaming if you don’t like what you’re watching, you can stop and have a multitude of choices available. Sometimes I find the initial choice of what to watch difficult enough, but if a movie doesn’t grab me quickly, I end up second guessing myself and wondering whether I should have picked something else. Having so many choices instantly available seems to cheapen their value, whereas a DVD that I need to watch and return to get another, seems more urgent to watch.

DVDs require much more maintenance of your queue, strategically placing new releases at the top to get them sooner. What you watch tonight is based on a decision you made several days or weeks ago. We’ve all experienced that feeling of disappointment receiving a DVD other than the one we expected. The queue for streaming isn’t that important, except to prioritize which items you want to watch first and to save items that you might want to watch later.

The mailing schedule of DVDs can require strategic maneuvering. To make sure you have your new DVDs by the weekend, you have to make sure that the previous DVDs are returned by Wednesday. How many families have been annoyed by a slowpoke who didn’t watch a movie and return it in time to get another for the weekend? To be the first in line to get a new release that comes out on Tuesday, you can time the return of your DVDs on Friday or Saturday. Send them back too soon or too late and you may not get the new release.

With the decline of DVD sales, studios are beginning to become unfriendly to the rental market by creating rental-only copies stripped of special features and filling them with previews and ads that you can’t skip through. I recently watched a new release from Warner Brothers that had 17 minutes of non-skippable previews and ads, including several promoting the purchase of the Blue-ray version. Streaming, for now, contains no previews or ads, but the studios’ influence can be felt in the limited quantity of streaming titles and Netflix’s price increase to cover the increased cost of licensing content for streaming.

So it’s the format and the content. Streaming is a more efficient format, but the content needs to vastly improve before it will replace DVDs.

Netflix: It’s About the Content

Netflix Instant QueueThe recent customer uproar over Netflix raising their prices was a little overblown. It’s a business, they need to make money. With the expense of negotiating contracts with studios over streaming content, Netflix needed to adjust their pricing. I understand that, and that’s their right.

Although as customers, we have to decide what to do. Keep what we have and accept the price increase or make changes. I started with the 3 DVD at a time, unlimited plan about 5 years ago. When Netflix added their Watch Now streaming, I saw it as a nice bonus. Most of the time I had enough on my hands trying to keep up with the 3 DVDs. I only watched the streaming when I didn’t have a DVD to watch.

But for me it’s all about the content. I couldn’t care less about the mode of delivery. Netflix’s DVD library is huge. I can get almost anything I want, and 3 DVDs at a time is more than I can handle most times. In contrast, their streaming Watch Now content is full of movies and TV shows I’ve either already seen or never wanted to see. Also there’s a lot of just plain crappy movies.

Sure there are a few things I’ll miss. I just started to get into Mad Men now that it’s on streaming, but then again I can also get those on DVD.

So I’ve decided to cancel the streaming from my 3 DVD at a time plan, and just go with the unlimited 3 DVDs. It’s actually going to save me about $5 a month. Ironically, I think that’s the opposite of Netflix’s hope to wean customers off DVDs and onto streaming only.

If they really want to get streaming going, I think their subscription model is a good plan over pay per view streaming services. But it will never be worth it to me until they get a better streaming selection and new releases. Again, it’s about the content.

Follow

Get every new post delivered to your Inbox.