Skip to content

A Researcher Observing User Research

I had an interesting experience recently observing another user researcher conducting user research. I was overseeing a project with a more junior researcher who was actually doing the work. She was conducting contextual inquiries, and I was the secondary person on the team, simply observing her lead the session. The experience made me realize several things.

What you notice as an observer

It’s difficult to let someone else lead

Sitting back and simply observing instead of leading the session was a very odd experience for me. I’m used to taking the active role and leading the session. It was difficult to restrain myself from asking questions and taking over the session. It took me a while to relax and just let the session unfold. Once I did that, I realized that I noticed different things.

You notice different things

When you’re simply observing and you don’t have to facilitate, you’re able to observe more. Since you don’t have to focus on taking notes, assessing the information that you’re taking in, thinking of the next question to ask, and maintaining a positive rapport with the participant; you can focus more on the context in which the tasks are performed. Since I knew my colleague was recording the session and taking detailed notes on what the participant was doing and saying, I found myself taking more notes on the work environment, the participant’s desk, the things posted on the cubicle wall, and tools that the participant used.

You notice how well the research works

As an observer, you notice much more about how well the primary researcher facilitates the session, you see how the participant reacts to the situation, and you pick up tips for your own facilitating. You can sit back and focus on the interaction between the researcher and participant with a critical eye.

What is the role of a secondary researcher?

What can we learn from this, and what’s the role of a secondary researcher on field research? The secondary researcher:

  • Should stay out of the way and let the primary researcher lead the session
  • Can note additional questions and ask them at appropriate intervals, without throwing off the direction of the session
  • Can focus on the environment and contextual cues, taking detailed notes and asking specific questions about these elements
  • Can be the note-taker who takes detailed notes, when there won’t be enough time to listen to the recordings later
  • Can evaluate how well the primary researcher facilitates the session, using that knowledge to provide constructive feedback later and also to improve his/her own skills in facilitation

Two is the magic number of people who should attend field studies. One is the primary researcher. The other can be another researcher, but it’s often better to include a designer to allow him/her to see the research first hand. So in most cases, a second researcher is not the best person to attend research. Although when a second researcher does attend, use the techniques I’ve discussed in this post to make the most of the experience.

Online Ticket Ordering Pressure!

Login problems make online ticket ordering difficultThere’s probably no higher pressure experience for a consumer than ordering tickets for a popular event online. Once the tickets go on sale, you’re competing against thousands of other people for the best seats. Time is of the essence! Once you enter the number of tickets you want and the search results come back, those tickets are reserved for five minutes. A countdown clock starts ticking down the time to complete your order. If you don’t finish within that time, your tickets are released and you have to start over again with much worse seats.

I recently ordered tickets for a Foo Fighters concert on Comcast Tix, the vendor for the Wells Fargo Center in Philadelphia. They went on sale at 10am on a Monday, and I was one of the first into the site at 10am. I was excited to see that I had floor seats. I’ve never been so close to the stage before in a major concert.

As the clock started ticking down, the first screen asked me to enter my user name and password. I had ordered tickets from Comcast Tix before, but it was a long time ago. I don’t usually remember or take the time to write down passwords for sites that I use infrequently. I tried my standard password that I usually use in situations like this, but it didn’t work. I tried another variation, but that didn’t work either. Glancing at the countdown clock, I saw the time ticking away and started to sweat.

Maybe I had the wrong email address entered as the user name? I tried another email address. That didn’t work. I thought about going through a password reset, but by the time I got that, I’d lose the tickets.

Okay, maybe I’ll just try the option to complete the purchase with a new account. I filled out the form, but I got an error message saying that my email address was already in use as an existing account. Now I was really getting nervous. I only had two minutes left.

Finally, I hit upon the solution. I’d create a new account using my work email address. That worked! Now I had less than two minutes to race through the forms to complete my purchase or I’d lose the tickets. Glancing at the countdown clock as I raced to complete the forms, I felt like I was defusing a bomb. Finally I completed the transaction and got the tickets. Whew!

If ever there was a situation that needed foolproof usability, it’s the online ticket buying experience. I recognize the need to only hold tickets for a limited period of time, but the checkout process has to be extremely simple. An easy solution would be to give people the option to checkout without creating or logging into an account.

Especially when they charge you a $5.00 “convenience charge” and $2.75 “delivery charge” to print the tickets yourself online, they should offer a much better experience. That, or give you a $20.00 “poor usability” discount.

Netflix: It’s About the Format

Okay, it’s also about the format. In my last post, I wrote about how the choice between Netflix streaming and DVD was about the content. Netflix’s streaming content is mostly poor quality and quantity, while you can get almost anything you want by DVD. But format also makes a difference. If the content was the same in both formats, I think streaming would have a definite advantage.

The ability to instantly watch a movie from any internet-connected device is a great advantage of streaming, although many people don’t have a wired internet connection near their TVs and setting up a wireless network is beyond them.

Streaming provides the freedom to choose what you want to watch, when you want to watch. Although sometimes that freedom of choice can be paralyzing. When you have DVDs, you feel a commitment to watch them. They came in the mail, and even if you turn them off and send them back, it will be a while before you receive more. With streaming if you don’t like what you’re watching, you can stop and have a multitude of choices available. Sometimes I find the initial choice of what to watch difficult enough, but if a movie doesn’t grab me quickly, I end up second guessing myself and wondering whether I should have picked something else. Having so many choices instantly available seems to cheapen their value, whereas a DVD that I need to watch and return to get another, seems more urgent to watch.

DVDs require much more maintenance of your queue, strategically placing new releases at the top to get them sooner. What you watch tonight is based on a decision you made several days or weeks ago. We’ve all experienced that feeling of disappointment receiving a DVD other than the one we expected. The queue for streaming isn’t that important, except to prioritize which items you want to watch first and to save items that you might want to watch later.

The mailing schedule of DVDs can require strategic maneuvering. To make sure you have your new DVDs by the weekend, you have to make sure that the previous DVDs are returned by Wednesday. How many families have been annoyed by a slowpoke who didn’t watch a movie and return it in time to get another for the weekend? To be the first in line to get a new release that comes out on Tuesday, you can time the return of your DVDs on Friday or Saturday. Send them back too soon or too late and you may not get the new release.

With the decline of DVD sales, studios are beginning to become unfriendly to the rental market by creating rental-only copies stripped of special features and filling them with previews and ads that you can’t skip through. I recently watched a new release from Warner Brothers that had 17 minutes of non-skippable previews and ads, including several promoting the purchase of the Blue-ray version. Streaming, for now, contains no previews or ads, but the studios’ influence can be felt in the limited quantity of streaming titles and Netflix’s price increase to cover the increased cost of licensing content for streaming.

So it’s the format and the content. Streaming is a more efficient format, but the content needs to vastly improve before it will replace DVDs.

Netflix: It’s About the Content

Netflix Instant QueueThe recent customer uproar over Netflix raising their prices was a little overblown. It’s a business, they need to make money. With the expense of negotiating contracts with studios over streaming content, Netflix needed to adjust their pricing. I understand that, and that’s their right.

Although as customers, we have to decide what to do. Keep what we have and accept the price increase or make changes. I started with the 3 DVD at a time, unlimited plan about 5 years ago. When Netflix added their Watch Now streaming, I saw it as a nice bonus. Most of the time I had enough on my hands trying to keep up with the 3 DVDs. I only watched the streaming when I didn’t have a DVD to watch.

But for me it’s all about the content. I couldn’t care less about the mode of delivery. Netflix’s DVD library is huge. I can get almost anything I want, and 3 DVDs at a time is more than I can handle most times. In contrast, their streaming Watch Now content is full of movies and TV shows I’ve either already seen or never wanted to see. Also there’s a lot of just plain crappy movies.

Sure there are a few things I’ll miss. I just started to get into Mad Men now that it’s on streaming, but then again I can also get those on DVD.

So I’ve decided to cancel the streaming from my 3 DVD at a time plan, and just go with the unlimited 3 DVDs. It’s actually going to save me about $5 a month. Ironically, I think that’s the opposite of Netflix’s hope to wean customers off DVDs and onto streaming only.

If they really want to get streaming going, I think their subscription model is a good plan over pay per view streaming services. But it will never be worth it to me until they get a better streaming selection and new releases. Again, it’s about the content.

Another User Experience Blog?

Another user experience guy with another user experience blog. Why read this one? Well, hopefully I’ll have some interesting things to write about, and this introductory post will be the least interesting of the lot. Who reads “welcome to my blog” posts anyway?

I don’t think most people read blogs regularly. They read interesting content when they come across it, regardless of where it comes from.

Why did I start a blog? Yes, part of it was for shameless self-promotion, but also as an outlet for my thoughts. For the last two years I’ve been writing a column for UXmatters, called Practical Usability. I publish articles in that column every two months, which seems like a good schedule for me. But I have a lot of ideas that don’t fit the format of a 2000 – 3000 word article. Those ideas tend to languish on a list that never gets published. Until now.

The topics will focus on user experience, user research, usability, design, and other related things in that vein. I hope you’ll find them interesting. At least much more interesting than this first post. So, welcome to my blog!