Five things I learned from usability testing

Summary: I love doing usability testing. There is no better way to test your assumptions than by putting them in front of your users. Not only can you see your work outside the development environment, but you can also get a lot of innovative ideas from users because they use the system every day. You have to arrange this as soon as possible, but surprisingly many developers don't.


I love doing usability testing.

There is no better way to test your assumptions than by putting them in front of your users. Not only can you see your work outside the development environment, but you can also get a lot of innovative ideas from users because they use the system every day.

You have to arrange this as soon as possible, but surprisingly many developers don't. They should spend less time developing and more time communicating with users. In other words, they should go out more.

I also learned how to get more effective feedback. If you focus on specific patterns, then you can improve your ability to spot hidden ideas.

It's best to test in an ambient context. The first few usability tests

I've worked on are what marketers like to do: the moderator sits on one side, and five or six other people sit on the other side. Instead, we should sit on the other side and take notes, watching users’ performances and interacting with them regularly while joking.

It's interesting to watch how users perform. But sometimes it's totally useless. All usability testing is biased, and more seriously, even if you pay your users, you can guarantee that their performance is real? So, your goal is to limit this bias as much as possible. We can learn a lot in the process, which is why we keep testing, with or without bias.

The best way is to simulate the environment in which the user is using the application. For example, I wouldn't test a mobile map app if the user is sitting, because the user typically uses the map while walking to find a place. I've done tests with users on handheld phones, and the metric is "Have they found their destination?"

For other applications, I'll do usability tests with users at my desk. This allows for a better understanding of the context in which users use the app in their daily lives.

A client and I recently did a test where the user has to interact with 3-4 apps in their daily life. I can learn about other applications that users use (such as Salesforce, Excel, Outlook, and six other browser windows). I take notes as I watch user interactions with the software and make a list of app-related workflow suggestions instead of what I originally designed.

Watch what they do, not what they say

Many people like to help others, so they will actively face the problems you put them in front of them, a condition known as Social Desirability. I've run several usability tests and users have expressed how they like the app. They'll say, "I love it! I'll totally use it."

Obviously, they don't know how to use the app.

When you call it "usability testing", end users may simply think of it as a test of knowledge, not a usability test of the app. They don't want people to think they're stupid, so they'll say how well they know how to use the site or how dedicated they are to it so they don't make mistakes. A lot of people like to help, so they are very positive about the problems you put in front of them. Based on that alone, I might have a bunch of stuff to write about.

Watch what they do and how they react to the app on the screen and try to connect with their comments. Sometimes, I record mouse movements to know exactly what the user is doing, but I usually only record the interactions roughly.

Getting users to talk is easier than you think I've interviewed many recruiters and hiring managers while working

at Jobvite. There's nothing easier than getting feedback from a recruiter.

You ask them three questions and they will tell you everything. What they ate for lunch, who they interviewed, whether they liked their job, etc. More importantly, if you guide them, they will show you how to use technology. Recruiters talk a lot every day, but no one listens to them. They'll be happy to talk when they get the chance.

They, like most people, want someone to listen because we are an open society. As designers and developers, we give them a chance to have a conversation, and we give us a chance to get real opinions. It's not just because they use the technology you're testing, their environment fits too.

All you need to do is ask why?

This feedback is very important for designing great products, because you understand your users' pain points and you can solve their problems.

The best ideas come from the off-script process.

I've seen usability tests like this: the moderator lists a series of questions, takes notes, easily obtains worthless feedback, and may not even know what to talk about.

I've seen this type of usability testing, too: the moderator only lists a few short questions and asks the user to talk about a few important points and follow the feedback on product changes.

The best case scenario is to have a loose script so you can think from your own perspective. If you learn how to do it, the test will have more occasional problem solving. That’s where the real value content comes in, because you can dig into a topic and get what users really think.

No need

to Before testing, I tell users that this is an idea in a test trial. I'll also tell them it's a prototype, so something might not work. Because prototypes are often so realistic, users forget that this is a test.

I send them a link. They crash as soon as they click.

Why does it crash?

Because this is a prototype, but it's not complete. We are trying other methods. So when you click on this link, what do you think should happen? Can you explain it step by step?

In most cases, they describe what to do next. Of course, it would be cool if you could be sure of what users were really thinking right as they described each step. Even better, they might give you an idea you never thought of.

Usability testing doesn't just test your current design, it uses the collective experience of users to improve it. Not only do these ideas validate what you're doing now, but they may also make their way into your product pipeline.

Did you learn from users?

hurry up.

Ask why.

listen.

It's not difficult, but you need to keep doing it every day, no matter when. What you learn may surprise you.

Here is a template. Go test it now.