User Testing: Reconciling What People Say and What They Do

For web designers, user testing (also known as usability testing) is an awesome opportunity to listen to your customers’ voices and observe how they use your website. During the Discovery phase of a project, we can see how actual people utilize the user interface to accomplish what they need to do, which can help us resolve many of the questions and uncertainties that arise during the design and development of the website.

However, while all of the users’ feedback is valuable, user testing is not as simple as just listening to people’s comments and watching their behaviors. Many times as a User Experience (UX) designer, I have found that what users say they do when describing how they use a website can be surprisingly different from what they actually do. Since we want to understand as much as possible about how people use the sites we design, we need to understand why these discrepancies arise and how to get the best feedback about how people use the site in order to solve their issues and create the best possible experience for them.

Where Do These Discrepancies Come From?

We’ve found that people censor themselves when describing how they use a website, since they don’t want to speak for everyone. They know they only represent a small portion of the audience, so they feel that their struggles might not be reflective of how others use the site. In addition, they sometimes blame themselves, thinking that any problems they experience are their own fault.

Politeness often plays a factor in how people answer questions about their use of a website. When queried about whether they use a certain feature, users often give an answer like “I don’t use it, but maybe someone else finds it helpful.” We call this type of answer a “polite no.”

Users also often complete specialized tasks which, since they are performed regularly, become nearly automatic for the person performing them. In these cases, the users often don’t think about each specific step, so when they are describing how they perform these tasks, they’ll probably omit details. Watching them actually complete these tasks will reveal each of the steps they take and help designers understand exactly how people use the website.

Getting the Best Feedback

While observing how people actually use a website is essential, our experience in user testing has helped us find ways to narrow the gaps between what people say and what they do. Here are some tips for how to get the best insights from users, whether they are answering questions or demonstrating how they use a site:

• Don’t just ask about general likes and dislikes. We want to learn about the specific ways people use the interfaces we design, so try to discuss real scenarios and tasks that they complete and watch the steps they take to accomplish what they need to do.
• Look for where they struggle. The main complaints and pain points of a website are often obvious, so what we want to get out of user testing is a personal look at the issues people have with the site. The smaller, less-obvious areas can help us find things that we might not have considered, since we’re not the people using the website every day.
• Keep it personal. Focus on the user and how they specifically use the site, rather than discussing general ideas about the entire organization. It helps to consider the context of the tasks they are trying to accomplish. Asking why they are doing something, how often they do it, and what exactly they need to get out of it helps us understand how to provide the best experience for them. We also try to push the users to be really honest and make sure to note anything they do that might be outside the norm.

When designing user interfaces, we understand what a website needs to accomplish from a high level and how the basic “flow” should work, but user testing lets us look at the small details and better understand what exactly people need from the design. Seeing the unique steps that people take that we might not have anticipated or the workarounds that they come up with to get past any issues that they’ve found gives us the insight we need to create the designs that meet their needs. These small details can make all the difference between a design that is functional but difficult to use and one that makes people’s lives easier and more efficient.

Do you have any questions about how to perform user testing in a way that reveals the best insights about how people use your website? Do you want to know more about how you can leverage this information to improve your organization’s efficiency? Please contact us to speak to a User Experience expert, or leave a comment below with any other questions you might have.