Steps to Conducting a Software Usability Test

Stakeholders have identified the need for a new software application. Business analysts have studied business practices and workflows, defined requirements, and requested functionality. Development teams have created prototypes and wireframes and coded a masterpiece that checks every box. But can real users use your product to accomplish real goals?

Software usability testing makes good software great by valuing the user experience it provides. 

Deciding Between Moderated or Unmoderated Usability Testing

Moderated usability testing is highly structured and provides detailed results with rich, qualitative insights. By utilizing the skills of a professional researcher, participants can be guided through tasks and asked spontaneous follow-up questions based on their actions and reactions.

However, it should be understood that the researcher could unintentionally influence the outcomes of certain tasks if users feel rushed, intimidated, or judged whereas other users may feel comforted by having real-time guidance. In addition, moderated usability testing can be expensive and time-consuming.

Comparatively, unmoderated usability testing is done without any supervision. Participants are able complete tasks in their own environment using their own tools.

Unmoderated usability testing can also be conducted quickly and with a lower budget. 

How to choose

Moderated usability testing is critical when tasks are complex or need to be completed in their entirety for your results to be meaningful.

Unmoderated usability testing is best for large-sample studies or when it’s valuable to know how your software application performs under uncertain conditions using a variety of devices and configurations. 

Conducting Usability Testing Remotely vs. In-Person

Current COVID-19 pandemic aside, remote usability testing offers considerable advantages when being physically present is prohibited by cost or logistics. By utilizing the internet and telephone, you can easily test large numbers of people from across any number of demographics.

Unfortunately, remote testing requires a few sacrifices. Researchers will miss the opportunity to judge body language and facial expressions. Even when using video conferencing, it can be very difficult to pick up on subtle behaviors and feelings.

The best approach is to employ a hybrid model. Consider conducting a smaller, in-person usability test, then supplementing this with remote follow-up or targeted tests.

For instance, if your in-person testing reveals a certain type of user struggles with a particular task, modify your software application and retest remotely with new participants that meet the requisite criteria. 

Choose a Testing Method that Delivers the Information You Need

Usability testing can take many forms and occur at various points in the software development and support lifecycle. Each testing method will generate different types of information. 

Exploratory

Exploratory usability testing provides a method for evaluating early stage design ideas. Participants are given ideas and concepts, then provided with the opportunity to brainstorm, offer opinions, and express emotions. This method of testing can be especially valuable when trying to identify wishlist items and pain points within existing software applications or business practices, ultimately helping to choose where to focus and prioritize new development efforts.

Be careful when using exploratory testing in groups as participants may influence each other or impede the free flow of thoughts.

Assessment

Assessment usability testing provides answers to specific measurable questions about a software application, such as whether users can accomplish given tasks and achieve certain goals, whether they are satisfied by the product, and whether there are any accessibility barriers.

Comparative

Comparative usability testing forces participants to make choices and state preferences. This or that? Us or them? Which is more inviting? Which menu label is easier to understand? Which font is easier to read?

Though many feel evaluating participants in this manner cannot truly be thought of as usability testing, the comparative method is ideal for presenting remediation options for known issues. 

Also read: Inclusive Software Design Best Practices

How Do You Recruit Participants?

Conducting effective user research depends on finding the best suited participants. A few strategies can help to make recruiting these people a little easier.

• Focus on participants that represent your target audience and potential users. If you are designing a software application that will be used by financial analysts, it likely doesn’t matter whether kindergarten teachers find it easy to use.
• Be aware of potential bias. While it may be important to ensure participants have the requisite industry-specific knowledge or background necessary to test your product, be careful that your group includes those with various roles and responsibilities. Don’t forget, many employees may not feel free to criticize a product if they fear repercussions, so do what you can to reassure (and protect) these individuals.
• Be sure rewards offered aren’t seen as conditional. While it is customary to compensate skilled professionals for their time or offer rewards as signs of appreciation, be sure participants understand that these benefits are given regardless of the testing outcomes.
• Don’t forget diversity. Look for participants that can provide insight into how accessible and inclusive your product is.
• Use a series of consistent questions during the screening process. Do you need participants within a certain age range? Do you need to confirm that participants have a particular level or type of education? Does income level matter? Has someone worked for one of your competitors? What are their hobbies? Do they speak multiple languages? While some of the answers to these types of questions may exclude a potential participant, they may also serve as ice breakers or provide logical ways to create groupings of similar participants for more targeted usability testing activities.

Be sure all potential participants feel valued and be prepared to communicate inclusion criteria, so it can be understood why they may not have been chosen. 

Creating Tasks for Usability Testing

Usability testing tasks provide participants with assignments that will require a series of actions and activities to complete.

Create quality usability testing tasks and avoid common mistakes by following these six guidelines:

1. Don’t confuse the purpose of your software with the tasks you are giving participants. You may be developing a comprehensive bookkeeping software application, but the task being assigned for testing may be to ‘Add a new client.’
2. Start with a task scenario. Participants are generally coming into your usability test with little to no information. Do not assume they can intuit backstory or context. That said, avoid giving more information than is reasonably required.
3. Make tasks realistic and actionable. Participants are testing your product, not your concept. Avoid creating tasks that encourage conversation instead of action. (Good action verbs include download, buy, subscribe, find, click, complete, sign up, log in, invite, create, and share.)
4. Avoid telling your participants what to do or where to go. Remember that, in the real world, users of your product won’t have a task in front of them offering clues or describing the steps. Avoid phrases like “click on” or “then, choose.” Although, if your product has a natural workflow or progression, it is okay to arrange a series of tasks in a sequence.
5. Be sensitive and kind to your participants. Review each of your tasks to ensure they will not invoke unnecessary emotion or cause offense. When necessary, use harmless relationships as references, such as “friend” or “colleague,” and eliminate wording that communicates judgment, like “exercising to lose weight.”
6. Verify every task ahead of time. Make sure your tasks can actually be completed using the version and configuration of the software application being provided to participants.

No matter the tasks, it’s critical your participants understand the product is being tested, not the users.                            

Also read: Using Journey Maps to Understand Your Software Users

Conducting Your Software Usability Test And Recording Results

Unfortunately, this isn’t a one-size-fits-all activity. A lot will depend on how you’ve chosen to perform your software usability test, but there are a few tips and tricks that should work for you regardless of those details:

• Identify tasks that record their own data and don’t worry about them during your usability test. You should know if users were able to create accounts on your system if there are the same number of new accounts as test participants. If you will not have a moderator or observer present during the test, ensure all of your tasks leave this kind of information trail to review later.
• Start your usability test with an introduction and communicate your expectations. If you need users to record their actions on a data sheet, be sure to provide it and walk them through how to fill in the details and submit it for review. Ideally, this should be a stream of consciousness type activity, so it is recommended that you discourage participants from erasing anything they write. 
• When possible, take audio and video recordings of your usability test (always obtain consent ahead of time, of course).
• Always make participants feel valuable and appreciated. Tell them when things they do or say are helpful, and always thank them for their time.
• If participants offer narration while completing tasks, ask for clarification when needed, and consider paraphrasing your understanding to be sure it is correct.
• Finish your usability test by giving participants the opportunity to ask follow-up questions and provide any final feedback they may have. 

Evaluating Usability Test Results

After you’ve conducted your usability test, it’s advisable to review the data you collected as soon as possible. As much as you or your participants may try to take comprehensive notes, it’s likely you will see details like “clicked on the wrong link.” It’s much more likely you will remember that “clicked on the wrong link” meant “clicked on x link instead of y,” which provides significantly more insight.

Everybody will collate their results in a different way, but what’s most important is that the method you choose makes it easy to look for patterns. Identify problems that happened once and separate them from those that were more frequent. Highlight problems that sabotaged tasks, causing complete failure.

You may also need to evaluate the same results data in multiple ways or using different groupings.

When prioritizing the issues to be remediated and retested, look for quick wins that check multiple boxes. For example, a fix to the layout of your user interface may also improve accessibility for users with visual impairments.

Software Usability Testing Is An Investment

There isn’t a software application out there that is too efficient, too easy to use, and perfect just as it is. By leveraging software usability testing, you have the opportunity to better understand how actual users feel and think when using your product, while ensuring their needs are being understood and met.

Read next: Using Prototyping to Accelerate Software Development