Usability evaluation methods deal with assessing the usability of a product. The main goal of evaluating usability is to define usability problems and obtain measures to make products more usable and pleasant to use. Since usability has already become a large part of the user experience (UX) and most of the methods can also be used to optimize more than just the usability, the following methods can also be called UX research methods.
Because of that, these methods can also be divided into qualitative and quantitative research.
Quantitative UX research is about putting numerical data to problems that can be modified to more usable statistics. Classical quantitative research methods include all types of surveys and polls. Although they started including a small amount of qualitative data, Google Analytics is also a big and important source for quantitative data, especially for digital products.
Qualitative UX research is about observations, user behavior, and the user’s perceptions. It involves collecting non-numerical data like text, audio, video, and images to understand the user experience in detail. This type of research is commonly used in all social sciences. Classical Qualitative research methods include interviews, observations, field studies, and also usability tests. According to Jakob Nielsen and William Bruce Cameron, you should also focus on qualitative research during UX projects because it often delivers more convincing results for less money.
“quantitative studies are usually too narrow to be fruitful and are oftentimes directly misleading.”– Jakob Nielsen
“Not everything that can be counted counts, and not everything that counts can be counted.”– William Bruce Cameron
Because there are a lot of different UX research methods and each method has its own unique advantages and disadvantages there is no list of best UX research methods. It always depends on the goal of your usability evaluation, the time and budget you have, the target audience of your product, the stage of the process you are in, and many more factors.
To get a better overview about which UX research you could use for your product, Susan Farrell created the UX research cheat sheet. It should not work as a checklist, it just shows which methods are available along the design process.
Usability Inspection Methods
Additionally to all the testing methods, there are also a number of different inspection methods available. Inspections are done by evaluators, usually usability professionals, and their main goal is to find usability problems in the design. Some inspections methods are also suitable for getting an overview about the usability of an entire system. Like the tests, these methods can also be performed early in the process of the project. Additionally to the following methods, you can also do a Heuristic Estimation but this method is not used very often.
The heuristic evaluation is the most informal inspection method. Usability professionals identify where a product follows predefined usability principles (heuristics) and where it doesn’t.
Heuristic evaluations are a quick and lower-cost way to measure the usability of a product before conducting user tests. They should be part of the process and used in conjunction with usability tests. According to a test conducted by Jedd Sauro, heuristics evaluations are able to detect between 30 and 50% of all usability issues, including all severe ones. Because this number also depends on the number of evaluators you should have about 3 to 5 professionals testing it to get the best value for your money.
The most well-known usability heuristics are based on the fifth chapter of Jakob Nielsen’s book “Usability Engineering” from 1993. Despite being nearly 30 years old, these guidelines are well established and still up-to-date. Jakob Nielsen has even slightly reworded them and added some newer examples last year.
But during a heuristic evaluation, a professional evaluator might also use different and more specific heuristic or additional product-related ones. Arnie Lund, Bruce Tognazzini, and Ben Shneiderman have also defined additional sets of heuristics. For modern apps, you can also use Apple’s Human Interface Guideline or Google’s Material Design Guidelines as Usability Heuristics.
Arniel Lunds “Expert Ratings of Usability Maxims”
Bruce Tognazzini’s “First Principles of Interaction Design”
Ben Shneiderman’s “Eight Golden Rules of Interface Design”
The process of a heuristic evaluation is divided into three sections: Planning, Executing, and Reviewing.
Since it is a usability inspection method with clear objectives, you should plan your desired outcome in advance. Setting goals and understanding exactly what needs to be evaluated is one of the most important parts of the planning phase. The evaluators must also be aware of the demographics, needs, motivations, and behaviors of the target audience. Although there are no real users present during the evaluation, the usability professionals should always keep the users and the situation the product is used in mind. During the planning phase, it is also important to choose the number of evaluators – in this case ideally between 3 and 5 to get the most results for as little money as possible. Additionally, you should also define the set of heuristics you want to use for the inspection.
During this phase, the usability professionals are actually doing the inspection of the interface. They will go through the user flow and analyze all pages based on the previously defined heuristics. This part of the inspection is normally recorded to get the best result out of the evaluation and be able to see exactly where the problems are, what the attempted tasks were, why it is a problem and usually they are also suggesting different ways on how to fix the problems.
After the inspection, it is important that the experts are creating a list with all usability issues and additional findings that should be addressed. Ideally, these usability problems should also be rated and prioritized based on how severe they are. These results are normally presented in the form of a report that describes the process of the inspection, all usability problems, and different suggestions on how to fix them. This report, together with the recordings, should also be the basis for the next steps the project team is going to make.
During a cognitive walkthrough, one or more evaluators work through some tasks and a set of questions from the perspective of the user. The goal is to simulate the user’s problem-solving process. It was originally designed for ticket machines, ATMs, postal kiosks, or other interactive systems where users have little or no training before using the system. Currently, it is mainly used for evaluating the learnability of a system for new and occasional users.
A pluralistic walkthrough is a group evaluation of a design where users are guided through tasks while designers and developers address their concerns or questions about the interface. This method works best in the very early stages of the design process with a paper prototype or rough wireframes.
Consistency inspections are usually done by designers on larger projects. It is used to ensure that the interface does things the same on all pages. The ultimate goal of the consistency inspection is that it looks like one designer did the whole project and every element that does the same thing also looks and feels the same.
The feature inspection emphasizes the importance of functionality in order to achieve good usability. It identifies critical tasks a user would perform while using a system and the features that would be used to perform these tasks. After that process, the features are then evaluated.
For a standard inspection, you need an expert on an interface standard that has to inspect the whole interface for compliance with the standards. Good examples for standard inspections of apps include checking if the Apple Human Interface Guidelines or Google Material Design Guidelines are met. Another important standards inspection determines if the product complies with all the ISO standards.
Apple Human Interface Guidelines
Google Material Design Guidelines
Formal usability inspection
A formal usability inspection combines different elements from the heuristic evaluation and from the cognitive walkthrough. It is a process with six steps and clearly defined roles for every expert on the team.
A competitive review is a process where a usability expert analyzes a series of related websites or products from competitors or other companies from the same sector. The expert tries to look for strengths, weaknesses, trends and differences and identifies room for improvement for his/her own product. A competitive review is also possible with just a small portion of a website. For example if you are building a webshop you can take a closer look at amazon and their ordering process.
The Elements of Successful UX Design, Best Practises for Meaningful Products