Usability metrics are commonly used to understand the usability of software, product, and other applications. These metrics can help pinpoint issues with the system you are evaluating, metrics can be used to compare products or issues, or metrics can be a quick way to address usability issues.  

Post-task assessments are one specific type of usability metric. These questionnaires are administered immediately after a user completes a given task. They are typically 1-3 questions in length and often take the form of Likert scale ratings. The goal is to provide insight into the task difficulty, based on the participant’s perspective. In this blog, we share four examples of post-task metrics and provide advantages and disadvantages to using each.  
 

SEQ (Single Ease Question) 

Creator: Jeff Sauro 

Resources: 10 Things to Know About the Single Ease Question (SEQ) 

Details: The SEQ measures how difficult users find a task to be. It is administered after every task in a session and involves only one question, which asks the user to rate the difficulty of the task from very easy to very difficult on a 7-point scale (see below).  

Pros: Since this tool is a single question, it is efficient and minimally disruptive to the test flow. Additionally, the administration is versatile – it can be administered on paper, electronically, on a web service, or verbally. It also gleans fresh and uninfluenced impressions, as it is administered immediately post-task. This tool can be helpful when comparing which features or workflows are perceived as the most problematic. In order to get additional insights from this tool, it can be helpful to ask “why” after any rating of 4 or lower. 

Cons: There is not a lot of data available for comparing SEQ results across companies, so the major limitation of this tool is that it is mainly restricted to comparing tasks within your own system or organization.

SMEQ (Subjective Mental Effort Questionnaire)

Creators: Fred R. H. Zijlstra and L. Van Doorn 

Resources: Comparison of Three One-Question, Post-Task Usability Questionnaires 

Details: The SMEQ is one question that measures the mental effort that people feel was involved in a task. Its scale has nine labels from “Not at all hard to do” to “Tremendously hard to do”, which are shown as millimeters above a baseline from 0-150. The upper bound is very open-ended, leaving considerable space above “Tremendously hard to do”. The question can be administered on paper or online, and participants either draw or drag a line through the scale to indicate how much mental effort they had to invest to complete the task.  

Pros: The SMEQ is particularly helpful when trying to measure and regulate the cognitive load required for a task. Like SEQ, SMEQ is quick and easy to administer. The near-continuous number of response choices on the scale is also an advantage, as user sentiments are more precise with more choices to select from. 

Cons: The scale may be confusing initially for both participants and researchers, as it is not a uniform, close-ended scale like we are typically used to seeing. 

ASQ (After-Scenario Questionnaire) 

Creator: J.R. Lewis 

Resources: Psychometric evaluation of an after-scenario questionnaire for computer usability studies: the ASQ 

Details: The ASQ includes 3 questions that assess the perceived level of difficulty of a task. The 3 ratings are based on the ease of the task, the time taken to complete the task, and the level of support received in order to complete the task (see below). The score is calculated by averaging the 3 responses.

Pros: The ASQ is short and simple to administer. Determining your ASQ score is straightforward, only requiring averaging the responses received. 

Cons: When used in its most basic form, ASQ lacks the context (or the “why”) behind the scores. It could be beneficial to supplement the ASQ ratings with asking participants why they gave each score.  

NASA-TLX (NASA Task Load Index) 

Creator: NASA Ames Research Center's (ARC) Sandra Hart 

Resources: NASA-TLX Website 

Details: NASA-TLX involves 6 questions on an unlabeled 21-point scale, ranging from very low to very high. Each question addresses one dimension of perceived workload: mental demand, physical demand, time pressure, perceived success, overall effort, and frustration. Participants weigh each category to indicate which mattered most throughout their tasks

Pros: This tool is most useful for complex products and tasks in high-consequence environments, like healthcare, transportation, aerospace, the military, or complex financial domains. It would be less useful in studies of consumer products or simple workflows. There are many industry benchmarks available for the fields in which this tool can be applied. 

Cons: There are some challenges associated with administering this assessment. First, as it is 6 questions long, it is time-consuming, creating the potential for participant fatigue and disruption of the test flow. Since it a relatively complicated metric, it also often requires many explanations or clarifications from the facilitator.  
 

Expectation Measure 

Creator: William Albert and Eleri Dixon 

Resources: Expectation Measures in Usability Testing 

Details: Expectation Measure revolves around the idea that an expectation is the anticipation of a future experience based on prior knowledge. The use of the Expectation Measure in UX helps determine the relationship between customer expectation and satisfaction. Expectation Measures can inform design strategy by focusing attention on usability issues that negatively impact user satisfaction, as well as increase satisfaction through positive usability experiences. The expectation data is collected prior to any user interaction via a 5 or 7 point Likert scale. The experience rating is collected after each task based on the same Likert scale. This data is analyzed and plotted. The tasks are classified into four groups: 

Don’t Touch It – These are easy to complete tasks. These tasks have no opportunity to improve user satisfaction. 

Fix-It Fast – These tasks are expected to be easy but are difficult to complete. These tasks should be given immediate attention. 

Promote It – These tasks are expected to be difficult but are easy to complete. These tasks have the opportunity to increase user satisfaction

Big Opportunity – These tasks are expected to be difficult and are difficult to complete. These tasks are thought to be big opportunities to improve user satisfaction, but overall are less harmful. 

Pros: The ability for the Expectation Measure to identify key usability problems that may be driving down user satisfaction. This leads to focus and a more complete picture of the relationship between usability and satisfaction. 

Cons: The reliability of Expectations Measures can get tricky. Some participants have a wide range of confidence level or confuse expectations, this can impact the results. Additionally, Expectations Measures should be used as a complement to other metrics and testing methods.  
 

Conclusion:  

Utilizing post-task assessments enables us to better understand and quantify the level of difficulty of a given task according to users. This can lead to an improved product, service, or experience design and ultimately improve the overall user experience.