Published by Katharine Betteridge.

The SUS score is the metric that has become the industry standard for measuring the usability of websites and products. As the SUS score is a critical piece of information we give to many clients, I wanted to dig into its history to understand it a bit more. I’ve shared my findings below to help future clients better leverage the SUS score we give them to make data-informed decisions about UX direction. When used correctly, the SUS score can be a powerful tool to tie Usability into business success.

A Brief Overview: 

SUS (sometimes written SuS) stands for System Usability Scale and was originally created in 1986 by John Brooke to measure the usability of new computing systems that were being installed in offices at DEC at the time. It was quickly adapted to a wide variety of applications. 

To obtain the SUS score, participants respond to 10 statements to gauge their perception of the usability of the system in question. By responding to each question with a 1-5 rating of how strongly they agree or disagree with the statements, an overall score can be calculated. 

The power of the scale lies in its simplicity. The test can be completed in mere minutes. You can gain valuable insight from even two participants though it is suggested to have 8-20. The language used in the scale can apply to a wide variety of systems – whether physical products, websites, or digital software. 

One critical thing to note is the relationship between the questionnaire and the SUS Score. The answers to individual questionnaire questions are not meant to be evaluated on their own. The SUS Score a single unit of measure derived through a series of calculations as noted below.  

Calculating the SUS Score

Converting questionnaire responses into a SUS score is a multi-step process. Various calculators are available to convert the questionnaire handed participants into the SUS score. You can also find a walkthrough of the calculations at the bottom of this article. 

One of the most confusing aspects is the process of converting multiple questionnaires into a single score. The odd-numbered questions ask about the positive aspects of the system and the even-numbered questions ask about the negative aspects of the system. This is by design - these "speed bumps" prevent individuals from providing the same numbered answer by rote, but does make it confusing to calculate because the numeric ratings of the even-numbered questions have to be inverted before being converted into points before averaged into a total score. Users' total scores are then averaged and scaled to get a final SUS score. SUS is only the overall score - the scores related to individual questions within the SUS are NOT ever considered individually.

Evaluating your SUS score

One of the best ways to tell how your site is doing is to compare it to other SUS scores. It is hard to evaluate the SUS Score as a number because there is no context provided by the SUS creator to establish what is a “good” or “bad”. 

In 2008 Bangor, Kortum and Miller measured how various user interfaces perform on the SUS scale. They found that websites “averaged” a score of 68. This was actually pretty good compared to other technologies such as cell phones and TV interfaces:

While a website receiving a score of 68 might be “average”, web owners should be careful not to assume that this equals a “passing grade”. An “Excellent” score is closer to 80. and as market saturation increases and new technology keeps vying for our attention, users will often choose the site that is more easily usable. Having just an “average” experience may not be enough to keep users from your leaving for a more usable one. 

3 Reasons to Measure the SUS Score 

When properly understood, the SUS score can be a powerful tool connecting user experience to business objectives.

1. SUS can help you evaluate whether a system needs to be updated  

The SUS score is a trusted measure of usability and has been cited in over 1,200 peer-reviewed publications. The results can be used to inform business direction prior to making any changes. A lower score can be used to back-up the argument that significant updates are needed, a higher score can make the case for major updates not being necessary. 

2. SUS can help evaluate the effectiveness of system improvements over time

Repeated studies measuring SUS score can help ensure your website or experience is improving. By having groups of individuals test the site at different times you can gather feedback at different stages of the development to understand if you are making progress towards creating a better user experience. 

3. SUS can help build a business case to invest further in UX

Quantifiable numbers can serve as a common language and a unifying measure across departments.  Because higher SUS scores have been shown to correlate with positive revenue growth, the SUS score can support a business case for investing in improving the user experience across multiple departments. 

Jeff Sauro, Ph.D. at MeasuringU.com found a positive correlation between SUS score revenue growth consumer tech companies such as Dropbox, AutoCad, TurboTax, McAfee, Microsoft Office, and Mint.com.  

As you can see in the graph below that DropBox, which had a SUS score of 77 in 2014, doubled their revenue over the following two years. On the other side of the spectrum you have Autocad, who had a SUS score of 58 in 2014 and whose revenues shrank by half significantly over the same period. There were multiple other companies on the middle of the spectrum that ran the gamut, but what came of the study was an indication that a higher SUS score can predict increases in future revenue. 

2 Limitations of the SUS Score

Despite being an industry-standard tool, the SUS as a number has very specific limitations that are important to keep in mind when evaluating performance. 

Limitation 1: A SUS Score is not a letter grade

The grading of SUS does not accurately correlate with the alphanumeric grading scale we received in school. 

Imagine you received the news that your site received an 85 SUS score. With no context, you might think, “Oh, okay, we got a B. That’s good but we can keep improving.”

What can be more useful is to see how your site stacks up with competitors. As shown below a SUS Score of 85 would be excellent and put you on par with the iconic experiences Amazon and Gmail deliver. And keep in mind many industry-leading products have scores that are far lower, such as Excel, which has a score of 57. 

Limitation 2: A SUS Score is not a diagnostic test 

The SUS score cannot tell you specifically which parts of your website or product cause users issues. To get that data, you need to observe users interacting with a website or product. For this reason, the score is most useful when coupled with UX study results. The SUS score can be measured to indicate overall how the usability is, and then you can utilize UX study results to hone in on where to start making improvements.

Conclusion: 

Numbers can be a shared language between departments to improve business outcomes. Having SUS score data, and the understanding of what it means, can lend clients the credibility needed to continue to get funding for UX research and design studies they need to bring the product to its fullest capacity. 

Now that you understand a little bit more about what the score means, I hope you will be able to further utilize the SUS score and share this data within your groups around this organization to measure and improve the performance of your user interface. 

---

Appendix: Calculate Your Own SUS Score

If you’d like to collect and calculate your own SUS score, I’ve broken down the steps you can follow along to convert your user questionnaires. 

Why is it so complicated? 

The math for the calculation confused me until I took a closer look on the questionnaire. Questions that are odd (1,3, 5, 7, 9) generate a positive response, but questions that are even (2, 4, 6, 8, 10), generate a negative response which is why their scores have to be inverted. All the points added up together could gain a maximum of forty, thus the multiplication by 2.5 to make the scale out of 100.

Step 1: Convert the user ratings from the 10 questions into points

For odd numbered questions 1, 3, 5, 7, 9:

• [User Rating] – 1 = ___ points

For even-numbered questions 2, 4, 6, 8, 10:

• 5 – [User Rating] = ___ points

Step 2: Add up the points from the 10 questions into a user’s total points

• [Question 1: __ points] + [Question 2: ___ points] + … [ Question 10: ___ points]  = ____ total points from user 1

Step 3: Multiply the user’s total points X 2.5* to get an individual user’s score. 
*This converts the scale to be out of 100. 

• [Total points from user 1] x 2.5 = User #1 score

Step 4: Repeat steps 1-3 for all users, and then average all user’s Scores together to get a SUS score:

• User #1 score + user #2 score + user #3 + [etc.]  / # of users = SUS Score

Get in touch to learn more about Bentley UX consulting services and how we can help your organization.