User Experience Center
Thumbnail

Make Your Language Reflect the User’s Vision, Not Yours

Your website offers a product, service, or experience that your users may have never heard of. Though the text on your website may accurately describe how a product or service technically works, a user will draw on their individualized mental map of prior knowledge and experience to understand how a new product is similar or different from what they already know. How can you find out if users understand your product through your website’s language?

Here are 7 methodologies to see if your website’s language actually describes what your product is:

Think-Aloud Protocol

For use within in a usability testing environment

This tried-and-true method is one of the most powerful tools in a usability specialist’s tool-belt according to Jakob Nielsen, and can be adapted for in-depth text evaluation. When testing passages of text, you can use the think-aloud protocol in two ways:

  1. Ask the participant to read a passage quietly to themselves, then instruct them to paraphrase what they read after they have completed it. Rather than taking notes about their sentiments, take notes on the words they use to describe the passage or what sections they note as confusing so you can revise them later.
  2. Ask the person to read the passage aloud, and ask them to pause if they come upon anything that is confusing or unclear to them. This method may be better for longer passages where summarizing in their own words could be difficult or stressful.

Doing this type of a test with the Rapid Iterative Testing and Evaluation (RITE) method might yield favorable results for shorter passages of text that can be edited quickly.

Readability Tests

For benchmarking and early stakeholder buy-in

There are many tools that analyze a passage’s required reading-comprehension level such as the Flesch-Kincaid readability test or SMOG (Simple Measure of Gobbledygook). These tools determine a passage’s readability (usually in terms of grade-level) by analyzing factors such as the number of syllables per word and the number of words per sentence.

Although these tests are based on algorithms that don’t take the reader’s background into account or provide recommendations for improvement, it’s a cheap method that may assist in getting initial stakeholder buy-in. A readability test should ideally be followed up with another form of testing that gathers more user-centric insight.

There are many free online tools available to quickly evaluate a passage of text against these formulas such as read-able.com and readability-score.com.

Analytics & A/B Testing

For high-traffic pages with less information-dense text

A/B testing can be used to get quantitative data on text with lower information density, such as button copy and headlines (a headline A/B test resulted in a 90% increase in sales for Movexa). Elements with more information density, such as paragraphs or articles, might not yield any significant findings with this method due to their complexity.

By creating two live versions of the website with different text, looking at analytics such as session duration and button clicks may show which text drives more conversions. This method also may require more technical resources than other methodologies.

Expert Review

For projects with constrained time and budget

While keeping the persona in mind, use a set of guidelines to review the document to ensure all of the standards are met. Multiple specialists can also review the same document and compare their results afterwards.

Some popular guidelines include:

Single-Question Survey

For time-constrained projects

A short passage of text may be tested through a survey to quickly gather data. If your company is already conducting surveys for other purposes, this may be an easy way to get feedback on your website’s language. If the survey service you are using has tools to target towards your personas, use them to get more accurate results. This type of survey could take two forms:

  1. A quantitative multiple-choice survey where the participant is tested on their comprehension of a body of text (“What was this paragraph about?”).
  2. A qualitative free-form response survey where a participant is asked to rephrase text in their own words (“Describe this paragraph in a sentence”). This can reveal insights as to what words participants use to describe the text.

Long Survey

For longer documents (e.g. legalese or website documentation) with high information density.

Ask the participant to read through a long document, then ask a series of questions to see if they understood it or not. Questions can also be asked during the test for longer passages to see how scannable it is (e.g. “find information about our cancellation policy”).

This methodology can be used for testing large bodies of text when a question-answer format would result in too much fatigue. An experienced moderator is not required and testing could possibly be done remotely.

Combined Approaches

For getting creative!

GLPi and Vicki Schmolka improved the readability of the Canadian Employment Insurance Act using a combination of questionnaires, focus groups, and expert reviews (see the study). Participants were asked to read one of three variations of the document: one was the original, while the other two were reviewed and edited according to the Plain Language guidelines. Through a self-administered questionnaire, participants were asked to find answers about employee insurance, then to compare the different versions of the document. The remainder of the test was dedicated to a focus group discussion regarding the questionnaire portion of the test.

Things to Consider

If a participant is interested in your product, it could be difficult to get them to talking about the features in the top bar when you’re trying to get them to focus on the paragraph of text below it. Testing within an interface can be too distracting, so sometimes it may be best to have the participant read the text from a piece of paper instead. If the other website content is not too distracting, testing in the correct context could change the meaning of the passage due to its surrounding content.

Some participants might be reminded of reading comprehension tests from their childhood for some of these methodologies (especially the questionnaire). It is essential to reinforce that the study is not a test of their personal abilities or reading comprehension, but of the text that they are reading.

Reach Out

If you decide to use any of these methodologies, or use one that I didn’t list here, please share your story by reaching out to me at johnhrobb@gmail.com. I hope you have the chance to evaluate the text on your website and to add your findings to this scarce area of UX design.

JohnJOHN ROBB | JOHNROBBDESIGN.COM

I’m a research associate at the User Experience Center at Bentley University, and my business cards say “UX Designer.” Before pursuing my MS at Bentley’s Human Factors in Information Design program, I studied Communications at Pace University, then freelanced while working as a designer for a medical startup in Baltimore. You can reach me at johnhrobb@gmail.com

 

 

BACK TO ALL UXC BLOGS