Over 15 years of practicing and teaching UX research, I have observed that it’s the analysis of qualitative research data that often presents the most daunting challenges to new practitioners.

A formative session with eight participants can generate many pages of text data. Just looking at so much text can be overwhelming. Three common questions that come up in these situations are:

• How do you take loads of qualitative data in the form of session notes and transform them into concise findings and recommendations?

• How do you go through the data methodically and don’t lose any important insights in the process?

• How do you make sure that your findings are based on data from all the participants rather than your recollections of the most memorable sessions or your own personal impressions of the issues?

There are many tools and approaches to help us analyze qualitative data. In this blog, I’m going to share two simple recipes that have worked well for me over hundreds of projects that I have conducted. I hope they will be helpful to those feeling overwhelmed by qualitative data analysis.

two simple recipes that have worked well for me over hundreds of projects that I have conducted.

I'm going to refer to the first recipe as Bottom Up. As the name suggests, the general idea of this approach is to start with raw data and let the data dictate the report findings. As you identify findings and assign participants that experienced an issue, you build your report with supporting details.

Recipe 1: Bottom up

INGREDIENTS

• Print outs of all high-quality* session notes – one set of notes per participant (*while note-taking recommendations are outside of the scope of this post, this recipe requires underlying notes to be comprehensive; otherwise, it won’t work )
• Digital versions of all session notes
• Highlighter
• Report Template– something that you will use as a repository of findings and recommendations - can be in a variety of formats such as Power Point, Word, Excel, Online, etc. I usually use Power Point and will use it as an example here.

Preparation time -  About one hour of analysis per one hour of session time

Step by Step instructions

PHASE 1: PREPARATION

1.  Print out all the session notes.
2.  Using your Report Template, create a basic structure for your report – for example, Methodology slide, Executive Summary and Detailed Findings. If possible, add relevant screenshots to detailed findings.

PHASE 2: DATA SYNTHESIS

3. Start reading at the beginning of the hard copy of your first participant notes.
4.  The moment you come across a positive or negative finding in the notes, add that finding to the report and add the corresponding participant number to the finding in parenthesis.

• When writing up the finding in the report, describe it in a way that gets to the core of the issue but is not overly verbose – the description should communicate the gist of the issue but should be easy to adjust later on, as you review more data and may decide to fine-tune it.

Example of  two findings: 

Positive Finding: “Search bar is easy to find in the upper right-hand corner.” (P1)

Negative Finding: “Shoes not grouped together by style makes it difficult to quickly scan and see all shoes available in a certain style together in one place.” (P1).

5.  If your notes have a good quote illustrating the issue, add it under the corresponding finding right away. If you don’t have a complete quote, add as much of the quote as you have in your notes together with a time stamp and participant number with a reminder to get a complete quote later.
6.  Once you are done documenting a finding in the report, use the highlighter to highlight all the hard copy notes conveying that finding. This way, you will know that everything highlighted is documented.
7.  Repeat steps 4 – 6 until you get to the end of your notes for your first participant (P1).
8. Repeat the process for every participant’s notes and add corresponding participant numbers in parenthesis.

THINGS TO PAY ATTENTION TO AS YOU ARE ADDING FINDINGS TO THE REPORT:         

• If you come across a situation when subsequent participant notes contradict a finding already documented in the report, add an explicit call out of the contradiction. For example,
  • Positive: “Search bar is easy to find in the upper right-hand corner.” (P1)
  • Search bar was difficult to find (P2)

• As you are adding findings to the report, keep it well organized: have everything neatly formatted, group related topics together, provide headings and subheadings throughout your narrative. This will help you keep your findings visually organized, which will subsequently help you make sense of the data. 

• If at any point during this process, you think of a good recommendation, add it to the relevant section right away. If you think of an important idea that you don’t want to forget, add it to the executive summary. Your ongoing recommendations and executive summary findings may change as you go through the analysis process, but I find it helpful to add them as they pop into my head during data synthesis.

PHASE 3: INTERPRETATION

Once all your raw hard copy notes are covered by a highlighter, you know that you have dotted all the i's and crossed all the t's. You can now feel confident that all your data has been documented in your report template. For every finding, you know which participants experienced that issue and which did not. It is now time for the interpretation of the data clearly laid out in front of you.

At this point in the process, you are finalizing your report – you are:

• going through all the findings, reviewing for clarity and making sure that descriptions are concise and recommendations are clear and actionable
• deciding which findings need to stay and which, in your professional opinion, can be removed
• fine-tuning the wording, organization, formatting, and removing participant numbers from the report
• taking a step back, reviewing what you’ve got, formulating main takeaways and finalizing your executive summary

When you are done, you feel confident about doing due diligence and producing a report with a clear picture of the insights your research uncovered.

Recipe 2: Top Down

The second recipe is called Top Down. This approach starts with documentation of findings based on your team’s impressions – using team's recollections to develop a hypothesis of what your research uncovered and then going through the raw data to prove or disprove various parts of the hypothesis.

Step by Step Instructions

PHASE 1: PREPARATION

The team gets together and discusses research findings. During the discussion, every positive and negative insight brought up by the team gets documented in the Report template. The team tries to keep observations documented in a logical manner – starting with larger takeaways and following with more detailed insights, grouping all related findings together, providing headings and subheadings to sections and subsections.

Once the team finishes documenting their recollections of the findings, the next phase is a modified version of Phase 2: Data Synthesis from the Bottom Up approach.

PHASE 2: DATA SYNTHESIS

1. Start reading at the beginning of the hard copy of your first participant notes.
2. The moment you come across a positive or negative finding in the notes, see if the finding is already documented in the report. If it is already there, add the corresponding participant number to the finding in parenthesis. If it is not documented yet, add it with the corresponding participant number in parenthesis.
3. Follow the remaining Phase 2 steps in the Bottom Up approach description until you finish reviewing all the participant notes.

PHASE 3: INTERPRETATION

Once all your raw notes are covered by a highlighter, turn your attention back to the Report Template.

4. Review all the findings you have documented.
5. Remove any findings that have no participant numbers in parenthesis next to them. If there are no participant numbers, it means that initial team recollections proved to be inaccurate once subjected to data verification. 
6. Follow the remaining steps from Bottom Up recipe until your report is finalized.

Voilà– you are done!

Both Bottom Up and Top Down approaches are easy to follow and involve review of all your session data. I find that Top Down works better when more than one person works on data analysis. It allows everybody on the team to develop an initial common understanding of the data and then work individually on parts of the report during data synthesis. Bottom Up approach may be more appropriate for solo practitioners or when it's challenging to formulate an initial hypothesis of what was learned during research. I hope that some of you may find them helpful.

Lena Dmitrieva

Senior UX Consultant, User Experience Center, Bentley University

Lena is passionate about empowering users of all ages and walks of life with efficient, easy to learn, and enjoyable products. She strives to help her clients create successful customer experiences by connecting with their users, understanding their needs, and translating those needs into product solutions.

Lena enjoys the challenge of identifying appropriate UX methodologies and has extensive experience conducting user experience projects as well as training others in User Experience work.

 BACK TO ALL UXC BLOGS

Get in touch to learn more about Bentley UX consulting services and how we can help your organization.