Introduction

UX researchers consider how different user populations will approach and interact with products in a research setting. When working with minors, a protected class under federal law, there are various factors to consider for ensuring the comfort of your participants and success of the study. For example, some of these factors include curating a calm test environment, developing rapport with the moderator, and establishing trust between the minor and the moderator so that the child feels compelled to report on their experiences transparently (Markopoulos & Bekker, 2003). While there is a plethora of literature and best practices for designing usability studies for adults, there are not as many publications available focused on conducting usability testing with children (Hanna et al., 1997). To help close this gap, we want to share our key takeaways with the UXR epistemic community based on our experience working with minors over the past year. We recently completed three different UX projects for a prominent education-related client. In total, we conducted 91 research sessions with 79 participants (some children participated in multiple studies). The three different studies spanned students ages 8-14, including English-limited and neuroatypical students who had IEPs (individual education programs).

These reflections serve as a practical guide for optimizing UX research with minors, but also serve to underscore the importance of including children in usability testing. Designers can fall into a common trap of believing they know how children think because they themselves were once a child. However, memory is fallible, and the experience of children today is filtered through a vastly different context than even five years ago, let alone fifteen, thirty, or more. Beyond the obvious reason of researching the product in the context of its intended user group, involving minors in user research can yield a myriad of benefits when conducted in an ethical and compliant manner. Some of these benefits include identifying socioeconomic factors influencing educational outcomes and gaining a deeper understanding of child development in order to improve educational outcomes for their age group. Based on our own experiences working with minors during the past year, we have collated the tenets below to share what we have learned about planning and conducting optimal UX research with minors.

Tenets

These tenets have been curated to improve interactions with minors when employing the usability testing research method of UX design. They can help researchers to navigate the unique dynamics of working with minor participants to elicit an enjoyable experience for study participants while supporting valuable data collection that meets the research goals of a given usability testing-based study.

Logistics

1. Over Recruit to Avoid No-shows/Cancellations: When recruiting and scheduling participants, over recruit the actual number of participants required to mitigate the risk of no-shows or last-minute cancellations. The risk of cancellation increases when conducting a study with minors because there is a large dependency on a guardian or parent to transport the child to the testing location and a session becomes more susceptible to cancelation if either the child or the guardian/parent has a scheduling conflict arise (e.g., illness, schedule shift, emergent priorities with siblings or other family members, etc.). We typically over-recruited our sessions by 3-5 participants per project the past year. However, the number by which you would over recruit depends on the specificity of your user population for the study. For example, one of our studies required neuroatypical students who had Individual Education Programs (IEPs) and our testing location was not easily accessible by public transportation, so we instructed our recruiter to find a surplus of IEP students to offset the risk of cancellation for this user population. In some cases, you might not utilize any or all the surplus participants, so account for this in the project budget because it’s best practice to compensate confirmed participants who are no longer needed when the project quota has been met.
2. Take Advantage of the School Year Calendar for Scheduling Sessions: Explore the vacations and holidays of the local schools near your testing site and have the recruiter take advantage of dates in which students do not have school and can devote time to completing a usability test virtually or in-person. Most sessions with minors will typically have to be scheduled around school hours, which in the US northeast, typically means sessions have to be scheduled from afternoon to late evening (~3:30PM-7:30PM). For example, we ended up having testing “super days” during the winter holidays, spring break, and summer vacation weeks because students were home from school and parents/guardians were able to bring them into our lab for testing during the day. Concentrating sessions during school vacations proved not only more convenient for the participants’ schedules, but also increased momentum for our research team because we processed more data and insights per day and were able to get into an enhanced rhythm when facilitating the protocol.
3. Prepare to Accommodate Additional Family/Friends: When working with minors, it is quite common for their siblings, classmates, or neighbors to accompany them to the session because the parent/guardians are responsible for their care as well. This occurred multiple times and having a plan to host the other attendees in a separate room or area that would not be distracting for our testing participant (or lab technician) proved helpful. It was particularly valuable when our recruiter let us know ahead of time that siblings or others would be attending so that were prepared to greet the group and hospitably provide a second space (classroom, conference room, cafe, etc.) where they could wait during a given 60-minute to 90-minute session.
4. Coordinate & Compensate Ride-Sharing Services: If you are conducting an in-person usability study, consider arranging and paying for a ride-sharing service, such as Uber or Lyft, to help transport participants to the lab if their parent/guardian does not have access to a car, or is coming from a significant distance (e.g., from downtown Boston to our suburban testing location). This could help increase recruitment numbers and reduce wasted time related to parking and on-site wayfinding to the lab facility. For example, one of our studies had majority of participants with an annual household income < $30K. We were experiencing significant difficulties with recruiting and no-shows, but once we decided to arrange and pay for ride-sharing services to pick up and drop off participants and their parent/guardian, we were able to more easily reach our study participant quota. However, if using ride-sharing services, incorporate extra time between sessions to account for travel-related delays and ride cancellations.
5. Adhere to Legal Policies for Working with Minors: Since minors are a protected class under federal law, there are various compliance items professional UX researchers must adhere to for conducting a usability study. Minimally, you will need to obtain a signature confirming informed consent from the parent/guardian for the child's participation in the study, and assent from the child to participate in the study. Be sure to receive consent for any study-related audio or visual recording as well as for their participation. Additionally, the research team should be cognizant of dressing appropriately and sitting/speaking in ways that make the minor comfortable. Best practices mandated that at least two researchers OR at least one researcher and the parent/guardian of the participant were always present in the room with the child during the session. If we didn’t have two adults that were available for the session, we would be unable to facilitate the session. In addition, best practices require the researchers' and the participants’ hands and interactions to be visibly on camera throughout the study which had some bearing on how we set up our physical space and recording equipment. Legalities vary from study to study, depending on the study goals, methods, locale, and the client sponsoring the study, among other factors, so be sure to commit time to understanding and fulfilling the appropriate legal requirements for your research with minors. Best practices for ethical and effective usability testing planning and execution, consent, assent, and honoraria provision are always applicable.

Environment

6. Ensure Minors are Physically and Emotionally Comfortable: Participating in a usability study can be very daunting for anyone, especially a child. Walking into a new environment where cameras and other equipment are present, meeting new people, and being concerned that they will be “tested” can be nerve-wracking for a child. It’s helpful to remember, and to remind participants, that in a study of this nature they themselves are not the focus of the study- the software, educational evaluation tool, or product - is what is being tested. In simple terms, it’s a study that involves human subjects, but not a study of human subjects. Therefore, when building rapport and putting a study participant at ease, it is paramount to establish a variety of practices aimed at making the minor both physically and emotionally comfortable. For example, we always offered water and the chance to use the restroom to the participant upon arrival and gave coffee/tea to the guardian/parent while providing an overview of the study. Presenting a thorough introduction of yourself as the researcher and the purpose of the study helps the children to understand their own role. We found increased engagement and participation when we were able to help participants reframe their role from “test-taker” who needs to perform well to “expert tester” who held the valuable insights we needed to learn. Usability literature supports this reframing as well (see: Griffin, 2019).  

   We also recommend the moderator clearly set expectations for what will occur during the given usability session, which we noticed calmed test anxiety some children might have been experiencing. In addition, we included conversation starters in our protocol (e.g., asking their favorite subject in school, activities they enjoy participating in, or what they’ve done over the summer that they found fun). This helped our research team to develop rapport with the study participants. Once we provided a warm welcome to make students feel comfortable, we toured them through our usability lab facility, showing them the cameras, one-way mirror, etc., so they were entirely aware of the space and felt more trustful of the UXR team. To make the lab feel more welcoming, we put encouraging signs and seasonal decorations around the lab and laid out school supplies on a table to make our testing facility feel less clinical and more like a classroom setting. These specific efforts to make the participants emotionally and physically comfortable during sessions are also supported by the academic literature (see: Hanna et al., 1997).

7. Manage the Parent/Guardian Presence During Testing: Seek to have the parent/guardian in a separate room while the minor is completing the usability test to minimize their influence or potential bias their presence might have on the child while testing. In cases where you need additional adult supervision or the child feels uncomfortable being away from their caregiver, literature suggests to “make provisions for the parent to stay in the testing room with his or her child (ideally out of the child’s line of sight), and explain to the parent that he or she may observe but should interact with their child as little as possible during the test” (Hanna et al., 1997). We learned this lesson after one of our sessions in which a mother was acting as the second chaperone in the study lab. She was very passionate about the study goals, and as a result, would talk over her child and question or even alter some of their answers. In response, we adjusted our approach for subsequent sessions to be consistent with what the literature suggests and instructed guardians/parents in the room to remain quiet during the session. Additionally we suggested that their comments or questions could be written down as the sessions unfolded and one of our researchers would address each item with them after the session was completed. We found that asking the parent to write down questions made them feel involved in the study while mitigating bias in the data; a win-win scenario.

Protocol

8. Incorporate Easy-to-Understand Language in the Study Protocol: We recommend using grade-appropriate language in the study protocol so that children understand the scenarios and tasks of the test fully. For example, we updated the language in our protocol from "transition” to "switch” (when participants were asked their feelings around moving from one activity to the next) to cater to the 8–10-year-olds we were working with on one of our projects this year.

   Research also suggests sticking to concrete vs. abstract concepts can help tailor language to the child’s developmental level (Griffin, 2019). We experienced this finding with one of our study tasks which required the student to adjust the angle of their device screen. Our original protocol used the abstract language of “Can you show me how you would adjust the screen angle?” which was confusing most participants because they would often think we were referring to the zoom in/out function of the software. After observing this issue, we adjusted the protocol to include a concrete, physical direction of “Please show me how you would tilt the screen backwards,” which proved more effective for students in understanding the intent of language in our task.

   Another way to make language more accessible is to repeat the participant’s own words back to them when discussing a concept. This requires active listening on the part of the moderator to be sure they are using the same meaning of the phrases that the students originally stated. (For more information on how this tactic meets the developmental language level of the participant see Griffin, 2019).

9. Consider Developing Metacognition in Children: Because children have developing communication skills, it can be difficult at times for them to clearly express and articulate their experiences with a device (Markopoulos and Bekker, 2003). However, "for children under 11, visual stimuli can be especially useful in the questioning process” (Scott, 2008). We followed this best practice by asking participants how a task made them feel and instructed them to select an emoji on a printout of 40 emojis to get them started in expressing their thoughts on the interface.

   Like communication, a child’s memory is still in a developing stage, so the literature also suggests employing visual aids to memory to foster richer discussion (Scott, 2008). We also accounted for this by providing printouts of screenshots when referring to previous screens and offering a printout of responses so students could see all the answer choices in one visual aid when responding to our questioning.

10. Don’t Underestimate Behavioral Data: A portion of our participants were shy, unresponsive in questioning, or gave verbal feedback that contradicted their observed action. For example, a student would choose a positive response on a Likert scale (e.g., “very easy”) when asked about the difficulty of a task. However, while completing the task, we noticed the same student would be cringing, rolling their eyes, or displaying other signs of frustration and struggle. We included a dedicated section in our notes template to record any behavioral observations and discrepancies like this so that we could leverage this engagement data for context and insights when reporting. In addition to our own realization of the importance of observational data, the use of behavioral data is also encouraged in scientific literature, especially for younger children (Hanna et al., 1997)

Additional Considerations for ESOL (English for Speakers of Other Languages)

There is a baseline level of trust required when recruiting a participant to any study and that level is increased when a language barrier is present. When recruiting participants who have Limited English proficiency, using the snowball method (i.e., leveraging referrals from previously recruited participants) may be helpful in establishing legitimacy and building trust within the community from which you are recruiting.

When thinking about moderating sessions with participants who have Limited English proficiency, although it may seem obvious, we highly recommend having a bilingual moderator who speaks the same language as the participant so that the study can be conducted in the participants’ preferred language. Translating all study materials, including consent forms, into both or all languages offered in a study is also a great way to overcome the language barrier present in sessions.

If the moderator is proficient in the study’s primary language but a translator is still present for the session, be sure to introduce the translator to the participant and explain how they can be leveraged to best communicate between the group during the session. Build in time during the session for interpretation and associated mental processing for the parties involved.  Be aware these sessions are likely to take longer due to the back-and-forth translation time.

Additional Considerations for Neuroatypical Students

When working with minors who are neuroatypical, specifically those who have IEPs for dyslexia, dysgraphia, ADHD, or ADD, there are many adjustments that can be made proactively for the sessions to be as fruitful as possible. First, give extra time to the students to complete tasks. For example, if the session length for neurotypical children is 60 minutes, increase the duration to 90 minutes for neuroatypical participants. Second, review the recruiting file with the accommodation notes that those students receive in school and try to mirror those accommodations (e.g., unlimited time, headphones for tasks, etc.) to the extent possible during the session. Third, consider reading the task aloud along with providing a written print out for the child; this will help mitigate reading or learning differences in participants. Fourth, aim to limit large amounts of text and chunk out the information to make it more accessible for students. Fifth, be prepared to gently nudge students to stay on task who are getting distracted more easily, particularly for ADHD or ADD students. Finally, be sure to offer participants multiple breaks so they can stay refreshed and focused throughout the session.

Conclusion

These tenets will help you to attain effective interactions with minors when conducting UX research. As discussed, there are a multitude of benefits to be reaped from facilitating ethical research with minors for the UXR community writ large. These include increased participation and engagement during sessions, more accurate data collection, enhanced insights into children's perspectives, and the establishment of a positive research environment. Adhering to some of these best practices can make research with minors a very rewarding, impactful, and valuable experience for all stakeholders involved.

References

Griffin, K. M. (2019). Participatory Research Interviewing Practices with Children. Participatory Research with Young Children, 55-71.

Hanna, L., Risden, K., & Alexander, K. (1997). Guidelines for usability testing with children. Interactions, 4(5), 9–14. https://doi.org/10.1145/264044.264045

Markopoulos, P., & Bekker, M. (2003). On the assessment of usability testing methods for children. Interacting with computers, 15(2), 227-243

Scott, J. (2008). Children as respondents: The challenge for quantitative methods. In Research with children (pp. 103-124). Routledg