User Research During COVID: How To Expand Your Playbook

What comes to mind when you think about doing user research? It used to be traveling to far off places to meet with users, conducting contextual inquiry wherever your users work or live, or meeting study participants face-to-face in a usability lab. But, as with many other things, the way we do user research during COVID looks different.

New constraints are influencing how we conduct research, and some of these constraints will have long-lasting impacts. Despite new challenges, one thing remains the same: User researchers are being asked by clients and development partners to continue to deliver user insights that propel innovation.

In this blog post, we’ll share a few ways to adapt your user research during COVID. These strategies allow you to answer the same fundamental questions while keeping your team and participants safe in this new global environment.

Our Go-To Research Methods Have Been Impacted

User researchers are used to being there in person, having face-to-face conversations and interactions with study participants. These types of interactions are central to our work, and figuring out how to proceed without them, or with a modified version of them, has made for a challenging (some of us would even call it “exciting”) COVID pivot. (You can read about some of the pivots we started making back in April in our post, Our Favorite Remote Research Methods for Product Design.)

Practices like home visits, contextual inquiry, gathering feedback on physical interactions, and validating products in simulated use environments may continue, depending on the settings and participants, but they now require additional forethought.

For instance, we’re using more disposable equipment and PPE, and we’re increasing our over-recruit rates from 20 percent to 40 percent to account for increased cancellations, no-shows, and unexpected screen-outs at the door. We’re developing new screening questions, considering new IP concerns, and planning in additional time for cleaning. We’re being especially mindful about how we work with high-risk populations, too.

We anticipate that many of these changes will continue well into the future.

Focus on the Questions You Want To Answer

While this may sound overwhelming, the core principles of user research during COVID remain the same as ever. To organize our thoughts, we’ve structured this discussion about new and modified approaches across four categories of research goals: Discovery, Framing, Evaluation, and Validation. Below, we’ll focus on how we can continue to satisfy the fundamental research questions that we always ask during each of these four phases, using the tools at our disposal.

In some cases, we may be digging up tools that we once used less often but that are just as effective at gathering the insights we need. This is less about replicating in-person research remotely and more about adapting our approach.


DISCOVER: Three Strategies for Remote Discovery Research

The discovery phase of research is a bit like playing in the ocean of possibility. We haven’t defined our design problem, and we’re not even close to a design solution. Here, we ask questions like:

  • What are the user needs in this opportunity space? Where can we innovate?
  • What type of solution might meet user needs? A service, a product, an app?
  • If we’re developing a next generation of an existing device, what has changed in the user’s context of use since the previous release?

To answer these questions remotely, we use remote contextual inquiry, diary studies, and remote interviews with a digital whiteboard component.

Discovery Strategy #1: Remote Contextual Inquiry

In our new COVID reality, interviews and home tours can be conducted via webcams. This allows researchers to ask targeted questions, observe users’ interactions with a product in real time, and ask follow-up questions immediately.

The main benefit is being able to see a user’s environment and the space a product is being used in.

Discovery Strategy #2: Diary Studies

While tried-and-true diary studies aren’t used frequently, the practice deserves more consideration in light of the pandemic. This method asks users to record their experiences in real time, via video or written journal. It allows researchers to understand reactions to a product, service, or activity as the participant is using it.

This also helps participants reflect on their experiences, so they’ll likely bring more developed, concise thoughts to wrap-up interviews.

Discovery Strategy #3: Remote Interviews + Digital Whiteboards

We also find it useful to conduct traditional one-on-one interviews online and to record insights, findings, and pain points on a digital whiteboard that both the researcher and interviewee can see. We’ll talk as if we’re sitting next to each other, and we use the screen between us to develop a visual representation of what we’re talking about.

This strategy allows the researcher and participant to quickly align on any processes, systems, and emotions that could drive design forward. This can be especially useful for understanding a complex process or system and mapping a user’s emotions as they use a product or service.


FRAME: Three Ways to Adapt Framing Activities and Methods

Once we’ve uncovered the ocean of possibilities, we use framing to identify more focused direction. A lot of the work here is about prioritization. We’re asking questions such as:

  • Of all the needs we discovered, which ones are unmet?
  • Of all the ideas we’ve come up with, which should we move forward with?
  • Of all the features we could include, which are must haves? What’s the MVP?
  • How does all of this align with our business strategy?

Although a lot of our go-to methods for framing are survey-based approaches, we’re still left wanting when it comes to collaborative workshops to help balance the business and the user perspective.

Framing Strategy #1: Small Group, Multi-Step Working Sessions

In the past, we might have brought large groups together for full-day workshops. That’s no longer an option, so now we break full-day workshops into several discrete, remote sessions.

We might schedule the sessions in two-hour increments spread out over a week, and we set just one or two goals per session. We often share a screen and use digital whiteboard tools here, too. This allows us to engage in meaningful conversations with end users and business stakeholders and to make sure that everyone’s voice is heard.

Framing Strategy #2: Use Multiple Methods

It’s important to dig in and think about your entire toolbox. Be willing to do smaller, qualitative studies quickly and iteratively, and don’t forget about quantitative methods. The key is to adopt a more agile research mindset.

For instance, you might ask six participants to write a love letter or break up letter to a product, apply those insights, and then move on to another group. You might conduct a few interviews to start and then expand into a larger-scale survey — or vice versa.

Framing Strategy #3: Use the Same Tool Twice

This is one way to get the most bang for your buck. Create two versions of your Kano Model or Jobs To Be Done surveys. Gather data from end users and business team members at the same time. Then, compare and contrast the results.

This can help identify misalignment between users and the business. It can also be useful for remote business teams that are struggling to stay on the same page across disciplines.


EVALUATE: Three Strategies for Adapting Evaluative Research

Evaluative research brings ideas to life. Once a solution emerges, we refine it into an easy-to-use final product that meets all users’ needs. We might be asking:

  • Which prototype should we fully refine?
  • Is the product easy to use?
  • How do users perceive and understand various aspects of the design?
  • How does the usability compare to the competitor product?
  • Is the product an improvement over the previous generation product?

This type of research is an iterative process that can involve a variety of qualitative and quantitative methods. For physical products, this stage almost always includes hands-on use by participants. As we adapt our approach, our main focus is ensuring people can use a product, even if we can’t be in the room observing those interactions.

Evaluative Strategy #1: Study Kits for Remote Interviews

This involves sending participants audio-visual equipment, a study product, detailed instructions, and a return shipping label. Participants become part of the research team, setting up recording equipment and the environment. While there are some limitations, this strategy allows for a close approximation to in-person testing.

Evaluative Strategy #2: Bite-Sized Research

We find it helpful to narrow the scope of a remote evaluation to decrease participant fatigue. Limit each effort to one or two high-priority research questions, and separate remote interviews into product and digital focused sessions, for example. This allows research questions to be answered quickly and effectively.

Evaluative Strategy #3: Unmoderated Quantitative Research

We also use quantitative methods like surveys and unmoderated web-based experiments to understand user behavior. This allows us to identify subtle differences and predict population-level feedback and behavior through larger sample sizes and statistical analysis.

This method is relatively quick, and you don’t need a researcher walking participants through each session.


VALIDATE: Three Ways to Adapt Validation Research

Validation research ensures a product is safe, effective, and meets our design goals. This phase of research answers questions such as:

  • Did we meet the user needs we targeted for this product?
  • Is the product safe and effective to use?
  • Does the product introduce any new issues / challenges for our users?
  • Have we completed a thorough assessment of foreseeable risks?

Unless testing an exclusively digital product, this usually requires that participants physically handle a product. While regulatory bodies have shown wariness about remote methods for validation, we need to keep users and researchers safe. Products designed for use with “at risk” populations may introduce more risk to the population if we require them to come to sessions.

Validation Strategy #1: Research by Mail

This tends to work best for small, simple products with few steps that are designed for in-home use. We highly recommend regulatory review of the protocol before you begin.

Validation Strategy #2: Behind-the-Glass Moderation

Much of the industry is shifting to research conducted from behind a clear partition. The goal is to produce the same type of interaction as we had before COVID, only with a physical barrier between the moderator and participant.

This requires appropriate PPE, new operational procedures, and structural updates to the space. But these validation sessions can be completed with minimal efficiency loss.

Validation Strategy #3: Off-Site Moderation

You might decide to have a local research facility set up the sessions and technology and utilize your experienced moderators to guide the sessions remotely.

This can be useful with hard-to-recruit populations, or when you need to run studies in multiple cities but cannot easily travel. The moderators can guide the sessions from thousands of miles away.

One variation is to build a remote moderation suite in a hospital environment. You might gain better access to people who are busy dealing with COVID if they can stop in before or after their shift.

The Thrill of the Research Challenge

Each challenge we encounter is a new opportunity to reframe the way we think about things and solve problems in new ways. As we navigate this “new normal,” our emphasis is on mixing methods. By bringing together different strategies, we’ll build a robust and flexible toolbox.

It’s important to keep in mind that this situation is new and changing rapidly. We’re all always learning. Our goal is to connect with the research community so that we can mitigate the new challenges, learn from each other, and advance user research. Don’t hesitate to reach out to the Bresslergroup team if you’d like to collaborate.

Conall Dempsey

Conall delights in finding unexpected ways to improve people’s day-to-day lives and considers research to be the means to this end. He first realized the potential of design to improve lives and drive innovation when he went off-script with a K’nex set at age ten and, after much trial and error, constructed a repeating rubber-band launcher.

Sarah Fairchild, PhD

Sarah holds a PhD in Cognitive Psychology from the University of Delaware, and she has spent years researching how people perceive objects and the written language around them. Through user research, Sarah is able to use her fascination with human behavior, as well as her knowledge and experience, to solve real-world problems.

Jemma Frost

Jemma prides herself on putting together fun, productive workshops, and she strives to make her research findings both visual and actionable. Her favorite moment is when the team has sifted through the research, sat with their thoughts, started to connect the dots, and the “ah-ha” hits.