Generative research

Generative research is used to gain a deep understanding of your users’ motivations, pain points, and behaviors. Its goal is to identify and frame problems, and gather evidence to move forward with developing user-centered solutions. 

Also known as exploratory or discovery research, generative research is usually conducted early on in the product development process. Studying your audience at this stage can help you understand the problem you’re trying to solve, as well as uncover new insights and opportunities for innovative solutions.

One of the key aims of generative research is to gather rich qualitative data that provides valuable insights into your customers – who they are as humans and what their everyday experiences are. By keeping an open mind during the research process, you can explore user needs and desires that may not be known yet, allowing you to empathize with users and develop solutions that will meet their expectations.

Generative research is particularly useful when you’re looking to discover new directions. You can use it to uncover user insights that could lead to new product ideas or improvements to existing products. 

What are the advantages of generative research?

When you don’t conduct generative research, you risk building a product no one needs or uses. Remember Google+? It was a social networking platform launched by Google in 2011 with the aim of competing with Facebook. Despite initial hype and anticipation, it struggled to attract a substantial user base and failed to achieve widespread adoption (not to mention a data leak). In the end, Google announced its discontinuation in 2018. 

This isn’t to say that Google+ didn’t conduct research, but it does show that it failed to understand user needs and wants – something that generative research can help to achieve.

Here are some of the main benefits of conducting generative research.

Gain a deep understanding of your users

Generative research helps you develop an understanding of your users. By exploring their motivations, pain points, and behaviors, you gain insights into their needs and preferences. This knowledge allows you to create user-centered products or services that resonate with your target audience.

Identify problems

One of the primary goals of generative research is to identify and define problems. It enables you to uncover hidden challenges, gaps, or opportunities that may not be immediately apparent. By identifying the right problems to solve, you can focus your efforts on creating meaningful solutions that address the core needs of your users.

Create innovative solutions

Generative research is a powerful tool for driving innovation. It helps you uncover insights and identify new opportunities for solutions and improvements. By understanding your users' experiences and behaviors, you can generate fresh ideas and concepts that lead to innovative products or services.

Develop empathy with your users

By conducting generative research, you can empathize with your users and gain a deeper understanding of their perspectives. This empathy allows you to design experiences that resonate with their needs, preferences, and goals. It helps you create products that genuinely connect with your users on an emotional level.

Make evidence-based decisions

Generative research provides you with rich qualitative data that serves as evidence to inform your decision-making process. It helps you make informed choices based on real user insights rather than assumptions or guesswork.

Early-stage insights

Generative research is typically conducted at the beginning of the product development cycle. This early-stage research allows you to gather insights before investing significant time and resources into development. It helps you make strategic decisions from the outset and lays a solid foundation for the rest of the design and development process.

Generative research methods

There are several methods you can use for generative research to gain deep insights into user behaviors, motivations, and needs. Here are some common approaches.

User interviews

Moderated user interviews allow for in-depth conversations and uncover valuable qualitative insights. When conducting interviews, it’s important to ask open-ended questions to explore your participants’ experiences, preferences, and challenges related to the research topic. 

Contextual inquiry

Contextual inquiries involve observing users in their natural environment while they perform tasks or interact with a product or service. The goal is to understand their needs, behaviors, and motivations in real-world contexts, so you can gain deeper insights that might not be apparent through traditional lab-based interviews.

Focus groups

In a focus group, a moderator facilitates a group discussion with a small number of participants (usually around 6–10) who share common characteristics or experiences relevant to the research topic. Focus groups can provide rich qualitative data through open-ended discussions and can foster new ideas.

Diary studies

In a diary study, participants maintain a diary or journal to record their experiences, thoughts, and behaviors over a specified period. This method can capture rich longitudinal data and provide insights into users’ daily lives, routines, and interactions with your product.

Open card sorting

Open card sorting is commonly used to understand how users categorize and organize information. In an open card sort, participants are given a set of cards and asked to group them into categories that make sense to them. 

This method allows you to gain insights into users’ mental models, how they perceive relationships between information, and how they organize and prioritize content. It’s a useful technique when you want to generate new insights, uncover patterns, and inform the information architecture and organization of your product or website.

Evaluative research

Evaluative research, also referred to as evaluation research, is used to assess and improve products or concepts by gathering data and insights. Its primary goal is to determine if an existing solution meets user needs, is easy to use, and provides an enjoyable experience.

This type of research is typically conducted in the early stages of the design process and continues iteratively throughout the development lifecycle, from initial concept designs to the final product. By conducting evaluative research at various stages, you can identify potential issues and make enhancements to the user experience.

The evaluative research process doesn’t stop once a new product is launched, though. For the best possible user experience, solutions should be continually monitored and improved based on customer feedback and evolving needs. This ongoing evaluation helps you identify areas for improvement and ensures that the product remains relevant and effective.

When it comes to evaluative research, there are two main methods: formative and summative. Formative research takes place earlier in the development process and focuses on identifying and resolving specific usability issues. By gathering feedback and observing user behavior, you can pinpoint areas that require improvement and make iterative adjustments to optimize the user experience.

Summative research typically happens toward the end of the design process, where the focus is on evaluating the overall performance and impact of the product. These studies help determine if the solution meets predetermined goals and objectives.

By using both summative and formative evaluative research, you can gain valuable insights into how users interact with your product. You can then use this information to make data-driven decisions, leading to more user-centered designs and improved overall user satisfaction.

What are the advantages of evaluative research?

Evaluative research offers several advantages that contribute to improving the user experience and fostering user loyalty. 

Identify usability issues

This might sound like an obvious one, but evaluative research helps to identify usability issues and challenges with your product. Through methods like usability testing and heuristic evaluation, you can uncover obstacles, pain points, and areas of confusion that can impact the user experience.

Increase user-centered designs

Evaluative research puts the user at the center of the design process. Gathering feedback and insights from users helps to ensure that design decisions are based on real-world user experiences and preferences. This increases the likelihood of creating products that meet user  needs and expectations.

Make iterative improvements

Evaluative research promotes an iterative design process. By continuously evaluating and testing your designs, prototypes, and features, you can gather feedback and make incremental improvements over time. This approach helps to ensure that your final product is refined based on user insights and ongoing feedback.

Increase user loyalty

By incorporating evaluative research methods into your design process, you can improve the user experience, leading to increased user satisfaction and loyalty. When users feel their needs are being met, they’re more likely to engage with your product, recommend it to others, and become loyal customers.

Evaluative research methods

When conducting evaluative UX research, it’s important to choose a method that aligns with your research goals. Below is a summary of some of the most common evaluative research methods.

Usability testing

Usability testing involves testing your product or prototype with real people. You set a series of specific tasks for users to complete and collect data on their performance, efficiency, and satisfaction. This helps you to identify usability issues, gather user feedback, and evaluate the overall user experience.

Read more: We have a thorough guide to usability testing, if you’d like to read more on this research method.

Surveys

Surveys are a good way to collect quantitative and qualitative data from a large number of users. This is useful for gathering feedback on a specific aspect of your product or design, measuring user satisfaction, assessing preferences, or gathering demographic information.

Read more: Our design surveys guide gives you all the information you need to gather considered feedback about your designs.

Closed card sorting

Closed card sorting is another common evaluative research method, as it’s used to understand how users categorize and organize information. In a closed card sort, you present a set of labeled cards – such as navigation labels, features, or content elements – and ask participants to sort them into predefined categories. It’s a useful method when you want to test your users’ understanding and evaluate the effectiveness of existing categories or when comparing different options.

Read more: Check out our card sorting guide for a deep dive into this research method.

A/B testing

A/B testing, also known as split testing, involves comparing two or more versions of a design or feature to determine which one performs better in terms of user engagement, conversion rates, or other relevant metrics. This method encourages data-driven decision making by directly comparing user responses to different variations.

How to choose between generative and evaluative research

By now you’ve got an understanding of what generative and valuable research is, but how do you choose between these methods? Well, it all depends on what your research goals are.

As the table above suggests, generative research focuses on understanding user behaviors, motivations, and pain points to inform product development. It helps identify problems and generate innovative solutions, and is typically conducted in the early development stages to uncover user needs.

On the other hand, evaluative research aims to assess and evaluate existing designs or prototypes to gather feedback and make improvements. It focuses on usability testing and gathering user feedback on specific aspects of a product, and is usually conducted later in the development cycle to refine designs. 

As you’ll no doubt hear us say many times throughout this guide, the best approach you can take is a hybrid one, and conduct both types of research at different stages of the product development cycle. Understanding when and how to apply each approach can lead to more user-centered and effective design outcomes.

Qualitative research

Qualitative research is a valuable approach that allows you to explore the intricacies of user experiences, perceptions, needs, and motivations, and provide rich insights to shape the development of user-centered products and services. 

Unlike quantitative research (more on that below), which focuses on numerical data, qualitative research gathers non-numerical data – like quotes, anecdotes, and observations – that uncover the human element of user experience. By analyzing and interpreting this qualitative data, you can identify patterns, themes, and user personas that inform design decisions, iterate on prototypes, and create solutions that resonate with your target audience. 

One of the strengths of qualitative research is its ability to explain and complement quantitative data. For example, if you discover the average time spent on a checkout page of your ecommerce website has significantly increased, you can use qualitative research to find out why. Through user interviews and usability testing, you might discover that users are confused or frustrated during the checkout process due to unclear instructions and a complex interface. You can use this feedback to improve the checkout flow, such as simplifying the steps, adding clear prompts, and addressing usability issues.

Another strength is that qualitative research is both formative and summative, so it can be conducted at any stage of the design and product development process. During product development, it can help inform design choices and make iterative improvements. It can also be conducted after the final design is implemented to assess its effectiveness and gather feedback for further improvements.

Qualitative research plays an important role in gaining a deep understanding of user perspectives. By exploring perceptions, behaviors, and motivations, you can make more informed design decisions and create solutions that meet the needs and expectations of your users.

Read more: Check out our article on types of qualitative research.

What are the advantages of qualitative research?

By putting your users at the center of your research through qualitative methods, you can design products that truly meet their needs and preferences, resulting in better overall user satisfaction and engagement. Below are some of the advantages that qualitative research offers.

Gain a deep understanding of your users

Qualitative research provides detailed insights into user behaviors, needs, motivations, and preferences. This allows you to delve into the why behind user actions and provide a deeper understanding of user experiences than quantitative research alone.

Make iterative design improvements

Because you can conduct qualitative research iteratively throughout the design process, you can gather feedback on early prototypes, identify usability issues, iterate designs based on user insights, and validate design decisions before releasing the product or feature.

Inform quantitative data

Like we mentioned above, qualitative research can complement quantitative data by providing explanations and context for numerical data. It can help you interpret qualitative findings, uncover underlying reasons or trends, and guide further exploration.

Puts your users at the center

Qualitative research focuses on capturing the perspectives of your users. This helps to ensure that your product is designed with a user-centered approach, leading to better user satisfaction and engagement.

Qualitative research methods

Qualitative research methods offer valuable approaches for gaining deeper insights into user experiences, motivations, and behaviors. Below is a summary of some of the most common qualitative methods.

User interviews

User interviews are a valuable method for collecting qualitative data and gaining a deeper understanding of your users’ motivations, needs, and behaviors. Interviews give you the opportunity to directly engage with users and gather their perspectives, insights, and feedback on your product or service. 

It’s crucial to ask open-ended questions during these interviews to encourage users to share relevant and useful information about their actions and frustrations. Whether the interviews are structured or semi-structured, the goal is to facilitate open-ended discussions that provide valuable insights for informing design decisions.

Read more: As a seasoned user researcher, Michele Ronsen knows a thing or two about user interviews. Check out the questions she asks the most during research interviews and see how they relate to the design and development process.

Qualitative usability testing

During a moderated usability test, you can gather qualitative data about the usability of your product or service by observing users while they perform a specific task, listening to their thought process and asking questions. 

If you’re conducting unmoderated testing, you can add different types of questions (like long answer or short answer) throughout your test to gather similar types of feedback.

Read more: Check out our article on questions to ask during usability testing.

Diary studies

As covered in the generative research methods section, diary studies involve participants maintaining a diary to record their experiences, thoughts, and behaviors using your product over a specified period. This allows you to gain deep insights into how your users interact with your product in their natural environment, and gather qualitative data that can uncover patterns, trends, and perspectives.

Focus groups

Bringing together a group of users to facilitate discussions and gather collective insights allows for interactive exchanges between participants, providing a broader understanding of shared experiences and opinions.

Quantitative research

Quantitative research involves collecting and analyzing numerical data to understand user behavior, preferences, and patterns. It focuses on gathering measurable and objective data that can be analyzed to draw conclusions and guide data-driven decision making.

To conduct quantitative research, you typically use large sample sizes to ensure the statistical significance of the data. You can conduct surveys and usability tests using a research tool like Lyssna, which automatically records and analyzes data and calculates metrics like time on task, error rate, or success rate so you can assess usability, identify trends, and make informed decisions.

This type of research is useful when evaluating an existing product or assessing a final design. It can be used at the beginning of a design cycle and then iteratively throughout to provide insights into usability.

Quantitative research complements qualitative research by offering a broader perspective and validating qualitative insights, creating a more comprehensive understanding of the user experience.

Read more: Check out our article on types of quantitative research.

What are the advantages of quantitative research?

Quantitative research offers several advantages that contribute to data-driven decision making and objective insights.

Numerical insights

Quantitative research gives you numerical data to analyze and measure usability, such as completion rate, task success rate, and time on task. By collecting from large sample sizes, you can also gather statistically significant results. This can be a compelling tool when presenting findings and making recommendations to your stakeholders.

Easier to avoid human bias

Quantitative research focuses on measurable and objective data, reducing cognitive biases and personal interpretations. This objective nature of data collection helps ensure that findings are based on concrete evidence rather than individual opinions or perceptions.

Data-driven decision making

Similarly, quantitative research encourages data-driven decision making processes. By analyzing the data you collect, you can identify trends and patterns that can inform design improvements, feature enhancements, and product iterations.

Ability to make comparisons

Quantitative research facilitates comparisons between different user groups, product variations, and design iterations. By establishing benchmarks, you can measure the effectiveness of design changes over time and track improvements in usability and user satisfaction.

Efficiency

By using automated research tools, you can conduct quantitative research efficiently and at scale. This enables you to collect data from a large group of participants within a relatively short period of time, enhancing the efficiency of your research processes.

Quantitative research methods

Quantitative research methods offer effective ways to collect numerical data and measure user interactions. Below are some common quantitative methods.

Surveys

Surveys are a popular way to collect quantitative data. They’re relatively easy to set up and send to your target audience, allowing you to collect data from a large group. Because surveys include structured questions with predefined response options, you can collect consistent data like frequencies, percentages, ratings, and ranking for numerical analysis and measurement. 

A/B testing

A/B testing is a method where you compare two or more versions of a design or feature to see which one performs better. By randomly assigning users to different groups and showing them different versions, A/B testing allows you to collect objective, numerical data on user interactions and preferences.

Quantitative usability testing

Usability testing methods like first click tests and five second tests can be used to gather numerical data like time on task, completion rate, and success rate. You can use this data to set a benchmark as you make improvements to your product over time. 

It’s worth mentioning that these methods are often gathered alongside quantitative insights, such as a participant’s reasoning, thoughts, and expectations. The combination of quantitative and qualitative insights can offer a more comprehensive understanding of user behavior and provide valuable feedback for improving a design’s usability and effectiveness.

Analytics and tracking

Analyzing data from website or app analytics tools can provide valuable quantitative insights. These tools collect data on user interactions, such as page views, clicks, and conversion rates. By analyzing this data, you can identify user patterns, behaviors, and preferences.

How to choose between qualitative and quantitative research

When it comes to choosing between qualitative and quantitative research, it’s important to understand the differences and consider the pros and cons of each approach. As the table above shows, quantitative research focuses on numbers and statistics, while qualitative research delves into the meaning to understand the “why” behind user behavior.

Ideally, a mixed methods approach where you combine qualitative and quantitative research is recommended to obtain a comprehensive understanding of the usability of your product. However, in certain situations where you’re pressed for time or lack resources, you might opt for one method over the other.

The reason a mixed methods approach is considered best practice is because it allows you to leverage different insight sources. For instance, you can begin in the discovery phase with qualitative research, conducting user interviews to uncover people’s needs and preferences. Based on these insights, the product team could then move into developing a low-fidelity prototype and testing this using a combination of qualitative methods like interviews and surveys, gathering feedback to iterate and improve the design. Then, you can conduct quantitative user testing on the final design to ensure it’s intuitive and meets user expectations, and identify any potential issues before moving into the product development phase.

One advantage of using a research tool like Lyssna is that it allows you to incorporate both qualitative and quantitative methods in a single test. For example, you can run a prototype test to gather numerical data such as the percentage of participants who successfully complete a task, average clicks (including misclicks), average time to complete the task, and the exact number of participants who reached the goal screen. Additionally, you can include questions to gather qualitative insights, like asking participants about the ease or difficulty of the task, their likes or dislikes about the design, and their overall thoughts on the product.

By combining qualitative and quantitative research methods, you can gather a broad understanding of user experiences, make informed decisions, and create user-centered solutions that meet the needs and expectations of your target audience.

Attitudinal vs behavioral research

Two more UX research methods worth exploring are attitudinal and behavioral UX research, which are used to understand user behavior and preferences. While they serve different purposes, both methods offer valuable insights when used together.

Attitudinal research focuses on what people say and aims to measure their beliefs, attitudes, and feelings toward a product or experience. It delves into users’ preconceived notions and gathers qualitative insights through methods like interviews, focus groups, surveys, and diary studies. 

By asking questions about users’ thoughts, opinions, and perceptions, attitudinal research uncovers valuable insights that help to understand user motivations and preferences. For example, asking users why they like or dislike a particular product feature before they use it provides insights into their expectations and initial impressions.

On the other hand, behavioral research focuses on what users do and provides quantitative data on their interactions with a product or website. This approach measures user actions and behaviors, such as where they click or how they navigate through a site. 

Methods like eye tracking, A/B testing, and usability testing are used to observe and measure user behavior objectively. Behavioral research helps answer questions about user actions and provides insights into user engagement, conversion rates, and usability issues.

To gain a comprehensive understanding of your users, it’s worth combining attitudinal and behavioral research methods. By conducting both types of research, you can capture a more well-rounded picture of your users’ preferences, motivations, and behaviors. 

Behavioral research provides insights into user behavior, and attitudinal research helps uncover the why behind those actions. By combining both approaches, you can develop a deeper understanding of your users and create user-centered experiences that align with their needs and preferences.

Remote vs in-person research

When conducting UX research, one important decision to make is whether to opt for remote or in-person methods. 

Remote UX research offers several advantages, including cost-effectiveness, global reach, convenience for participants, and the ability to gather insights from users in their natural environments. Remote methods, such as moderated or unmoderated usability testing, user interviews, and online surveys, allow for flexible scheduling and accommodate participants’ diverse locations and time zones.

In-person UX research provides its own set of unique benefits as well. It enables researchers to observe and interact with participants directly, capturing nuanced non-verbal cues and body language that may be missed in remote sessions. In-person methods like moderated usability testing, contextual inquiries, and focus groups offer the advantage of immediate feedback, deeper rapport-building opportunities, and the ability to create a controlled testing environment.

The choice between remote and in-person research depends on various factors, such as your research goals, budget, target audience, and timeline. It’s important to consider the specific requirements of your study and weigh the pros and cons of each approach. In some cases, a hybrid approach that combines both remote and in-person methods may be the most suitable option. By selecting the appropriate research method, you can effectively gather valuable insights from users and make informed design decisions that enhance the user experience.

Choosing the right UX research approach

In this chapter, we’ve explored various UX research approaches, each offering unique advantages and insights. 

We began with generative research, emphasizing its role in understanding user motivations, pain points, and problem framing early in the development process. Then, evaluative research took the spotlight, highlighting its iterative nature and ability to assess existing solutions for user-friendliness. 

Qualitative research was discussed as a means to uncover the “why” behind user behaviors and complement quantitative data. Quantitative research, on the other hand, was presented as a way to gather numerical insights efficiently and make data-driven decisions.

The chapter also covered attitudinal and behavioral research, showcasing how they offer complementary perspectives on user behavior and preferences. Remote and in-person research was also explored, weighing the pros and cons of each method.

Here are a few key takeaways from this chapter:

• Research alignment: Choose research methods that align with your specific goals and the stage of your project. Generative research for understanding, evaluative for assessment, qualitative for depth, and quantitative for breadth.

• Complementary insights: Combining qualitative and quantitative methods often leads to a more comprehensive understanding of user experiences, allowing you to uncover not only what users do but also why they do it.

• Remote vs. in-person: The choice between remote and in-person research depends on your research objectives, budget, and logistical constraints. Consider the pros and cons of each method, and don't rule out a hybrid approach for the best of both worlds.

• Iterative process: Where possible, UX research is an iterative process that should continue throughout the product development lifecycle, from the initial idea to post-launch evaluation. Using a mix of research methods at different stages can lead to more user-centered and effective design outcomes.

Regardless of the chosen research method, always keep the user at the center of your research. This ensures that your products and services meet their needs, ultimately leading to higher user satisfaction and engagement.