Hypotheses in product design

A product designer tells how to formulate hypotheses and use them to improve the interface.

Hypothesis generation is an important step in the design process. When designing interface solutions, the designer is asked, “What can improve the user experience?” and puts forward various hypotheses.

What is a hypothesis, and why is it needed

A hypothesis is an assumption that needs to be tested. The product team uses it to look for ways to improve a product, business process, or interface.

A hypothesis is the starting point for research and data analysis. For example, you might assume that a different button color on a website would increase conversions by 10%. The experiment will help you check if this is true and decide whether to implement or reject the changes.

Here are some tasks that hypotheses will help you with:

• Make a better product. For example, improve functionality, increase conversion, and more.
• Make the interface more convenient: change the location of buttons, forms, or content.
• Determine what needs to be improved first. This helps to focus on the most important changes for users and saves time and resources.

What are the hypotheses?

It is conditionally possible to distinguish product and interface hypotheses.

Product hypotheses reflect the impact of a feature on product metrics and are associated with key indicators: conversion, profit, and user satisfaction.

Examples of product hypotheses:

• The number of orders will increase if we add a new payment method.
• If we change the cost of goods / add a discount, the number of orders will increase.
• If we change the terms of delivery, user satisfaction will increase.

Interface hypotheses also affect key metrics – but they directly relate to the appearance of the feature and interaction with it:

• interface elements – the location of buttons, forms, color palette, and size of elements;
• Structure and navigation – how to navigate the site or in the application, what blocks the sections consist of, how to interact with the menu;
• Content – what kind of text is on the page, headings, images, tone of voice.

Examples of interface hypotheses:

• Adding a Buy Now button to your home screen will increase conversions.

• If you add the “Customer Reviews” block on the product page, user loyalty and the number of purchases will increase.

• The average check will increase if you add the “Related Products” block on the checkout page.

• If you change the color of the “Buy” button to red, the conversion on the product page will increase.

Hypotheses in the design process

1. Define the goal. For example, increase the conversion on the site, the average bill, and the number of purchases. A specific goal will help filter your hypotheses and focus on those relevant to the main goal.

2. We collect data. It can be interviews with users and analysis of Google Analytics data. Based on the new information, highlight problem areas: for example, which pages have a low conversion, high bounce, and few purchases.

3. We formulate hypotheses. Based on the received data, we suggest what changes in the interface can improve performance. The hypothesis must be specific, clearly formulated, and measurable. Determine what metrics and KPIs can be used to measure the effectiveness of the hypothesis.

A well-formulated hypothesis consists of two parts:

• the first part answers the question, “What do we want to test?”;
• the second part is “How do we understand that the hypothesis has worked?”.

Examples of hypotheses:

• If you change the color of the discount element to red, the number of purchases in the category will increase.
• If you add fields for entering a promotional code on the checkout page, the number of purchases will increase.
• Adding a share button on the site page will increase the number of new users.

4. Evaluate the hypotheses. From all the hypotheses, select those that you will test. If you are working on a big feature, there can be many hypotheses – the assessment will help you understand where to direct your efforts right now.

How to rate:

• Your hypotheses are relevant to the intended problem and are based on facts and real data.
• Sort hypotheses by the level of risk and level of impact on users. High risk where there is a high degree of uncertainty – that is, we need more data to rely on. For example, if you don’t know how often users click the “More Info” button, removing it without specific data is dangerous.

If high-risk hypotheses are not tested, they can lead to critical consequences. For example, after introducing a new feature, the user cannot perform the target action.

5. We test hypotheses. After the evaluation, we go to check the resulting list of hypotheses. There are many options for testing; what to choose depends on the hypothesis itself and the context. The task of the researcher and product designer is to choose a testing method that can confirm or disprove the hypothesis.

Possible verification options:

• Qualitative methods – provide information about the user’s interaction with the interface, views, judgments, and preferences. This includes in-depth interviews and focus groups. In most cases, you will need prototypes to conduct a qualitative study.
• Prototypes allow you to test hypotheses and determine how they affect users quickly.
• Quantitative methods are a good way to collect a large amount of data in a short amount of time. This includes surveys, questionnaires, and A/B tests. This will provide digital data that will help confirm or refute hypotheses. For example, using A/B testing, we can compare two versions of an interface and determine whether our hypothesis is effective.

Quantitative and qualitative methods complement each other. Qualitative ones provide a deep understanding and details of the interaction, while quantitative ones confirm this with numbers.

6. We analyze the results. After the experiment, it is necessary to study the data obtained and conclude which hypothesis was confirmed and which was not.

To manage these conclusions, the statuses will help us:

• Confirmed – the hypothesis is confirmed. Congratulations! What you put into your decision has been confirmed, and you are moving in the right direction.
• Refuted – the hypothesis has been refuted. The hypothesis was not confirmed during the test and was rejected. This is also a result – you can change the decision, put forward new hypotheses and test them. Be able to refuse failed hypotheses, no matter how successful, in your opinion, they may be.

What other statuses can be:

• Open – the hypothesis is open. The hypothesis has been created, but its verification has yet to begin.
• In progress is a hypothesis in progress. The hypothesis is in the process of testing and analyzing the results.
• Paused – the hypothesis is paused. Hypothesis testing has been temporarily suspended, for example, due to a lack of resources or a change in priorities.

It is necessary to monitor the status of hypotheses and regularly update them according to the current situation.

What is important to remember

• It is possible to involve, for example, analysts, developers, and marketers in working on hypotheses. This will allow you to look at the problem from all sides, using the different experiences and expertise of the participants, and achieve better results.
• Interface changes can affect different metrics in different ways. For example, displaying a product banner on a page may improve its conversion but affect the number of pages viewed or worsen user loyalty in the long run. Keep in mind key indicators and possible scenarios.

A good hypothesis will allow you to formalize the approach and systematically approach the verification of the solution. Try to carefully check the places where the interface plays an important role for the user.

And also, remember that working with hypotheses is a cyclical process. Track implemented changes and develop new hypotheses to improve the product.