Smart insights: Comparative analysis

Comparative analysis helps you understand how different groups of people responded to your form. Instead of switching filters, comparing charts manually, and exporting data, comparative analysis analyzes group differences for you and highlights where respondents behave differently.

Here's what comparative analysis looks like. Keep reading for a full breakdown.

This feature is available on Growth Essentials, Growth Pro, Growth Custom, and Enterprise plans.

Comparative analysis will appear only if your form includes at least one multiple choice question and one numeric question (NPS, Opinion Scale, Rating, or Number) and if your form has collected 10 responses. Learn more about question types here.

Comparative analysis uses a standardized statistical method to compare groups fairly across different question types. It does this with a measure called Cohen’s d, which helps identify whether differences are practically meaningful. 

You can find a full explanation of how Cohen’s d works at the end of this article.

Want to see what other features are available in Smart Insights? Check out this article.

What comparative analysis does

Comparative analysis looks for differences in how predefined groups responded to numeric questions. Groups are based on the answer options in your multiple choice questions, and comparative analysis checks whether people in different groups scored higher or lower on questions like NPS, opinion scales, ratings, or number.

For example, you might see how different age groups rated your content, or whether people with different job roles found your resources equally useful.

To do this, comparative analysis calculates an effect size, which is a way of measuring how big the difference is between two groups. Comparative analysis uses a statistical measure called Cohen’s d to standardize these differences across different numeric question types. Based on this effect size, each comparison is sorted into Big, Moderate, Small, or None, so you can quickly understand how meaningful the difference is. You can find a full explanation of Cohen’s d at the end of this article.

Supported question types

Categorical (used to form groups)

• Multiple choice (single select and multiple select)

Numeric (used as outcome values)

• NPS
• Opinion scale
• Rating
• Number

Comparative analysis compares every answer option from every multiple choice question with every numeric question. It checks every possible combination behind the scenes. If your form includes several multiple choice and numeric questions, you may see many insights. Some will be more meaningful than others. You can browse them and focus on the ones that are relevant to you.

How to access comparative analysis

Before you can access the comparative analysis features in Smart Insights, check the following based on your plan:

• Growth Essentials, Growth Pro and Talent plans: You will not see the AI-Powered Insights toggle in your Admin settings. No action is required. Smart Insights is available by default, and you can access it directly in the Smart Insights tab.
• Enterprise and Growth Custom plans: You'll see an AI-Powered Insights toggle in your Admin settings, and it is turned off by default. Turn it on to access Smart Insights. This setting will apply to all forms and workspaces within your organization. 

You must be an Admin or Owner in your organization to turn the AI-Powered Insights option on or off.

If you change your Typeform plan and have not previously altered the default settings for AI-Powered Insights, your account will automatically adopt the default settings of the new plan. However, if you have previously modified these settings, your new plan will retain the settings you selected.

Your data won't be sent to the AI model unless you actively use the feature (even if the toggle is enabled). Specifically, the data transmission and analysis are initiated when you click on the Smart Insights tab.

You can find comparative analysis in Smart Insights for any form that includes at least one Multiple Choice (categorical) question and one numeric question.

1. Open your form and click on the Results tab.

2. Select Smart Insights.

3. Scroll down to the Comparative analysis section.

How to read your insights

On the right side of the page, you can see when the comparative analysis was last updated. If your form has collected new responses, you can select Refresh comparative analysis to rerun the calculations with the latest data. 

Please note that this may take some time, depending on the number of insights and the amount of data being compared.

Below you'll see the key differences between groups. You can switch between the Big, Moderate, Small, and None tabs to explore insights based on the size of the difference. These categories are based on the effect size calculated using Cohen’s d, which helps identify how meaningful each difference is.

Each tab shows the number of insights found for that category and includes all possible combinations of your categorical and numeric questions.

When looking at differences between groups, you'll see insights that summarize how different groups responded. Each insight shows:

• A short headline describing the difference between groups and the percentage difference between them
• The average score for each group, shown side by side
• How each group is defined based on the answers selected
• Which questions were used, including the grouping question and the numeric question
• Where one group scored higher, so you can quickly understand what changed between groups

For example, our form includes the questions Which kind of hotdog did you go for? and How would you rate the value for your money?, and we see an insight showing that people who chose Classic Frank rate value for money 140 percent higher than those who chose Cheesy Bite. 

You also see the average score for each group, 4.0 vs 1.7, so you can understand the size of the difference at a glance. In our example, it suggests that people who ordered Classic Frank felt they received much better value, while customers choosing Cheesy Bite were less satisfied. 

For our hotdog stand example, insights like this can help us explore whether certain menu items might need improvement, a price adjustment, or a clearer description so customers know what to expect.

Select See all differences to view more insights in that category.

How Cohen’s d works

Comparative analysis uses Cohen’s d to standardize the difference between groups. This method keeps comparisons fair and consistent even when numeric questions use different scales.

What Cohen’s d measures

Cohen’s d shows how far apart two group averages are relative to the variation within the groups. This helps identify meaningful differences rather than minor variations.

The formula

Where:

•  x̄₁ and x̄₂ are the means of sample 1 and 2.
• sₚ is the pooled standard deviation.

Pooled standard deviation for unequal sample sizes ( n₁ ≠ n₂ )

When the two groups have different sample sizes, n₁ and n₂, the formula for the pooled standard deviation becomes a weighted average of the two sample variances, with the weights based on each group's degrees of freedom. 

Where:

•  n₁ and n₂ are the sample sizes of the two groups.
• s₁² and s₂² are the variances of the two samples.

This produces a standardized effect size that can be compared across different numeric question types and makes the comparison balanced and interpretable.

Interpreting effect sizes

We use industry-standard guidelines to interpret the effect size:

• Big difference: d greater than 0.8
• Moderate difference: d greater than 0.5
• Small difference: d greater than 0.2
• None: d 0.2 or below

Effect size focuses on the practical size of the difference rather than statistical significance. This standardization improves the interpretability of results and supports accurate comparisons across groups. Practical significance focuses on whether the effect is substantial enough to matter in real-world contexts.

Check out this site to learn more about Cohen’s d.