Weighting in modern reports

Weighting lets you adjust your report to ensure that the results of your analysis represent the target population you are studying, or to address biases introduced during data collection.

Note: This feature is currently only available to early access program (EAP) customers. It will be available to all customers in the January 2025 release. For information on joining the EAP, contact your Customer Success Manager.

You can use weighting to make your sample more representative of the target population and correct for sampling bias, by adjusting the proportions of defined subgroups within the total sample.

When used appropriately, the benefits of weighting include:

  • Increased accuracy: It enables you to align your sample data more closely with the target population, which leads to more accurate analysis and conclusions.
  • Reduced bias: It minimizes biases introduced by sampling methods and other factors that cause certain groups to be underrepresented or overrepresented.
  • Results can be easily extrapolated: Your findings are more applicable to your target within the whole population.

You should use caution when using weighting to analyze survey results for the following reasons:

  • Weighting is not a substitute for a well-designed survey and sound research practices. You should only use weighting if the survey design requires it to correct for known biases.
  • If weighting is based on external benchmarks (e.g. census data) that are outdated or inaccurate, weighting can introduce errors rather than correct them.
  • Weighting is not a good option when the difference between groups is large. In general, weighting should not be used to increase the proportion of a sub-group by double, or to reduce it by half.

    You need to review your survey results for outliers and disproportionally small groups. For example, if you receive 100 responses from senior citizens aged 65-79 and 5 responses from those aged 80+, weighting could distort your data by under-representing the importance of the 100 responses and overinflating the importance of the 5 responses.

Choosing between weighting and quotas

Quotas can be used as an alternative method to control for proportions and ensure accurate representation of the target population. Quotas are defined when you build your survey and before the survey data is collected. Quotas work by setting limits on how many respondents are allowed to complete the survey from each subgroup.

Weighting is applied after the responses have been received and works by adjusting the influence of respondents based on how closely the sample aligns with the target population.

When possible it is preferable to use quotas in order to avoid manipulating your survey data, but weighting can assist your analysis in the following situations:

  • You want the flexibility to adjust the data after the survey is complete to ensure that you have a representative sample of your target population.
  • You want the ability to adjust multiple variables simultaneously. For example, you want to adjust the age, gender, and region variables to match your expected distribution. This is much easier to accomplish using weighting.
  • You need to correct for sampling biases caused by certain groups not responding as frequently. Weighting enables you to gain more insight from low incidence, harder to capture groups. These adjustments may be required even when you are using quotas.
  • You have access to accurate, reliable data about the target population you can use to make precise adjustments to the survey results.

For more information on quotas, see Quotas.

Supported types

You can use the following field types as weight variables:

Questions Profile Variables Response Data
  • Single Choice
  • Rating
  • Single Choice
  • Display Type
  • Respondent Locale
  • Response Status
  • Dataset Type

Weighting schemes

A weighting scheme is a saved configuration of the desired proportions of subgroups within your total sample. The weighting scheme includes the following information:

  • The fields selected as weighting variables.
  • The target percentage assigned to each answer for each weighting variable.
  • The filters applied when the weighting scheme is created or updated.
  • If specified, the maximum and minimum weight factor limits.

You can add multiple weighting schemes to your report with the specific settings you require for each part of your analysis.

Filters and weighting schemes

When you create a weighting scheme, any filters currently applied to the report are associated with the weighting scheme. If there are any differences between the filters applied to your report and the filters in the weighting scheme, the discrepancy is highlighted at the top of the report.

To ensure accurate results, you need to either click the Restore Weight Scheme Filters link to the right to apply the filters defined for the weighting scheme, or create a new weighting scheme that uses the currently selected filters and apply it to your report.

Significance testing and weighting

You can combine weighting and significance testing in your report. Weighting enables you to ensure that your data reflects the target population, and significance testing enables you to identify significant differences between groups within the target population.

When you run significance tests on weighted reports, the tests are run on the effective base, the distribution of responses for each group after weighting has been applied. If you want to run tests on the unweighted base, the original distribution of responses to your survey, you must first remove the weighting.

For more information on significance testing, see Significance test calculations in modern reports.