Conversion Lift Tests, or just lift tests or lift studies, allow you to measure how many of the conversions you get are actually caused by your ads.

While it is next to impossible to find the cause of any individual conversion – why did you buy a new phone: because you saw an ad or because a friend recommended it? –  we can measure the total effect quite accurately with a randomized controlled trial. At a high level, lift tests work like this:

• The audience is split into treatment group that is targeted normally and control group that does not see any ads.
• Meta measures the total number of conversions in both groups.
• Incremental conversions in the treatment group are caused by your ads.

Note that Conversion Lift Tests allow you to measure conversions from people who did not even see any ads. Normal reporting is based on last-click attribution and only allows you to see conversions that happened within 28 days after click or view.

The most common obstacle to running lift tests is that in order to get useful insights, you need to collect enough data, in some cases 2000 conversions or more. This can mean that you need to run the Lift Test for several months, during which time nobody in the control group can see your ads. For more information, see our article on lift testing best practices.

Please note that Smartly currently supports only Conversion Lift Tests. Brand Lift Tests – a lift test with an additional questionnaire to users to measure brand recognition and recollection – can only be created with 's support.

If you want to create a lift test to measure the impact of your advertising campaigns, contact your Smartly Customer Success Manager to get started.

Before you start

Before creating lift tests, you must:

• Select a Business Manager for your ad account in Account Settings.
  
• Connect your account to your Smartly user

Create a lift test

1. In the main navigation, click on Ad Studies.
2. Click Create Ad Study > Lift test.
   If you cannot see the option to create a lift test, contact your Smartly Customer Success Manager.
3. Under Start time, set the time when the lift test starts.
4. Define either the end time or the observation period. For more information about timing, see Duration and observation period.
5. Under Power analysis, adjust the percentages to estimate how many conversions you are likely to need in order to get a statistically significant result. For more information, see Lift test power analysis.
6. Select whether to compare accounts, campaigns or ad sets.
7. To define the study cells, click Add New Cell for each additional cell. For more information, see Study cells.
8. To define the advertising objectives to measure, click Add new objective.
9. Select an objective type, enter a name and define the conversion event. For more information about objectives, see Objectives.

Duration and observation period

Duration defines how long the lift study lasts. The start and end dates, or the duration, can be edited as long as they are in the future: when the date has passed, it can no longer be edited. You can still edit the end date while the test is running, to either extend the duration or end the lift study earlier. However, once the lift study ends it is no longer possible to extend it. Typically you want to set the end time far into the future, and edit it later if needed. The time between the start and end times is called the test period.

The observation period takes place after the test period ends. It does not stop the delivery of your campaigns, but you should do that yourself. During the observation period, no new users are added to the study population. Conversions will be measured from those who were added to the population during the test period (in the control group, users who were considered for delivery during the test period).

The purpose of the observation period is to make sure that late conversions caused by the campaign are also recorded and attributed to the test. Typically the duration of the observation period should not be more than a few days.

If you want to stop ads during the observation period, do it manually.

While the Lift Test is running, the treatment group is delivered normally but the control group does not see any ads from campaigns included in the Lift Test. Make sure you do not accidentally target the same audience in campaigns that are not included in the Lift Test. Otherwise, the control group will not be a proper control, and the measured effect will appear smaller!

Power analysis

Read about power analysis in more detail: Lift Test power analysis

Study cells

The simplest Lift Test has only one study cell split into a treatment group and a control group. This setup allows you to test how big of a lift is caused by the test campaign(s) compared to not running them. If you want to measure the lift of all campaigns, you can include the entire ad account in the cell.

In general, you can create one or more cells and assign them treatment and control percentages. Then you choose which campaigns, ad accounts or ad sets for each cell.

If you have more than one cell you will in effect get a combined Split & Lift Test. This setup allows you to both measure the lift in each cell, as well as compare the cells' lift against each other. However, you will have less audience in each cell and therefore it takes more time to collect enough data to get a statistically significant result.

The sum of treatment and control group sizes across all cells must be 100%.

Objectives

In order to measure lift we need to know

• how many conversions we get from those who see the ads, and
• how many conversion we get from those who do not see the ads.

Normally does not record the latter, which is why you need to tell beforehand what you want to measure.

Each objective records the results for one or more conversion event. Lift Test results are available separately for each objective. Objective type is either Sales, Non-Sales or Mobile App Installs. For Non-Sales objective you get results for the number of buyers and the number of conversions. For Sales objectives you get results also for the amount of sales revenue. For Mobile App Installs objectives, you simply define your app, and measures the number of app installs.

You can have as many objectives as you want and typically you want to create a separate objective for each conversion event in the sales funnel.

If you include multiple conversion events in a single objective, the result will include the sum of all events, and it is not possible to split those afterward. However, the combined objective might give you statistically significant results earlier as it has more data. The typical use case for this is when you have both in-app and web sales, and you are interested in the total amount of sales. In addition to this combined objective you can also create an objective for each conversion event separately to get results also for each individual event.

When specifying an objective make sure you choose an event from the correct pixel or app (see screenshot below). Alternatively, you can combine events from multiple pixels or apps into a single objective, as described above.

Results

Lift test results are available by clicking the ad study name in the Ad Studies page. For more details on interpreting the results, see Lift test results.

Limitations

• Due to tracking limitations introduced in iOS 14, you can't create lift tests targeting iOS 14 and above. Campaigns targeting these systems may appear disabled for lift tests.
• Changes to Cookies Consent Prompt and Privacy Controls in Europe may affect your ads and reporting, and have a direct effect on lift studies.