Your ad platforms all claim credit for the same sale. Google says search drove it, Meta says the retargeting ad did, and your email tool counts it too. Add the claims together and they often exceed your actual revenue. The question every marketing budget really depends on is simpler and harder: would this sale have happened anyway?
Incrementality testing answers that question with an experiment. By deliberately withholding marketing from a comparable group of people or places and comparing outcomes, you measure the extra conversions your marketing actually caused. This guide explains how incrementality differs from attribution, the main types of tests, how to run one step by step and how to read the results without fooling yourself.
Key Takeaways
- Incrementality is the difference in outcomes between people exposed to marketing and a comparable group who were not.
- Attribution assigns credit; incrementality testing measures causation. You need both.
- Common designs are user-level holdouts (conversion lift), geo experiments and on/off tests.
- Plan sample size, test length and the success metric before you start, and do not change them mid-test.
- Use results to calibrate attribution and marketing mix models, not just to judge one campaign.
Incrementality vs attribution
Marketing attribution looks at the paths of people who converted and distributes credit among touchpoints. It cannot see what would have happened without a given touchpoint. Retargeting and branded search are classic examples: they reach people who were already likely to buy, so attribution can make them look far more valuable than they are.
Google's Ads documentation on Conversion Lift puts the purpose of incrementality experiments plainly:
“It's the way to measure the causal impact of ads.”
— Google Ads Help, About Conversion Lift
| Attribution | Incrementality testing | Marketing mix modeling | |
|---|---|---|---|
| Question answered | Which touchpoints were on the path? | Did this activity cause extra outcomes? | How much did each channel contribute over time? |
| Method | Rules or algorithms on user paths | Controlled experiment | Statistical model on aggregate data |
| Speed | Continuous | Weeks per test | Periodic refresh |
| Weakness | Credits correlation, not causation | One channel or question at a time | Needs history; correlational without calibration |
For the third column, see our plain-English guide to marketing mix modeling.
How an incrementality test works
Google's Think content on incrementality testing describes it as a randomised, controlled experiment with two groups of people, one exposed to the campaign and one not. The difference in conversions between them is the lift.
Incremental conversions = conversions (test group) − conversions (control group, scaled to the same size)
Lift (%) = incremental conversions ÷ control group conversions × 100
Incremental ROAS = incremental revenue ÷ ad spendIncremental ROAS is often much lower than platform-reported ROAS. That gap is the point: it shows how much reported performance would have happened anyway.
Types of incrementality tests
| Test type | How it works | Good for | Watch out for |
|---|---|---|---|
| User-level holdout (conversion lift) | The platform randomly withholds ads from a control group of users | Meta, Google, YouTube campaigns | Availability and minimum spend vary; you rely on the platform's measurement |
| Geo experiment | Run or increase marketing in some regions, hold out matched regions | Any channel, including TV, radio and offline sales | Needs enough comparable regions and volume |
| On/off (time-based) test | Pause a channel for a period and compare to before/after | Small budgets, quick reads | Seasonality and other changes confound results |
| Ghost ads / PSA control | Control group sees an unrelated ad or is logged where an ad would have shown | Display and video | Mostly run by platforms |
Google Ads Conversion Lift studies can be based on users or on geography, and Google notes the feature is not available for all accounts. Meta has open-sourced GeoLift, a tool for calculating lift at a geo level, which is useful when you want to run your own regional tests.
How to run your first incrementality test
- Pick a decision. For example: "Should we keep spending on branded search?" or "Is retargeting worth its budget?" A test without a decision attached rarely changes anything.
- Choose the success metric. Revenue, orders or qualified leads from your back end or CRM, not platform-reported conversions.
- Select the design. Use platform lift tools if available; otherwise a geo test with matched regions.
- Estimate power. Check you have enough volume to detect a realistic lift. Low-volume tests often end with "inconclusive".
- Freeze other changes. Avoid major price changes, promotions or creative overhauls in only one group.
- Run for a full buying cycle. Include enough time for delayed conversions to occur.
- Analyse and decide. Calculate lift and incremental ROAS with confidence intervals, then act: scale, cut or restructure.
- Repeat. Results change with creative, competition and seasonality. Re-test important channels periodically.
Within Google Ads, simpler questions such as bidding strategy or landing page variants can be answered with Google Ads experiments, which split traffic between two campaign versions, much like a classic A/B test. For paid social creative questions, see our guide to Meta creative testing.
How to read the results
A test result is an estimate with uncertainty, not a single truth. Report the lift together with its confidence interval, and be honest when the interval includes zero: that means the test could not show a reliable effect, which is different from proving the channel does nothing.
- Clear positive lift with healthy incremental ROAS: keep or scale the channel, and test a higher budget next.
- Positive lift but weak incremental ROAS: the channel works but costs too much; test cheaper targeting, creative or bids.
- No detectable lift: reduce spend gradually and watch your sales data, or redesign the test with more volume.
Feed every result back into how you read platform dashboards. If retargeting shows a fraction of its reported ROAS in a lift test, discount its attributed numbers accordingly in future reports.
Which channels to test first
- Branded search. Often captures people who would click your organic listing anyway. See bidding on brand keywords.
- Retargeting. Reaches people already close to buying; incrementality can be low.
- Upper-funnel video and social. Attribution under-credits them; tests can reveal hidden value.
- Affiliates and discount codes. Coupon sites can intercept sales that were already happening.
Common mistakes in incrementality testing
- Too little volume. An underpowered test cannot tell a real lift from noise.
- Ending the test early because the first week looks good or bad.
- Contaminated control groups. In geo tests, national campaigns or spill-over between neighbouring regions blur the comparison.
- Measuring with platform conversions only. Use a neutral source of truth, ideally your sales data.
- Treating one result as permanent. A retargeting test from two years ago does not describe today's account.
- Ignoring long-term effects. Short tests capture immediate sales, not brand building; pair them with longer-term measures such as share of search.
Related Guides
- Marketing Mix Modeling: A Plain-English Guide
- Using Regex in Search Console: Filters That Save Hours
- How to See Organic Search Keywords in GA4
Frequently Asked Questions
What is incrementality in marketing?
It is the extra outcome, such as sales or leads, that happened only because of a marketing activity, measured by comparing an exposed group with a similar unexposed group.
How long should an incrementality test run?
Long enough to cover at least one full buying cycle and reach the sample size you planned. For many advertisers that means several weeks.
Can small businesses run incrementality tests?
Yes, but low volume makes results noisy. Simple geo or on/off tests on a high-spend channel, run for longer, are usually the most practical starting point.
Is incrementality testing the same as A/B testing?
They are related. A/B tests compare two versions of something; incrementality tests compare marketing against no marketing (or less) to measure causal impact.
Does incrementality replace attribution?
No. Attribution helps with day-to-day optimisation, while incrementality tests check whether attributed results are real and help calibrate models.
Conclusion
Incrementality testing is the most direct way to know whether your marketing is creating sales or simply standing next to them. Start with one important budget question, design a clean test, measure with your own sales data and use the result to recalibrate how you read attribution. If you want help building a measurement plan and paid media programme around real incremental results, see our PPC management services or get a quote.
