Incrementality for PMax: How to Run Holdout Tests and Interpret the Results

Performance Max campaigns have become a central element of modern Google Ads strategies, yet their automation often makes it difficult to understand how much real value they generate. Marketers frequently see rising conversion numbers without knowing whether those conversions would have happened anyway. Incrementality testing addresses this problem by measuring the true additional impact created by advertising. Holdout experiments are one of the most reliable methods for evaluating the real contribution of PMax campaigns and separating genuine growth from attribution noise.

Why Incrementality Matters in Performance Max Campaigns

Performance Max relies heavily on machine learning to allocate budgets across search, display, YouTube, shopping inventory and other placements. While this automation improves efficiency, it also blurs the line between incremental demand and conversions that would have occurred organically. Standard attribution reports often overstate the impact of campaigns because they credit interactions that happened late in the customer journey.

Incrementality testing focuses on measuring the difference between two comparable groups: one exposed to advertising and one intentionally excluded from it. The gap in performance between these groups reveals the real additional conversions generated by the campaign. In other words, incrementality shows how many results would disappear if the campaign stopped running.

For PMax campaigns this distinction is particularly important. These campaigns frequently capture branded searches, remarketing traffic and returning visitors. Without incrementality analysis, marketers may assume that the campaign is driving growth when in reality it is only intercepting users who already intended to convert.

Typical Measurement Challenges in Automated Campaigns

Automation in advertising reduces manual optimisation but also reduces transparency. Performance Max decides where ads appear and how audiences are targeted, making it difficult to determine which part of the system actually produces value. Standard reporting interfaces rarely reveal whether the campaign influenced a user or simply recorded a conversion that would have happened anyway.

Another challenge involves overlapping targeting between different campaigns. For example, a brand search campaign, a remarketing campaign and a PMax campaign may compete for the same users. Attribution systems often credit whichever campaign interacted last, which can distort performance evaluation and lead to inflated results.

Incrementality experiments solve these issues by introducing a controlled comparison. Instead of analysing attribution alone, marketers observe real behavioural differences between groups that see advertising and groups that do not. This approach provides a clearer picture of true campaign impact.

How to Set Up Holdout Tests for PMax Campaigns

A holdout test divides traffic or audiences into two segments: a test group that receives advertising and a control group that does not. Both groups should be similar in size, behaviour and historical performance. The closer these groups resemble each other, the more reliable the experiment becomes.

For PMax campaigns the most common method is a geographic holdout test. Certain regions continue receiving advertising while others are excluded from the campaign. After a defined period, marketers compare metrics such as conversions, revenue or sign-ups between both groups.

The duration of the test is also important. Experiments that run for only a few days rarely produce reliable insights because normal fluctuations can distort results. In most cases, running a holdout test for at least three to four weeks allows the campaign’s algorithm to stabilise and generate statistically meaningful differences.

Best Practices for Reliable Experiment Design

Randomisation is essential when assigning control and test groups. If regions with different purchasing power or seasonal patterns are unevenly distributed between groups, the results may reflect economic differences rather than advertising impact. Careful selection of comparable markets helps reduce this risk.

Budget stability during the test period is another critical factor. Large changes in advertising spend, creative assets or targeting signals can introduce new variables that affect outcomes. Ideally, the campaign structure should remain consistent throughout the experiment.

It is also advisable to track multiple performance indicators rather than relying on a single metric. Conversions, revenue per user and customer acquisition cost together provide a more balanced understanding of incremental value than conversion volume alone.

Interpreting Incrementality Results Correctly

After the testing period ends, the next step is to compare the performance of the test and control groups. The key question is how large the difference is between them. If the group exposed to PMax advertising generates significantly more conversions than the holdout group, this gap represents incremental lift.

For example, if the test regions produced 1,200 conversions while the control regions produced 1,000 conversions under similar conditions, the campaign generated approximately 200 incremental conversions. This figure reflects the additional demand created by advertising rather than existing demand captured through attribution.

However, marketers should also consider the cost required to produce this lift. Incremental cost per acquisition is calculated by dividing the additional spend by the incremental conversions. This metric reveals whether the campaign produces growth at a sustainable price.

Turning Incrementality Insights into Strategy

Incrementality results can guide strategic decisions about budget allocation and campaign structure. If a PMax campaign produces strong incremental lift at an efficient cost, increasing the budget may lead to further growth. Conversely, weak or insignificant lift may indicate that the campaign primarily captures existing demand.

Another valuable insight involves identifying where automation performs best. If incremental growth appears mainly in new customer segments or non-brand traffic, marketers may choose to prioritise those signals in campaign settings while reducing overlap with branded campaigns.

Regular testing is also essential because market conditions change over time. Consumer behaviour, competition and seasonality can all influence incremental performance. Running structured holdout experiments periodically allows marketing teams to maintain an accurate understanding of how much real value their PMax campaigns deliver.