What is incrementality?

A common issue in the mobile marketing industry is that we can’t always distinguish our organic traffic from our paid installs. This can lead to miscalculations with our marketing spend and, in a worse-case scenario, end up leaving marketers paying for installs that would have occurred free of charge.

These issues can be solved by measuring incrementality, showing you the impact of your marketing campaigns and the extent of your organic traffic. With this knowledge, you can discover the cost of each incremental conversion (an install that occurred due to marketing spend) and scale that channel accordingly.

Incrementality testing can be complicated, but if you’re familiar with regular A/B testing, you already have a good starting point.

The basic principle is to segment two audiences (let’s say, Group A and Group B) who show similar behavior, and then only running campaigns for Group B. The installs from Group A will be entirely organic, so any increase seen in Group B will show you the incremental difference caused by your ad spend. For example, you could have the following results:

Group A (control group, showing no ads): 100 installs Group B (exposed group, showing ads): 120 installs

This would suggest that your ad spend caused 20 additional installs. From these figures, you can calculate the lift and incrementality:

• Lift is the increase from Group A to Group B (20 installs, 20% increase)
• Incrementality is the percentage of Group B that converted due to marketing spend (20 installs, 16.7% of Group B total).

So, how much did each incremental conversion cost you? You can calculate this by dividing your ad spend for Group B by the measured uplift. If the campaign cost $100 and 20 installs were proven to be incremental, the cost for each incremental user was $5.

If you were then to slowly scale up this campaign, you could also determine whether an increased ad spend is converting a larger percentage of incremental users, lowering the cost for each install.

The example above may be extremely simplified, but it hopefully gives you a starting point for analyzing incrementality. When implementing these tests yourself, you’ll want to ensure that you fully understand:

• Your primary outcomes: what do you expect to learn from each test?
• Confidence levels: is this test representative of a larger audience?
• Mitigating factors: can we trust these figures? If not, why?

Another thing to consider is that user behavior may change over time, and there’s always a chance your findings won’t reflect future results. For this reason, your strategy needs to be adaptable and your hypothesis should continue to develop over time.