The last few articles I’ve written have been very specific and about iOS14, so I’ve decided to widen the lens and write more broadly about how anyone can run incrementality testing. This is a concept that I find to be immensely pivotal for any growth engine. In a sense, this is connected with iOS14 as more of these tests will be paramount to validate paid channels + campaigns with attribution degradation. This test — plain and simple — allows companies and growth teams to determine the effectiveness of their growth marketing initiatives.
It’s weird because when I started my first growth role, I vividly recall how much emphasis there was on incrementality from specific colleagues whereas it didn’t seem to matter to others. It was very much black or white — truly a stark distinction. However, this easily could have been a matter of prioritization because more and more started to pay attention to this (including myself) as we started to scale.
There’s so much to unpack with incrementality testing but I want to laser in on how to leverage the power of this test. No matter how big your growth team and the resources at your disposal, I’ll showcase how you can add this test to your growth arsenal.
Whether you’re reading this because you’re just learning about incrementality testing or are an incrementality test veteran, the value of using this test is necessary to growth marketing. You can be spending thousands or millions on any given channel, but if your users would have converted regardless of that ad spend, was it worth spending that? This is why measuring incrementality for each channel provides companies with actionable insights on where to throttle or ramp spend.
Take the example of spending $2,500 on Facebook + $2,500 on Google for remarketing campaigns with the goal of subsequent purchases. If we ran an incrementality test and found that Facebook had 80 incremental conversions whereas Google had 110 — we would want to spend more of our budget on Google. So how do we figure out which channels are more incremental?
It may seem daunting to set one of these tests up but I’ve broken it down into three easy buckets below:
In the above example, we can see that during our test period between Jan 15–21, we still had 50 conversions with Facebook ads paused. Meanwhile, when running our Facebook ads we had 150 conversions. This tells us that Facebook is 200% incremental. It’s a simple example, but this analysis gives us a new baseline CAC that we can apply and compare with other channels.
If you want to truly measure your ad performance and distribute your budget in an increasingly methodical way, incrementality tests will be your best friend. My hope is that incrementality becomes synonymous with ubiquitous. It should be a tool in every growth marketing engine arsenal, whether the team is one or five hundred.
My hope is that incrementality becomes synonymous with ubiquitous.
I’ll leave everyone with one thought — if you’re running five channels with multiple touch points, do you truly know the impact that any specific one is driving without incrementality?
The results from incrementality tests can be used to form channel scalars and multipliers — which can be used to make a case to cross-functional teams on the importance of marketing efforts, etc. If diving deeper and into the weeds on incrementality testing is something you would like to learn more about, please let me know and I’ll write a follow-up piece.
How does a founder implement a growth framework to scale to their first million dollars in revenue?