Why your top of funnel metrics are misleading

Threads (by Meta) is the perfect case study on how top of funnel numbers can make things look much rosier than they actually are. Shortly after surpassing 100 million users during launch week, Similarweb reported that time spent on the app dropped by more than 50% from 20 minutes to 8 minutes. While I’m not saying that Threads was a huge bust, it does show that the metrics which really matter are often overlooked by startups.

I’ve worked at startups that needed to focus on capturing market share at all costs, so it didn’t really matter if we acquired low-engaging users, as long as overall user numbers kept growing. But this is not sustainable. We were only able to do this, because of large funding amounts fueled by the chests of Silicon Valley VCs.

Put another way: what all this means, is that your startup should focus on finding the metric that matters, and setting up for optimization success as quickly as possible. Fear not, I’ll show you how to do that here.

Picking your metric carefully

Sure, it’s nice to see overall user sign-up counts going up and to the right. But that’s all it is – nice. There is no guarantee that these users will convert, and there are many examples of startups that grew rapidly at the top of funnel: Groupon, Quibi, etc. These startups increased their user count like it was nobody’s business but it wasn’t sustainable. Quibi, a short-form video platform, signed nearly one million users in its first few days, but then only converted about 8% after the free trials ended.

So, the question you should be asking yourself, is what kind of metric leads to sustained revenue? What’s your north-star metric?

Sometimes it’s not an obvious one, and a sign-up to purchase is not the right answer. Take for an example, Netflix. They should not be using the metric of trial users converting to purchase. While this leads to revenue, is it sustainable? That’s the key question.

Instead, a better metric for Netflix to target could potentially be the average time spent streaming per user. This would accurately reflect the effectiveness of their product and content. By using this metric, it’s safe to assume that these users will be stickier and lead to sustained revenue.

A best practice I’ve utilized to pick correct metrics, is to look at “whale” (high propensity) users and identify what commonalities they have. The actions that they’re performing, are what you want to get all other customers to do.

Optimizing for sustained revenue

Following up on the Netflix example I proposed, if Netflix only cared about sign-ups to purchase, then they would focus on making their sign-up process the easiest in the world. However, this might not lead to sustained revenue for the company. Instead, by focusing on average time spent watching per user, they might focus on the following:

• User recommendation engine
• User interface / experience
• Thumbnails for titles
• Social watching
• Quality titles

A much different strategy than just optimizing for the sign-up flow.

To validate that the right metric has been selected for sustained revenue, these cuts of data should then be measured against LTV (lifetime value) of users on a regular basis.

When I was on the growth team at Coinbase, every test I ran had a hypothesis that ultimately rolled up to the north-star metric we used. If the tests didn’t roll up to that north-star metric, they were quickly deprioritized. We created a document for each test that kept us on track towards optimizing for the metric that mattered to us. Your startup can easily do the same by setting up a Google doc for each individual test which includes the following:

• Hypothesis
• Target metric
• Test start date
• Test end date
• Results
• Learnings

I’d highly recommend setting up dashboards and regular reporting to keep a close pulse on the metric you’ve chosen. Additionally, set up cohorts of users for different levels of engagement, so their behaviors can be tracked. Why do the cohort of users at engagement level X differ from those of engagement level Y? It could be the channel they’re being acquired from, or the features they’re initially using. Analyze this like your startup’s life depends on it – because it most likely does.

Building sustainable growth

It’s crucial to ensure you’re spending the right amount of time in the growth medium that’s bringing you the users who matter most. Instead of purely looking at CPA for paid acquisition, you should focus on measuring more down funnel. The same goes for every other medium (i.e., social media, lifecycle, etc.). Cheap traffic doesn’t typically equate to high quality users.

I’ve often seen that referrals through word of mouth have the highest engagement of users; in this case, you’d want to make your referral program as seamless as possible. Furthermore, you might want to consider making the referral offers more lucrative than your paid acquisition CPAs, if it’s bringing in more revenue. These types of decisions should be constantly evaluated, because economics change (i.e., diminishing returns on paid acquisition) and other potential changes, will call for a shift to your strategy.

In the early stages of growth, with minimal budgets and resources, it may not be fruitful to look at these metrics daily or weekly. The return may be too small. Instead, for your first few hundred users, more of the emphasis should be placed on user research calls and emails. You’ll get more content on what separates different segments of users this way, than by looking at a dashboard with say, 131 users. You need to have significant sums of data to build sustainably in a highly quantitative manner. This is why early-stage startups should use their advantage of being closer to the customer, and building for what continues to bring folks back.

If you’ve selected your sustainable revenue metric and want to learn more, you can read more about why building multiple onboarding experiences is important or what it takes to have a world-class growth funnel.