Lesson 43: Funnel Analysis
Lesson 43: Funnel Analysis
Lesson 42's Case Study ended with a company replacing a flawed North Star Metric ("total registered accounts") with a better one: the share of new accounts reaching a defined activation milestone within their first 30 days. That replacement metric raises an immediate, practical question this lesson answers: activation doesn't happen all at once — it's the end result of a user moving through a specific sequence of steps, and understanding where users drop out of that sequence is essential to actually improving the metric, not just measuring it.
This lesson introduces funnel analysis, the discipline of breaking a multi-step user journey into its component stages and measuring conversion between each one. This matters because aggregate metrics, however precisely defined (Lesson 41), can hide enormous variation in where a product is actually succeeding or failing — a 20% overall signup-to-activation rate could mean many different things depending on whether the biggest drop-off happens at step one (visiting a page) or step four (completing a specific setup task), and the right fix is completely different in each case. Funnel analysis is how a PM finds out which story is actually true, rather than guessing.
Learning Objectives
- 1
Decompose a user journey into discrete funnel stages and calculate step-by-step conversion rates.
- 2
Identify the single largest drop-off point in a funnel and explain why it typically deserves the most immediate investigative attention.
- 3
Explain Simpson's Paradox in a funnel context and describe why aggregate conversion rates can mislead without segmentation.
- 4
Distinguish absolute drop-off (raw number of users lost) from relative drop-off (percentage lost), and explain why prioritizing by the wrong one can lead to a misallocated fix.
- 5
Combine quantitative funnel data with qualitative investigation to move from "where users drop off" to "why they drop off."
This lesson assumes Lesson 41's definitional discipline, since every funnel stage needs the same precision (what counts as reaching this step, over what window, from what source) that any other metric requires. It also directly assumes Lesson 42's activation-metric Case Study, since this lesson picks up exactly where that Case Study left off — a company now needs to understand the specific journey leading to its newly chosen activation metric, not just track the metric's aggregate value.
Decomposing a Journey into Funnel Stages
Decomposing a Journey into Funnel Stages
A funnel represents a user's journey as a sequence of discrete stages, each with a measurable conversion rate to the next. A common generic template, often summarized by the mnemonic AARRR ("pirate metrics," coined by Dave McClure), is Awareness → Acquisition → Activation → Retention → Revenue → Referral, though the specific stages should always be tailored to the actual product journey being studied, rather than applied mechanically regardless of fit.
Each arrow's percentage represents the conversion rate from one stage to the next — the single most important number for finding where to focus improvement effort, since it isolates each transition rather than only reporting the overall, aggregate conversion from start to finish.
Finding the Biggest Drop-off: Absolute vs. Relative
Finding the Biggest Drop-off: Absolute vs. Relative
Two different ways of measuring "biggest drop-off" can point to different priorities, and conflating them is a common mistake. Relative drop-off is the percentage of users lost at a given step (in the diagram above, Stage 1→2 loses 30% of users). Absolute drop-off is the raw number of users lost (Stage 1→2 loses 300 users, the largest raw number in this example, even though Stage 3→4's 55% relative loss is a larger percentage).
Stage Transition | Relative Drop-off | Absolute Drop-off |
|---|---|---|
Stage 1 → 2 | 30% | 300 users |
Stage 2 → 3 | 21% | 150 users |
Stage 3 → 4 | 55% | 300 users |
Stage 4 → 5 | 28% | 70 users |
In this example, Stage 1→2 and Stage 3→4 tie on absolute drop-off (300 users each), but Stage 3→4's relative drop-off (55%) is much higher, suggesting a more severe, specific problem at that step relative to how many users reached it — while Stage 1→2's large absolute number may simply reflect that far more users reach that early stage in the first place. Generally, relative drop-off is more useful for diagnosing where a specific step itself is unusually broken, while absolute drop-off is more useful for prioritizing where fixing a step would recover the most total users — both numbers matter, and a PM should look at both rather than defaulting to just one.
Simpson's Paradox: Why Aggregation Can Mislead
Simpson's Paradox: Why Aggregation Can Mislead
A funnel's aggregate conversion rate can obscure dramatically different underlying realities across segments, a phenomenon related to Simpson's Paradox — where a trend visible in aggregated data reverses or disappears entirely once the data is broken into meaningful subgroups. In a funnel context: an overall signup-to-activation rate might look stable or even improving, while masking the fact that mobile users are converting far worse than desktop users (or vice versa), because a shift in the mix of traffic (more mobile users overall, who convert at a lower baseline rate) is hiding a genuine, segment-specific problem worth investigating separately.
From "Where" to "Why": Combining Quantitative and Qualitative Investigation
From "Where" to "Why": Combining Quantitative and Qualitative Investigation
Funnel data reliably tells a PM where users drop off, but rarely tells them why on its own. Once a specific stage transition is identified as the priority (using both absolute and relative drop-off, and checked for segment-hidden variation via Simpson's Paradox), the next step is typically qualitative: session recordings, targeted user interviews, or usability testing focused specifically on that step, to understand the actual user experience causing the drop-off. A PM who treats funnel data as sufficient on its own, without this qualitative follow-up, risks guessing at a fix based on assumption rather than genuine understanding of the underlying cause — precisely the discovery discipline from Lesson 8, applied here to a specific, quantitatively-identified problem area rather than a broad, undirected exploration.
Common Mistakes to Avoid
Only looking at the overall, start-to-end conversion rate without breaking it into stages
An aggregate rate tells you that something is wrong somewhere in the journey, but gives no guidance on where to focus — exactly the gap funnel decomposition into discrete stages is designed to close.
Prioritizing by relative drop-off alone, ignoring absolute numbers
A step with a dramatic 60% relative drop-off affecting only a handful of users may matter less, in terms of total recoverable users, than a milder 20% relative drop-off at a much higher-volume step — both figures should inform prioritization, not just one.
Trusting an aggregate conversion rate without checking for Simpson's Paradox-style segment variation
As covered in Theory, a stable or improving aggregate rate can mask a serious, worsening problem in a specific segment, especially when the overall traffic mix is shifting over the same period.
Assuming funnel data alone explains why users drop off
Quantitative funnel data identifies where to look; it rarely explains why without qualitative follow-up — proposing a fix based purely on quantitative funnel data, without any qualitative investigation of the actual user experience at that step, risks solving the wrong underlying problem.
Defining funnel stages inconsistently with the metric definitions established in Lesson 41
If a funnel stage's definition (what counts as "starting" the signup form, for instance) isn't as precise as Lesson 41 requires for any other metric, the resulting funnel analysis inherits the same definitional risk — comparing numbers across time or across teams that were never actually computed consistently.
The Leaky Bucket
This lesson's core takeaway tool visualizes a funnel as a bucket with leaks at each stage, directing attention to the leak that matters most rather than treating every leak as equally urgent:
Use the Leaky Bucket as a standing discipline whenever a funnel review surfaces multiple problem steps: rather than intuitively fixing whichever step "feels" most broken, calculate which leak, if patched, would actually recover the most users given both its relative severity and its absolute volume — the same dual consideration this lesson's Theory section establishes.
Key Takeaway: How will you apply "The Leaky Bucket" when evaluating trade-offs in your product decisions?
Ready to test your product judgment?
Take the interactive practice quiz for Lesson 43 and build your skill radar dashboard.