Cohort Analysis: How to Judge a Marketing Channel by Retention, Not Sign-ups
What cohort analysis is, how to read a retention cohort table, and how to use cohorts by acquisition channel to decide where to spend, with a worked example showing why the cheapest sign-ups are often not the cheapest customers.
By Suren, Founder & Growth Strategist at Gufy · Published
Cohort analysis groups users who started at the same time, usually the week or month they signed up, and follows what each group does next: how many come back, how many pay, how much they spend. To judge a marketing channel, split those groups by where users came from and compare how many are still active weeks later. The channel with the cheapest sign-ups is often not the one with the cheapest customers.
This guide covers how to read a cohort table, how to compare channels with it, what distorts the numbers, and how to turn it into a spending decision.
What a cohort is
A cohort is a fixed group. Nobody joins it later, so you can compare it fairly with the groups before and after.
| Type | Grouped by | Answers |
|---|---|---|
| Acquisition cohort | When (and where) users joined | Are newer users staying longer than older ones? Which channel brings people who stay? |
| Behavioural cohort | Something users did | Do people who complete setup, or invite a colleague, stay longer than those who do not? |
This guide is mostly about acquisition cohorts, because that is where channel decisions are made. Behavioural cohorts are how you find your aha moment.
How to read a retention cohort table
Each row is a group of users who joined in the same week. Each column is a week after they joined. Each cell is the share of that group still active.
An illustration, for a product people use about once a week:
| Joined | Users | Week 0 | Week 1 | Week 2 | Week 3 | Week 4 |
|---|---|---|---|---|---|---|
| Week 1 | 400 | 100% | 38% | 30% | 27% | 26% |
| Week 2 | 350 | 100% | 36% | 29% | 26% | |
| Week 3 | 500 | 100% | 30% | 22% | ||
| Week 4 | 420 | 100% | 41% |
Read it three ways:
- Along each row. A large drop after week 0 is normal for almost every product. What matters is what happens next. In week 1's row the curve flattens at around a quarter of users: those people found something worth coming back for. A curve that keeps falling towards zero means nobody is staying, and no amount of acquisition will fix that.
- Down each column. Week 3's group kept fewer people at every step. Something changed that week: a new campaign, a new channel, a broken step in sign-up. Find out what before you spend more.
- The newest rows. Week 4's group kept more people after one week. If that follows a change to onboarding, it is early evidence the change worked. Wait for more weeks before you call it.
Measure "active" at the pace the product is naturally used. A product people need weekly should be judged on weekly return, not daily. Expecting daily use from a weekly product makes healthy retention look like failure.
Comparing channels with cohorts
Now split the same groups by where users came from. This is where most channel decisions go wrong, because the first number anyone sees is cost per sign-up or cost per install.
An illustration with the same spend on two channels:
| Channel A | Channel B | |
|---|---|---|
| Spend | $3,000 | $3,000 |
| Sign-ups | 1,500 | 600 |
| Cost per sign-up | $2.00 | $5.00 |
| Still active in week 4 | 8% (120 people) | 30% (180 people) |
| Cost per user still active | $25.00 | $16.67 |
| Paying after eight weeks | 15 | 24 |
| Cost per paying customer | $200 | $125 |
On cost per sign-up, Channel A wins easily. On what the business actually needs, it loses. The calculation is simple: divide what you spent on a channel by the users it kept, not the users it brought.
When you do not yet know what a user is worth
The table above ends in paying customers. Many early products, especially free apps, cannot get that far yet: there is no revenue to count. It is a common position: one channel brings installs for a fraction of the price, the other brings fewer people who come back noticeably more often, and nobody can say whether the extra retention is worth the extra cost.
In that position:
- Pick the closest thing to value you can measure. A completed profile, a booking, a message sent, an event attended. Whatever your north star metric is, or the step just before revenue.
- Compare the cost of reaching that step, not the cost of an install.
- Say plainly that the answer is unknown, and make putting a value on a retained user the next job. Our guide to unit economics covers how.
Do not choose the cheaper channel by default because its first number looks better.
Setting it up
- Record the source at sign-up. Tag every link with UTM parameters, store the first source on the user record, and for apps use a mobile measurement partner to tie installs to campaigns. A "how did you hear about us?" question catches word of mouth that tracking misses.
- Define "active". Choose an action that means real use (not just opening the app) at the product's natural frequency.
- Choose the cohort size. Weekly if you have hundreds of sign-ups a week. Monthly if you have fewer, so one or two people do not swing the result.
- Build the report. Mixpanel and Amplitude both have retention reports you can break down by a user property such as first source. Google Analytics 4 has a cohort exploration with a breakdown option. For apps, measurement partners such as AppsFlyer report retention by media source. Our comparison of AppsFlyer and Mixpanel explains which tool does what.
- Add money. Once people pay, add cumulative revenue per user for each cohort. That is the number that decides how much a channel can cost.
If none of this is tracked yet, that is the first job, before any more spend. Best Product Analytics Tools for Startups compares the options.
What distorts the numbers
- Broken tracking. If attribution for one platform or operating system stops working, that channel will look as if it has died overnight. Before you cut a channel, check that its installs or sign-ups still match what the app store or your database shows.
- People who install and never open. Count from the first real action, or the channel that brings more of them will look better than it is.
- Cross-channel exposure. Someone may see your ad on one platform and sign up from another. Last-click attribution gives all the credit to the second. Treat channel figures as close, not exact.
- Partial data on iOS. Apple's privacy rules limit how much user-level attribution you get from iPhones. Compare channels on the same platform, and do not compare an iOS figure with an Android one as if they were measured the same way.
- Offers and events. A cohort that came for a discount or a one-off event behaves differently from one that came for the product. Users who arrive with a specific reason to come back often retain better, which is a lesson in itself.
- Small numbers. A cohort of thirty people is a few individuals' habits. Look at several cohorts together before deciding.
When every channel retains badly
If users from every channel leave at the same rate, the problem is not the channel. It is the product, the onboarding or the message that brought people in. A good product keeps people wherever they came from.
That is the point to stop moving budget between channels and fix what happens after sign-up. SaaS Onboarding and the aha moment are where to start.
Turning it into a decision
| What the cohorts show | What to do |
|---|---|
| Cheap sign-ups, poor retention | Check tracking first. If it holds, cut or change the audience and message; the channel is bringing the wrong people |
| Expensive sign-ups, strong retention | Work out the cost per retained or paying user. It may be your best channel |
| Good retention, too few people | Test more audiences and creative on that channel before adding new ones |
| Poor retention everywhere | Pause scaling and fix onboarding and activation |
| Retention improving in newer cohorts | Keep the change that caused it, then scale |
| Not enough data to tell | Run longer, use monthly cohorts, or concentrate spend on fewer channels |
Report it every week
Channel decisions made in a hurry, because cash is tight or a bill has arrived, tend to be blunt: cut everything, or cut whatever costs most. A short weekly report showing spend, sign-ups, retained users and paying users by channel means you see a weak channel weeks before it becomes an emergency, and cut the right thing.
If an agency or contractor runs your channels, ask for that report. A report of installs and clicks alone tells you what was bought, not whether it was worth buying.
Common mistakes
- Judging channels on cost per install or sign-up. It is the first number, not the one that matters.
- Expecting the wrong frequency. Measure return at the pace the product is meant to be used.
- Cutting a channel without checking tracking.
- Reading one cohort. Look for a pattern across several.
- Blaming the channel for a product problem. If everyone leaves, fix what they leave from.
- Waiting for perfect data. Close and consistent is enough to decide.
Common questions
What is cohort analysis?
Cohort analysis groups users who share a starting point, usually the week or month they signed up, and tracks what each group does over the following weeks: how many stay active, how many pay and how much they spend. Because each group is fixed, you can compare them fairly over time.
How do you use cohort analysis to compare marketing channels?
Group new users by the week they joined and the channel they came from, then compare the share of each group still active, and paying, several weeks later. Divide what you spent on each channel by the users it kept, not the users it brought, to get the cost of a retained user.
What is a good retention rate for an app or SaaS product?
It depends on how often the product is naturally used, so compare against your own earlier cohorts rather than a published average. The pattern matters more than the number: a healthy product loses many users early and then the curve flattens. A curve that keeps falling towards zero means people are not finding lasting value.
What is the difference between an acquisition cohort and a behavioural cohort?
An acquisition cohort groups users by when, and often where, they joined. A behavioural cohort groups them by something they did, such as completing setup or inviting a colleague. Acquisition cohorts show whether things are improving over time; behavioural cohorts show which actions go with users who stay.
How many users do you need for cohort analysis?
Enough that one or two people do not swing the result. With small numbers, use monthly instead of weekly cohorts, look at the direction across several cohorts rather than one, and treat differences of a few percentage points as noise.
Where Gufy fits
Gufy sets up the tracking that makes cohorts possible and reads them with you: product analytics consulting to record where users come from and what they do next, and performance marketing that is reported on retained and paying users by channel, not just installs and clicks. For apps, our mobile app growth work connects campaigns to what users do after they install.
If you are deciding which channel to cut and the only number you have is cost per install, start there.