A True Number Can Still Mislead You

One event, four percentages

Twelve customers cancelled this quarter. Walk into the room and that's the one fact everyone agrees on. What splits the room is the percentage: 80%? 60%? 40%? 10%? Every one of those numbers is correct.

That's not a reporting error. A percentage is a claim about a defined group, not just a calculation, and each of those four numbers is measuring the twelve against a different group. Before arguing about whether 80% is worse than 10%, it's worth asking a simpler question. Who exactly does each number describe?

The count is only half the question

A percentage has two parts: a count of what happened, and a group that count gets measured against.

Get the count wrong and you've made an arithmetic mistake. Measure it against the wrong group and you may produce an accurate answer that's irrelevant to the decision being made. A percentage only works if everyone in the count is actually part of the group you're measuring it against (CDC guidance on ratios, proportions and rates).

Here's the core idea in one line: what happened is only half the number. Who it's measured against is the other half. Two people can both be right and still be talking past each other, because they've quietly picked different groups to count against.

Four correct percentages, four different decisions

Here's where those four numbers come from. They should not all be given the same name, because they describe different stages of the renewal process.

Four groups applied to the same 12 cancellations

Scroll sideways to see every column.

Each group sits inside the next: the 15 who explicitly responded are part of the 20 who reached a final outcome, who are part of the 30 exposed to the price increase, who are part of the 120 who opened the quarter. The same 12 cancellations sit inside all four, just as a shrinking share each time.

None of these percentages invalidates the others, but their names matter. If the organisation defines churn as customers lost divided by the opening active base, 10% is the formal portfolio churn rate; the other three are diagnostic rates that explain what's happening inside the renewal process. Strip away the group label, though, and 80%, 60%, 40% and 10% all look equally authoritative. That's the real mistake: presenting one as the only honest answer.

The opening-base rate informs portfolio forecasting. The exposed-cohort rate informs the pricing review. The completed-outcome rate helps assess the renewal process. The response-conditional rate helps diagnose what customers are explicitly saying. Naming and defining each group is what keeps one rate from being read as another.

Choosing what to measure against is a decision about relevance

Choosing what a number gets measured against is a decision about who was eligible to be counted. The unemployment rate makes this concrete: it isn't unemployed people divided by the total population, it's unemployed people divided by the labour force, those employed or actively seeking work. Divide by the total population instead and the rate is diluted by everyone who was never part of the labour market to begin with (ILO guide to interpreting unemployment). The same logic applies to churn: ask who was actually exposed to the event, and who had a genuine chance to be counted at all.

Time changes who belongs in the group

A customer base isn't frozen in place: people join partway through a period, and some outcomes are still pending when the report gets written. In this example, only 20 of the 30 exposed customers have a final outcome. The 60% completed-outcome rate uses only settled outcomes, so it can rise or fall as the remaining ten decisions are resolved. The 40% exposed-cohort rate uses the complete cohort, so only its count can increase: it can stay at 40% or rise, but it cannot fall. Neither figure should be presented as final.

Counting only the resolved cases is a version of survivorship bias: whichever outcomes settle fastest get counted first, and there's no guarantee they're representative of the ones still pending. If cancellations tend to resolve faster than renewals, an early read will always skew towards cancellations, independent of where the final number lands.

Give the group enough time to reach an outcome before treating a rate as final, and when comparing periods, make sure both use the same definition of "exposed," "active" and "final outcome." Comparing a completed cohort to a half-finished one isn't a fair comparison, even if both numbers are accurate (OECD statistical quality framework).

Weighting can move the aggregate result

Standard and Enterprise loss rates by period, unweighted and combined

Scroll sideways to see every column.

Standard customers are losing 20% both periods. Enterprise customers are losing 5% both periods. Nothing changed inside either group. But the combined rate drops from 17% to 8%, purely because Enterprise customers made up more of the base in Period 2.

Simply averaging 20% and 5% gives 12.5% in both periods, but that number doesn't represent the actual pooled result unless the two groups are the same size, which they aren't here. This is a composition effect: the mix shifted, and the aggregate followed the mix. The right weights depend on the population you're trying to represent (US Census Bureau weighting guidance), and it's worth naming the effect, because a leadership team that only sees the headline 17% to 8% improvement might credit a retention initiative that never happened.

Different units answer different questions

Customer loss rate counts people, treating someone who paid ₵50 a month the same as someone who paid ₵5,000. That's fine for some questions and misleading for others: if the question is really about revenue, both the count and the group it's measured against need to be revenue too. Dividing a customer count by a cedi figure doesn't produce a usable number, it just produces two incompatible units sitting next to each other.

Not just who's counted. Who's counting.

Every section so far has assumed one team is doing the counting, and only the group changes. In practice, different teams inside the same organisation often count the same event at different moments. Sales might mark a cancellation the moment a customer submits a downgrade request. Finance might not count it until the final invoice clears, weeks later. Both are real, defensible points in the process. They're just not the same point.

This is a different problem from the one the rest of this article is about. Two people can agree completely on which group to measure against and still land on different counts, because their teams never agreed on what counts as the event itself. Fixing that isn't a matter of choosing the right group. It's agreeing on the moment being counted, before anyone compares a single number.

When picking a comparison turns into picking a flattering one

Two people rarely land on different percentages out of bad faith. Most of the time, it's two teams working from different real questions without realising it, or a metric nobody ever documented clearly. Where it does tip into a bias is confirmation bias: reaching for whichever comparison confirms what you already believed, then defending it as the correct one. The clearest sign of that is a definition that changes right after an inconvenient result, rather than one written down before anyone saw the number. Statistical practitioners are expected to resist exactly this kind of pressure to predetermine or selectively interpret a result (American Statistical Association ethical guidelines).

Questions to ask before debating the percentage

Before arguing about whether a number is good, bad, or believable, it's worth working through a short list:

1. What group does this number actually measure against?
2. Did everyone in that group have a genuine chance to be part of the count?
3. Is this a completed outcome, or is the group still resolving?
4. Would a different, equally reasonable comparison group produce a meaningfully different answer?
5. Are the count and the group it's measured against expressed in compatible units, for example customers against customers, or revenue against revenue?
6. What decision is this number actually meant to support?

When two people read a percentage differently, it's rarely the arithmetic they're seeing differently. It's usually one of these questions that was never asked out loud.

Conclusion

The twelve cancellations never changed. What changed each time was the group they were measured against, and that's a decision, not a rounding error. The room doesn't need to choose one percentage and discard the other three; it needs to name each measure honestly, know which group it describes, and match it to the decision on the table.

Before debating whether a percentage is good or bad, ask what group it describes. Which comparison best matches the decision you're trying to make?