Measurement · 7 min read

Why your opt-in rate may have the wrong denominator

A precise numerator cannot rescue an ambiguous population.

Analytics charts displayed on a monitor

Start with eligibility

“Opt-ins divided by users” sounds clear until the user population is examined. Some users already made a system-level choice. Some never reached the relevant value moment. Others used a platform or app version where the prompt behaves differently.

Define at least three populations: users eligible to be asked, users exposed to the in-app context, and users shown the native prompt. Each answers a different question.

Use rates that match decisions

Exposure-to-native-prompt measures progression through your own experience. Native-prompt-to-allow measures the decision at the operating-system prompt. Eligible-to-allow gives an end-to-end view. Show all three before compressing the journey into one KPI.

Check time and identity

Keep the observation window stable, deduplicate repeated events and decide how reinstallations are handled. Document whether the analysis is user-, device- or installation-based. In a Malaysian product with multilingual audiences, also check whether locale is missing systematically.

A trustworthy rate names its population, event, window and unit of analysis.