Measurement · 7 min read
Why your opt-in rate may have the wrong denominator
A precise numerator cannot rescue an ambiguous population.
Start with eligibility
“Opt-ins divided by users” sounds clear until the user population is examined. Some users already made a system-level choice. Some never reached the relevant value moment. Others used a platform or app version where the prompt behaves differently.
Define at least three populations: users eligible to be asked, users exposed to the in-app context, and users shown the native prompt. Each answers a different question.
Use rates that match decisions
Exposure-to-native-prompt measures progression through your own experience. Native-prompt-to-allow measures the decision at the operating-system prompt. Eligible-to-allow gives an end-to-end view. Show all three before compressing the journey into one KPI.
Check time and identity
Keep the observation window stable, deduplicate repeated events and decide how reinstallations are handled. Document whether the analysis is user-, device- or installation-based. In a Malaysian product with multilingual audiences, also check whether locale is missing systematically.