Experiment statistics overview - Docs - PostHog
posthog.com · 2,017 words · saved by 1 readers
A working understanding of statistical methodology is helpful to feel confident about interpreting experiment results. For those without prior…
Bayesian statistics Contents Input data: What goes into the analysis Funnel metrics Mean metrics Ratio metrics What the experimentation pipeline does Step 1: Data aggregation Aggregation into sufficient statistics Outlier handling (winsorization) Step 2: Data quality validation Step 3: Calculate effect size and variance Effect size calculation Variance calculation Step 4: Bayesian posterior update The prior distribution The posterior distribution Step 5: Generate results Chance to win Credible interval Significance (decisiveness) Configuring the confidence level Mathematical formulas reference
related reading
- Legacy statistics methodology - Docs - PostHogposthog.com
- PostHog – We make your product self-drivingposthog.com
- How much do you believe your results? — LessWronglesswrong.com
- Running time and sample size - Docs - PostHogposthog.com
- Statsig | The modern product development platformstatsig.com
- How to Measure Anything — LessWronglesswrong.com
- Seeing Theoryseeing-theory.brown.edu
- Writing - betanalpha.github.iobetanalpha.github.io
- Against NHST — LessWronglesswrong.com
- Is power analysis necessary in Bayesian Statistics? - Cross Validatedstats.stackexchange.com
- You Are Not Measuring What You Think You Are Measuring — LessWronglesswrong.com
- Power (statistics) - Wikipediaen.wikipedia.org