Mindmap on A/B tests designing and experiments

Udacity has a quite interesting course on A/B testing. I’ve made a mind map for one of the lessons “Designing and Experiment” to better understand the mechanics of tests. Also, there is a python script for empirical estimation of the required group size for a experiment. It’s a modified version of the R script by Udacity and uses a binary search to speed up computations.

Plus, there are articles to read:

  1. Overlapping Experiment Infrastructure: More, Better, Faster Experimentation. Description of the Google’s overlapping experiment infrastructure.
  2. Large-Scale Validation and Analysis of Interleaved Search Evaluation. Comprehensive analysis of interleaving using data from two major commercial search engines and a retrieval system for scientific literature.

Designing and experimet