Mindmap on A/B tests designing and experiments
Udacity has a quite interesting course on A/B testing. I’ve made a mind map for one of the lessons “Designing and Experiment” to better understand the mechanics of tests. Also, there is a python script for empirical estimation of the required group size for a experiment. It’s a modified version of the R script by Udacity and uses a binary search to speed up computations.
Plus, there are articles to read:
- Overlapping Experiment Infrastructure: More, Better, Faster Experimentation. Description of the Google’s overlapping experiment infrastructure.
- Large-Scale Validation and Analysis of Interleaved Search Evaluation. Comprehensive analysis of interleaving using data from two major commercial search engines and a retrieval system for scientific literature.