• experimental design
• statistical philosophy
• statistical tests & assumptions
• analysis platforms

• random assignment to treatments
• poorer alternative: haphazard assignment
  (“convenience sampling”)
• stratification
  (i.e., randomize within groups)
• related: experimental blinding

• need to guess effect size and variability
  • minimum interesting biological effect size
  • previous studies
  • ask your supervisor
• OK to simplify design (e.g. ANOVA \(\to\) \(t\)-test)
• methods
  • seat-of-the-pants
  • web calculators
  • in R

apropos("^power")  ## base-R functions
library("sos"); findFn("{power analysis}")

With \(n=15\) per group, are we likely to see a clear difference between 10% (control) and 20% (treatment) mortality (a doubling of mortality)?

power.prop.test(n=15,p1=0.1,p2=0.2)

## 
##      Two-sample comparison of proportions power calculation 
## 
##               n = 15
##              p1 = 0.1
##              p2 = 0.2
##       sig.level = 0.05
##           power = 0.1141268
##     alternative = two.sided
## 
## NOTE: n is number in *each* group

Uh-oh!

How many samples per group would we need to get power=0.8?

power.prop.test(power=0.8,p1=0.1,p2=0.2,sig.level=0.05)

## 
##      Two-sample comparison of proportions power calculation 
## 
##               n = 198.9634
##              p1 = 0.1
##              p2 = 0.2
##       sig.level = 0.05
##           power = 0.8
##     alternative = two.sided
## 
## NOTE: n is number in *each* group

Uh-oh!

• increase sample size (ugh)
• increase rejection threshold (e.g. \(\alpha=0.1\))
  (probably can’t get away with this)
• maximize desired variation (e.g. large doses)
• measure a clearer outcome (gene expression, hormone levels, etc.)

• minimize undesired variation
  • within-subjects designs
    (paired, randomized-block, crossover)
• minimize environmental variation
  (e.g. environmental chambers; clonal or inbred lines)
• isolate desired effects: positive/negative controls
  (vehicle-only, cage treatments, etc.)
• control for variation statistically
  (e.g. include body size as a covariate)

https://xkcd.com/882/

(or “researcher degrees of freedom” (Simmons et al. 2011))

• https://artsharela.org/event/carillon-quartet-the-garden-of-forking-paths/

1978 automobile data (Chambers et al. 2018)

• independence (hard to test!)
• homogeneity of variance (vs. heteroscedasticity)
• linearity
• Normality (least important)
  • outliers; skew; “fat tails” (Student 1927)
  • distributional assumptions apply to the conditional distribution of the response variable

• hypothesis tests are not generally appropriate:
  they answer the wrong question
• graphical diagnostics
  • residuals plots (linearity, heteroscedasticity)
  • influence plots (outliers)
  • Q-Q plots (Normality)
  • Box-Cox plots (transformations)

• drop outliers (report both analyses)
• transform (e.g. log transform: Box-Cox analysis)
• non-parametric (rank-based) tests
  (e.g. Mann-Whitney-Wilcoxon, Kruskal-Wallis)
• relax assumptions/do fancier stats, e.g.
  • logistic regression (0/1 outcomes)
  • quadratic regression (nonlinearity)

• try to connect scientific & statistical questions
• data type
  • see decision tree or table
  • if your question doesn’t fit in this tree,
    think about how much you like statistics …
• nonparametric stats
  • slight loss of power
  • stronger assumptions than you think
  • \(p\)-values only - no effect size

• simple/weak vs. complex/powerful
• GUI vs command-line
• default: use what your lab uses

• ubiquitous
• open alternatives (Open Office)
• data in plain sight
• good enough for simple stuff
• occasional traps (McCullough et al. (2008); date handling; etc.)
• archive your data as CSV, not XLSX

• SPSS, JMP …
• more reliable than Excel
• more powerful than Excel
• point & click (mostly)

• powerful; free & open
• reproducible: script-based
• hardest to learn
• graphical interfaces: R Commander or Jamovi
• great for data manipulation, graphics
  (once you learn how)

x1 = c(1.5,2.5,2.1)
x2 = c(1.1,1.4,1.5)
t.test(x1,x2)

## 
##  Welch Two Sample t-test
## 
## data:  x1 and x2
## t = 2.226, df = 2.6648, p-value = 0.1236
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -0.3759079  1.7759079
## sample estimates:
## mean of x mean of y 
##  2.033333  1.333333

• Nature web collection, “Statistics for Biologists”
• UCLA statistics consulting (many examples in SAS, SPSS, R …)
• CrossValidated
• bio data lunch
• source code for this talk