Test your knowledge of false positives vs.
A false positive in hypothesis testing is formally called a ______ error.
Type II
Type I
Power
Beta
Lowering alpha from 0.05 to 0.01 primarily reduces the probability of ______.
false positives
variance inflation
false negatives
sample ratio mismatch
Increasing statistical power mainly decreases the chance of a ______.
false negative
false positive
Type I error
SRM
The beta (β) value of a test directly quantifies the risk of ______.
prior odds
Type II error
effect size
Type I error
Applying a Bonferroni correction across many metrics shrinks alpha and therefore controls the family‑wise ______ rate.
false negative
posterior
false positive
power loss
Running an under‑powered test most often manifests as which problem?
inflated false positives
higher SRM
inconclusive results / false negatives
lower variance
Mistaking a non‑significant p‑value for evidence of no effect exemplifies confusing absence of evidence with ______.
Simpson’s paradox
Bayes factor
alpha spending
evidence of absence
Increasing sample size after peeking and seeing p ≈ 0.06 mainly risks elevating ______.
power
credible intervals
beta inflation
false positives through optional stopping
Sequential testing frameworks like alpha‑spending aim to keep the overall ______ under control while allowing interim looks.
false negative rate
false positive rate
Bayes factor
posterior odds
Lack of variant quality checks can lead to tracking bugs that mimic lifts, producing apparent ______.
false negatives
statistical power
effect heterogeneity
false positives
Starter
Brush up on the basics and avoid costly mistakes.
Solid
You know your stuff—refine a few nuances for mastery.
Expert!
Outstanding! You could teach this topic.
