Cut through the jargon and see what “statistically significant” really means. Test your grasp of alpha, error rates, and why context still matters.
Which term represents the probability of rejecting a true null hypothesis?
Effect size
Type I error (alpha)
Power (1‑β)
Type II error (beta)
A result with p = 0.049 under an alpha of 0.05 is best described as ______.
clinically significant
statistically significant but marginal
practically guaranteed
not significant
Lowering alpha from 0.05 to 0.01 will generally ______ the required sample size.
increase
halve
decrease
leave unchanged
If your confidence interval for lift includes zero, the test result is ______.
not statistically significant
subject to Simpson’s paradox
positively biased
over‑powered
The phrase “5% significance level” most directly sets a limit on ______.
false‑positive rate
sample segmentation
minimum detectable effect
missing data
Two‑tailed tests allocate alpha ______.
only to the upper tail
only to the lower tail
to the larger sample
across both distribution tails
Sequential peeking without correction mainly inflates ______.
statistical power
Type I error
standard error
effect size
Practical significance focuses primarily on ______.
business impact magnitude
alpha level
p‑value precision
chi‑square assumptions
Bonferroni correction is one method to control experiment‑wise ______.
beta inflation
family‑wise error rate
effect heterogeneity
statistical power
Failing to reject the null means the evidence was ______.
insufficient under chosen alpha
biased by outliers
proof the null is true
caused by low variance
Starter
Dive deeper into the fundamentals to boost your confidence.
Solid
You’ve got the core ideas—polish a few nuances for mastery.
Expert!
Outstanding—your stats savvy rivals that of a data scientist.
