Stopping rules shape error rates as much as p‑values do. Learn why peeking early requires different math from waiting until a fixed sample is reached..
Sequential testing differs from fixed‑horizon testing mainly in ______.
requiring equal sample splits only
allowing interim looks at the data
using non‑parametric metrics
banning multiple variants
Repeatedly checking significance without adjustment inflates the risk of a ______.
Type II error (false negative)
Type I error (false positive)
missing data
power loss
The ‘alpha spending’ approach in group‑sequential designs ______.
divides the total Type I error budget across looks
increases sample size each look
eliminates the need for p‑values
sets power to 100%
A fixed‑horizon test typically requires the full pre‑calculated sample to avoid bias because ______.
users drop out
stopping early on apparent wins skews the lift upward
variance estimates are constant
traffic spikes daily
Sequential probability ratio tests (SPRT) compare the likelihood of data under two hypotheses after ______.
every 1000 samples exactly
quartile boundaries
each new observation or batch
the final day only
‘Power’ in the fixed‑horizon context is defined at ______ sample size.
any interim
adaptive
halfway
the predetermined final
Bayesian sequential methods often monitor the ______ directly rather than p‑values.
Bonferroni‑adjusted alpha
posterior probability of uplift
t‑statistic
critical z‑score
In a two‑look group‑sequential design using O’Brien‑Fleming, the early boundary is ______ than 1.96.
larger
unrelated
equal to
smaller
Running a fixed‑horizon test longer than the planned sample size mainly risks ______.
inflating Type I error
creating unequal cookies
changing randomization ratio
diluting power if effect decays over time
Adaptive tests that re‑allocate traffic during the experiment must update variance formulas to remain ______.
cheaper
faster
unbiased
monotonic
Starter
Solidify the basics of the topic before running live tests.
Solid
You understand the core ideas—focus on edge‑cases to boost accuracy.
Expert!
Outstanding—you can teach your team and steer experimentation strategy.
