Examine the trade‑offs between statistical power and opportunity cost when choosing test‑control proportions.
Larger holdout size increases statistical power but also raises ______ cost.
CPC
production
opportunity
impression
Meta’s 2025 guide suggests at least ______ users per cell for reliable lift tests.
10
1k
1
100k
Power rises with holdout size because variance of control mean falls as 1/____.
n
log n
n²
sqrt(n)
A 10% holdout is common when baseline conversion rate exceeds ______ %.
50
5
0.01
0.1
Small holdout fractions risk contamination from ______ effects.
pixel
font
network
cookie
Geo lift tests use ≥20 geos per cell instead of % holdouts because each geo is relatively ______.
cookie-based
identical
static
independent
Holdout duration should balance quick reads with enough ______ accumulation.
DNS
impressions
pixels
conversion
Sequential testing inflates type‑I error; pre‑planning a fixed holdout window preserves ______.
lambda
gamma
alpha
beta
Dynamic holdouts periodically swap users to control to mitigate ______ bias.
ISP
creative
seasonal
audio
Opportunity cost of holdouts can be estimated by applying historical ______ ROAS to withheld spend.
blended
incremental
search
cookie
Starter
Review the basics.
Solid
Nice work—refine the details.
Expert!
Exceptional command of the topic.
Getting comfortable with Lift vs. Holdout Size Interview Questions is key to designing experiments that really show your campaign’s true impact. Begin by exploring these Attribution & Marketing-Mix Modelling interview questions to see how holdout groups fit into your overall mix strategy. Then tackle the open source MMM tools practice MCQs for hands-on framework insights. After that, deepen your understanding of timing by working through the baseline decay half-life interview questions. Finally, round out your prep with the subscription SaaS MMM scenario guide to apply these concepts in real-world situations.
