A/A tests compare identical experiences to validate your experiment setup and variance assumptions. See if you know when and how to use them well.
The primary purpose of an A/A test is to validate the ______, not to find a winner.
creative concept
data warehouse schema
content management system
experiment pipeline and instrumentation
A common sign that something is broken in an A/A test is a persistent ______ between groups.
100% conversion
perfectly equal revenue
sample ratio mismatch (SRM)
zero bounce rate
A/A tests are most useful ______ running the first major A/B on a new surface.
during final rollout only
instead of
after
before
If an A/A test shows significance repeatedly, your ______ assumptions may be wrong.
viewport
image compression
variance or independence
cache eviction
Which unit is typically preferred to avoid cross‑session contamination in A/A tests?
ad‑impression‑level
pageview‑level
network‑level
user‑level randomisation
A/A tests help calibrate minimum detectable effect (MDE) by measuring ______ noise.
device pixel
baseline
color profile
CDN
A guardrail metric commonly checked in A/A and A/B tests to spot issues is ______.
file size of CSS
image ALT length
HTTP referrer string
bounce rate stability
To detect SRM early, many teams monitor live allocation charts and run a chi‑square test on ______ counts.
bucket
cookie
font
pixel
A/A tests should use the same randomisation, eligibility, and traffic filters as your ______ tests.
load
A/B
security
visual
One limitation of A/A tests is they consume time and traffic without producing ______ lift.
RGB
business
HTML
DNS
Starter
You’re new to a/a tests: why and how. Revisit definitions, guardrails, and simple examples to build intuition.
Solid
Good understanding of a/a tests: why and how. Strengthen your analysis habits and pre‑registration discipline.
Expert!
Excellent command of a/a tests: why and how. You’re ready to operate at scale and mentor others.
