Segment analysis reveals where lift concentrates—but it can also mislead. Check your knowledge on sound methods and common pitfalls.
Testing many segments increases false positives due to ______.
garbage collection
TLS handshakes
PNG compression
multiple comparisons
A pre‑registered analysis plan reduces the risk of ______ after seeing results.
UI theming
p‑hacking
image dithering
DNS caching
Simpson’s paradox can make a variant look worse overall but better in every major ______.
file
pixel
segment
byte
To compare segments fairly, ensure each arm maintains similar ______ mix.
image format
IP octet
device or channel
font
When sample sizes are small, a ______ model can share strength across segments.
lossless
hierarchical
bitmap
deterministic CSS
Analysts often control the false discovery rate (FDR) using procedures like ______.
Benjamini–Hochberg
LZW
Huffman coding
CRC32
Reporting uplift per segment should include uncertainty, typically as ______ with intervals.
JPEG exif
RGB swatches
point estimates
CSS z‑index
For long funnels, segment analysis should consider ______ attribution windows.
zero‑byte
hard‑coded
monochrome
appropriate
Randomisation units should align with segmentation; for geo tests a common choice is ______‑level buckets.
region
byte
pixel
port
When segments interact with the treatment, analysts test for ______ effects explicitly.
render
exposure
interaction
rotation
Starter
You’re new to segment analysis in experiments. Revisit definitions, guardrails, and simple examples to build intuition.
Solid
Good understanding of segment analysis in experiments. Strengthen your analysis habits and pre‑registration discipline.
Expert!
Excellent command of segment analysis in experiments. You’re ready to operate at scale and mentor others.
