Test your grasp of MAPE, hold‑out accuracy, R‑squared caveats, and Bayesian diagnostics for model validation.
Mean Absolute Percentage Error (MAPE) compares model predictions to ______.
actual observed values
Bayes prior
moving average
CTR
A hold‑out period evaluates the model’s ability to ______ future data.
backfit
ad‑serve
forecast
normalise
R‑squared can be misleadingly high in MMM if ______ variables dominate variation.
cookie
trend
creative
week
2025 Facebook Robyn 2.4 adds Deviation Penalty which penalises forecasts that exceed ______ deviation.
10 %
50 %
0 %
1 %
Normalized Root Mean Squared Error (NRMSE) divides RMSE by ______ to enable comparison.
range of actuals
sample size
CPC
mean spend
Cross‑validation in MMM often uses **time‑series split** to respect ______ order.
random
chronological
alph numeric
device
Model residuals should resemble white noise; detecting autocorrelation may require adding ______.
hashtags
pixel load
lagged variables
extra creative
Cook’s distance identifies influential observations; high values may correspond to ______ outliers.
device
promo
cookie
ctr
Out‑of‑sample MAPE below ______ is often targeted for weekly MMM forecasts.
25 %
10 %
1 %
50 %
Pareto Smoothed Importance Sampling (PSIS) k‑hat statistic above 0.7 in Bayesian MMM suggests ______.
good fit
problematic influence of observations
negative ROI
perfect independence
Starter
Review the basics.
Solid
Nice work—refine the details.
Expert!
Exceptional command of the topic.
Diving into MMM Validation Metrics Interview Questions helps you see if your media mix models hold up under scrutiny. Start with our Attribution & Marketing-Mix Modelling Interview Questions to learn which checks matter most. Next, test your understanding with the Bayesian MMM practice questions and then follow a structured approach by reviewing the Roberts four-step MMM process guide. Finally, sharpen your skills on handling variable overlap by exploring the multicollinearity remedies interview resource.
