StackedML
Practice
Labs
Questions
Models
Pricing
Sign in
Questions
/
Model Evaluation & Experimentation
/
Experimentation
/
Offline vs online evaluation
← Previous
Next →
461.
Limitations of Offline Evaluation
easy
Why is offline evaluation insufficient on its own before deploying a model to production?
A
It is insufficient because offline evaluation is biased since held-out test sets always overlap with training data in practice under typical data splitting procedures
B
It is insufficient because offline evaluation always underestimates true model performance since it uses a smaller dataset than production
C
It is insufficient because historical data reflects past user behavior — the deployed model changes the environment and user responses may differ
D
It is insufficient because offline evaluation cannot compute business metrics like revenue since it lacks transaction data
Sign in to verify your answer
← Back to Questions