StackedML
Practice
Labs
Questions
Models
Pricing
Sign in
Questions
/
ML Fundamentals
/
Core Concepts
/
Train / validation / test separation
← Previous
Next →
776.
Test Set Single Evaluation
easy
Why should the test set be evaluated only once, at the very end of the project?
A
Repeated evaluation on the test set increases computational cost beyond what is practical for most production pipelines during iterative model development
B
Repeated evaluation on the test set causes the test labels to be memorized by the training process over time
C
Repeated evaluation on the test set leaks information into model development, making performance estimates optimistically biased
D
Repeated evaluation on the test set violates the statistical independence required for valid significance testing across multiple evaluation runs
Sign in to verify your answer
← Back to Questions