454. Learning Rate Warmup Rationale
medium
A learning rate warmup schedule increases the learning rate gradually from near zero at the start of training. What problem does this solve?
A learning rate warmup schedule increases the learning rate gradually from near zero at the start of training. What problem does this solve?