What is the most likely cause of this loss behavior?
step loss 1 2.30 2 5.10 3 18.70 4 nan
- Learning rate is too high
- Batch size is too large
- Too much dropout is applied
- The model is overfitting
Loss that explodes to NaN right away usually means steps overshoot the minimum; lower the learning rate.
Play AI & ML ▸