What do residual (skip) connections mainly help with in very deep networks?
- Gradients flowing to early layers
- Shrinking the tokenizer's vocabulary
- Labeling unlabeled samples
- Lowering the input resolution
Adding the input back to a block's output gives gradients a direct path, easing vanishing gradients in deep nets.
Play AI & ML ▸