$ ./guess_the_languageAI & ML
AI & ML · medium · #40

Why can transformers train faster on long text than RNNs?

  1. Attention sees all positions at once
  2. They use far fewer parameters overall
  3. They need no training data
  4. They read text right to left
RNNs process tokens one step at a time, while self-attention handles every position in parallel.
Play AI & ML ▸