Definition
RNNs update a hidden state as they read each token or timestep, making them natural for sequences. Vanilla RNNs struggle with long-range dependencies due to vanishing gradients.
LSTMs and GRUs improved memory; transformers largely replaced RNNs for large-scale NLP.
In simple terms
An RNN is reading a book while jotting a running note — each new sentence updates the note before you continue.
Where you see it
- Early machine translation and language models.
- Time-series forecasting and sensor streams.
- Legacy speech systems before transformer ASR.
How it works
1.Read one step
Combine input with previous hidden state.
2.Update memory
Produce a new hidden state.
3.Emit output
Optional prediction at each step or at the end.
Why it matters
- RNNs taught the field how to model sequences — essential context for why transformers won.
Often confused
Nobody uses RNNs anymore.
They still appear in constrained devices, some speech models, and teaching curricula.