Definition
LSTMs introduce input, forget, and output gates plus a cell state, allowing gradients and information to flow across many timesteps more reliably than simple RNNs.
They were state-of-the-art for NLP and speech before transformers scaled better with parallel training.
In simple terms
If an RNN's notepad gets messy and overwritten, an LSTM has a protected vault (cell state) with locks (gates) controlling what enters and leaves.
Where you see it
- Pre-transformer machine translation.
- Handwriting and speech recognition systems.
- Some industrial time-series models.
How it works
1.Gate information
Decide what to forget, store, and output.
2.Update cell state
Carry long-term memory across steps.
3.Produce hidden state
Expose a filtered view of the cell.
Why it matters
- LSTMs bridged early sequence models and the transformer era — still useful conceptually and in niches.
Often confused
LSTM means any long-context model.
LSTM is a specific gated RNN design; long context in LLMs usually comes from transformers.