Definition
High bias means systematic error from oversimplification (underfitting). High variance means predictions swing wildly with small data changes (overfitting). Total error balances both.
Ensemble methods, regularization, and more data are practical tools for managing the tradeoff.
In simple terms
A rigid ruler (high bias) misses curved lines. A floppy noodle (high variance) fits every wiggle including noise. You want a flexible-but-stable measuring stick.
Where you see it
- Choosing tree depth in random forests.
- Regularizing neural nets to stabilize generalization.
How it works
1.Measure errors
Decompose or diagnose via train/val gaps.
2.Adjust complexity
Simpler models lower variance; richer models lower bias.
3.Validate
Cross-validation estimates out-of-sample risk.
Why it matters
- It is the conceptual backbone of model selection and regularization decisions.
Often confused
Bias always means unfairness.
In this context, bias means systematic prediction error — distinct from societal bias in data or systems.