Definition
Foundation models (large language, vision, or multimodal models) learn general representations from massive datasets, then specialize via prompting, RAG, or fine-tuning.
The term emphasizes reuse: one expensive pretraining run supports many applications.
In simple terms
A foundation model is a well-built foundation and frame — you still finish each house differently, but you are not pouring concrete from scratch every time.
Where you see it
- GPT, Llama, and Claude-style LLMs.
- CLIP and other vision–language foundations.
- Speech foundations adapted to new languages.
How it works
1.Pretrain at scale
Self-supervised objectives on huge corpora.
2.Align or adapt
Instruction tune, RLHF, LoRA, or prompts.
3.Specialize
Apps add tools, RAG, and eval harnesses.
Why it matters
- Foundation models changed AI economics — adapt, don't always train from scratch.
Often confused
A foundation model is finished and universal.
It still needs adaptation, eval, and often domain data — especially for low-resource languages.