Definition
The context window is how many tokens a model can attend to at once — your system prompt, conversation history, retrieved documents, and the answer it generates.
Exceeding the limit truncates or rejects input. Larger windows (128K, 1M tokens) enable longer documents but cost more to run.
In simple terms
The context window is the size of the model's desk. It can only spread out so many pages at once; anything that does not fit gets pushed off the edge.
Where you see it
- ChatGPT threads hit context limits on very long conversations.
- RAG systems must chunk documents to fit retrieval + answer in one window.
- Developers count tokens before sending prompts to APIs.
How it works
1.Count tokens
Tokenizer measures prompt + expected output size.
2.Fit or truncate
Stay under the model's max; drop oldest messages or summarize if needed.
3.Generate
Model attends to all in-window tokens when producing each new token.
Why it matters
- Context limits shape how you design chat apps, RAG, and document analysis.
Often confused
A huge context window means the model remembers everything perfectly.
Models may lose focus in the middle of very long inputs — "lost in the middle" is a known issue.