context window

In AI, a context window is the maximum amount of text, counted in tokens, that a model can read and use at once, including its own reply.

Every request to a large language model has to fit into one fixed space. The instructions, the conversation so far, any pasted documents and the reply being written all share it. None of it is part of what the model learned in training.¹ The space is measured in tokens, the word pieces a model reads, rather than in words or pages. Anthropic put 100,000 tokens at about 75,000 words, more than five hours of reading for a person.²

Sizes have grown quickly. GPT-3 took 2,048 tokens in 2020. By 2026, Google said many of its Gemini models take 1 million tokens or more, which it likens to about eight average-length novels.³

The limit shows up in everyday use. A long contract or codebase may have to be split before a model can work through it. In a very long chat, the earliest messages can fall outside the window, and the model loses track of what they said. Some companies claim far larger windows: in 2024, Magic said it had reached 100 million tokens.⁴

A bigger window does not guarantee better reading. Models tend to use facts near the start or end of a long input more reliably than facts buried in the middle.⁵

The window is also different from long-term knowledge. What a model learned in training stops at its knowledge cutoff. Material in the window serves only the current request and has to be sent again to be used again.

The idea is older than the phrase. In 1949 Warren Weaver proposed settling an ambiguous word’s meaning by also looking at the N words on either side of it. Language researchers later used context window for such a span: the few words around a target word, used to learn what words mean. By 2020, Google and OpenAI researchers used the phrase for the whole stretch of text a transformer model reads at once. Within three years its size had become a headline figure in model launches.