Start a project
Foundations Reviewed 24 Sep 2026 · 23 sources

large language model

LLM /ˌlɑːrdʒ ˈlæŋɡwɪdʒ mɑːdl/ noun Plural large language models · abbreviation LLM, plural LLMs
Definition

A neural network trained on a vast body of text to predict the next word, which lets it write, summarize, translate and answer questions.

Examples
  • OpenAI’s GPT-3, with 175 billion parameters
  • The GPT-3.5 model behind ChatGPT at its November 2022 launch
  • Meta’s LLaMA, released to researchers in four sizes
  • Anthropic’s Claude models, first released publicly in March 2023
A tall stack of books and web pages feeds into a box marked model. The box writes a sentence one word tile at a time, choosing the next tile from three candidates.
Inkling · The next word, predicted from patterns in vast textFig. L1
Earliest AI use
2007
Google’s n-gram models for machine translation, from 2 trillion tokens.
Current meaning
2018
Neural networks pretrained on vast text; Merriam-Webster’s first known use.
Also called
Language model
The broader term, often used for LLMs, as in the GPT-3 paper’s title.
§01

In plain words

In AI, a large language model (LLM) is a neural network trained on huge amounts of text to predict the next word, the engine behind chatbots like ChatGPT.1

A large language model is a program that has learned how text tends to continue. It learns this from an enormous amount of writing, such as web pages and books. Given a prompt, it predicts a likely next token, a word or piece of a word, adds it and repeats. That single skill lets one model draft emails, summarize reports, translate and write code.

The word large refers to both the training text and the model’s weights, the numbers adjusted during training. OpenAI’s GPT-3 had 175 billion of them in 2020 and could take on a new task from a few examples in its prompt. The label is relative. Google called its 2007 translation models large. They were simple word statistics, yet their training data rivals that of today’s models.2

Chatbots such as ChatGPT are products built on these models, with extra training for conversation. Two broader terms are often mixed up with this one. Generative AI covers any system that creates content, including images. A foundation model is any large model trained on broad data and adapted to many tasks, whatever kind of data it handles. An LLM is the kind that works with language. Some, such as OpenAI’s GPT-4, also accept images, which makes them multimodal.3

Critics question what these models understand. Emily Bender and Timnit Gebru co-wrote a paper on their risks, “On the Dangers of Stochastic Parrots”. It warned of energy costs, biased training data and fluent text that can mislead readers.4 The models’ confident errors are known as hallucinations.

§02

How it works

Building one takes two main stages. The first is pretraining. A transformer, the network design Google researchers proposed in 2017, reads the training text and guesses each next token. After each guess, its weights shift slightly toward the right answer. Researchers found in 2020 that prediction error falls smoothly and predictably as the model, the data and the computing power grow.5

Pretraining produces a base model. It continues text fluently but does not reliably follow instructions, and making it bigger does not by itself fix that. The second stage starts with fine-tuning on examples of good answers. Then comes reinforcement learning from human feedback: people rank the model’s replies, and it is trained toward the ones they prefer. In a 2022 study, people preferred a 1.3-billion-parameter model trained this way to the 175-billion-parameter GPT-3.6

In use, a step called inference, the model writes one token at a time. Each new token joins the input before the next is chosen, and the whole exchange must fit in its context window.

Fig. L2 · One request, three stages of trainingIllustrative, not measured

Write a one-line thank-you note to a colleague who covered my shift.

After pretraining
Continues the text as if it were a list, adding another writing exercise
After fine-tuning
Writes a thank-you note, following the instruction
After human feedback
Writes a warm, specific note in the style raters preferred
The same request sent to a model after each stage. The request and replies are invented to show typical behavior.
§03

Origin and first known use

A language model estimates how likely a sequence of words is. The earliest known AI use of the full phrase is a 2007 Google paper, “Large Language Models in Machine Translation”. Thorsten Brants and his co-authors built n-gram models, counts of short word sequences, from up to 2 trillion tokens. From 2018 the phrase came to mean neural networks, mostly transformers, pretrained on huge bodies of text. Merriam-Webster dates that sense to 2018 and added it in 2023. Some researchers argue the 2007 models were large in one real sense, since their training data rivals that of modern models.

  1. 2007 Google researchers title a paper “Large Language Models in Machine Translation”. Their models are statistics on word sequences drawn from up to 2 trillion tokens of text.
    Brants, Popat, Xu, Och and Dean, “Large Language Models in Machine Translation”, EMNLP-CoNLL 2007 aclanthology.org →: source for 2007
  2. 2017 Vaswani and colleagues, most of them at Google, propose the transformer, a network architecture based solely on attention mechanisms. OpenAI builds on it the following year.
    Vaswani et al., “Attention Is All You Need”, 2017 arXiv 1706.03762 →: source for 2017
  3. 2018 OpenAI pretrains a transformer on a few thousand books, then fine-tunes it for each task. It calls this a large model compared with prior work.
    OpenAI, “Improving language understanding with unsupervised learning”, 11 June 2018 openai.com →: source for 2018
  4. 2020 OpenAI’s GPT-3 has 175 billion parameters, ten times any earlier non-sparse language model, and takes on new tasks from a few examples in its prompt.
    Brown et al., “Language Models are Few-Shot Learners”, 2020 arXiv 2005.14165 →: source for 2020
  5. 2020 Timnit Gebru says Google forced her out after a dispute over a paper she co-wrote on the risks of large language models, titled “On the Dangers of Stochastic Parrots”.
    Karen Hao, MIT Technology Review, 4 December 2020 technologyreview.com →: source for 2020
  6. 2021 Bommasani and colleagues coin foundation model for models trained on broad data at scale and adaptable to many tasks, such as BERT, DALL-E and GPT-3.
    Bommasani et al., “On the Opportunities and Risks of Foundation Models”, 2021 arXiv 2108.07258 →: source for 2021
  7. 2022 OpenAI launches ChatGPT, a conversational model fine-tuned from its GPT-3.5 series using reinforcement learning from human feedback.
    OpenAI, “Introducing ChatGPT”, 30 November 2022 openai.com →: source for 2022
  8. 2023 Merriam-Webster adds large language model in its September 2023 update of 690 new words and meanings, alongside generative AI.
    Merriam-Webster, “We Added 690 New Words to the Dictionary for September 2023” merriam-webster.com →: source for 2023
§04

In the wild

9 sightings · 2019–2025 · oldest first
  1. Paper24 Aug 2019

    “Large language models have a range of beneficial uses: they can assist in prose, poetry, and programming; analyze dataset biases; and more.”

    Solaiman et al., OpenAI report arXiv 1908.09203 →: Solaiman et al., OpenAI report, 24 Aug 2019Archived →: copy of Solaiman et al., OpenAI report, 24 Aug 2019
  2. Headline4 Dec 2020

    “We read the paper that forced Timnit Gebru out of Google. Here’s what it says.”

    MIT Technology Review technologyreview.com →: MIT Technology Review, 4 Dec 2020Archived →: copy of MIT Technology Review, 4 Dec 2020
  3. Paper15 Jun 2022

    “This paper instead discusses an unpredictable phenomenon that we refer to as emergent abilities of large language models.”

    Wei et al., arXiv preprint arXiv 2206.07682 →: Wei et al., arXiv preprint, 15 Jun 2022Archived →: copy of Wei et al., arXiv preprint, 15 Jun 2022
  4. Blog24 Feb 2023

    “…today we are publicly releasing LLaMA (Large Language Model Meta AI), a state-of-the-art foundational large language model…”

    Meta AI, model announcement ai.meta.com →: Meta AI, model announcement, 24 Feb 2023Archived →: copy of Meta AI, model announcement, 24 Feb 2023
  5. Blog11 Mar 2023

    “That Stable Diffusion moment is happening again right now, for large language models…”

    Simon Willison, on his blog simonwillison.net →: Simon Willison, on his blog, 11 Mar 2023Archived →: copy of Simon Willison, on his blog, 11 Mar 2023
  6. Docs2 Feb 2024

    “Large language models (LLMs) will introduce epoch-defining changes comparable to the invention of the internet.”

    House of Lords Communications and Digital Committee, report summary parliament.uk →: House of Lords Communications and Digital Committee, report summary, 2 Feb 2024Archived →: copy of House of Lords Communications and Digital Committee, report summary, 2 Feb 2024
  7. Headline4 Mar 2024

    “Large language models can do jaw-dropping things. But nobody knows exactly why.”

    MIT Technology Review technologyreview.com →: MIT Technology Review, 4 Mar 2024Archived →: copy of MIT Technology Review, 4 Mar 2024
  8. Paper24 Jul 2024

    “A large language model (LLM) is a computational system, typically a deep neural network with a large number of tunable parameters (i.e., weights)…”

    Melanie Mitchell, Open Encyclopedia of Cognitive Science oecs.mit.edu →: Melanie Mitchell, Open Encyclopedia of Cognitive Science, 24 Jul 2024
  9. Court record23 Jun 2025

    “…the AI firm selected various sets and subsets of digitized books to train various large language models under development to power its AI services.”

    Judge William Alsup, order in Bartz v. Anthropic Opinion →: Judge William Alsup, order in Bartz v. Anthropic, 23 Jun 2025Archived →: copy of Judge William Alsup, order in Bartz v. Anthropic, 23 Jun 2025
§05

Not to be confused with

  • Foundation model

    A foundation model is the broader category: any large model trained on broad data and adapted to many tasks. It may work with text, images or other data. An LLM is a foundation model that works with language.

    See foundation model →

  • Generative AI

    Generative AI is the umbrella term for systems that create new content, such as text or images. LLMs are the text-producing part of it; an image generator is generative AI but not an LLM.

    See generative AI →

  • Chatbot

    A chatbot is the product a person talks to, such as ChatGPT. It runs on an LLM given extra training for conversation. At launch, ChatGPT used a model from OpenAI’s GPT-3.5 series.

  • Multimodal model

    A multimodal model handles more than one kind of data, such as images and text. Some LLMs are also multimodal: OpenAI called GPT-4 a large multimodal model because it accepts images as input.

    See multimodal →
§06

Questions

  • What is a large language model in simple terms?

    It is a program that has learned from a vast amount of text how writing tends to continue. Given a prompt, it predicts a likely next word or word fragment, adds it and repeats. That single skill lets it draft, summarize, translate and answer questions. Chatbots such as ChatGPT are built on these models, with extra training for conversation.

  • Is ChatGPT a large language model?

    Not exactly. ChatGPT is a chatbot, a product built on a large language model. When OpenAI launched it in November 2022, it said ChatGPT was fine-tuned from a model in its GPT-3.5 series using reinforcement learning from human feedback. The underlying model supplies the language ability, and the product adds the conversation around it.

  • What is the difference between an LLM and a foundation model?

    A foundation model is the broader category. Bommasani and colleagues coined the term in 2021 for models trained on broad data at scale and adaptable to a wide range of tasks. Their examples included the image generator DALL-E as well as GPT-3. A large language model is a foundation model that works with language.

  • What makes a language model large?

    Two things: the amount of text it learns from and the number of weights, the adjustable numbers inside it. OpenAI’s GPT-3 had 175 billion weights in 2020, ten times any earlier model of its kind. The label is relative. Google called its 2007 translation models large; they were simple word statistics, but their 2 trillion training tokens rival modern training sets.

  • How are large language models trained?

    In two main stages. Pretraining has the model predict the next token across a huge body of text, adjusting its weights after every guess. Fine-tuning then teaches it to follow instructions, often with reinforcement learning from human feedback, where people rank its replies. In a 2022 study, people preferred a 1.3-billion-parameter model trained this way to the 175-billion-parameter GPT-3.

§07

Watch

§09

Cite this entry

“Large language model.” SUPERR_dictionary, Superrduperr, 24 Sept. 2026, superrduperr.com/dictionary/large-language-model.

First published 24 Sep 2026 · Last reviewed 24 Sep 2026 · Report an error →