foundation model
A large AI model, trained on a vast and varied body of data, that can be adapted by fine-tuning or prompting to many different tasks.
- The large language models behind ChatGPT and Google Bard
- Stability AI’s Stable Diffusion, which makes images from text
- Meta’s Segment Anything, which outlines any object in an image
- Microsoft’s Aurora, which forecasts weather and air pollution
- Coined by
- Bommasani et al.
- Stanford’s Center for Research on Foundation Models, August 2021.
- Earliest AI use
- 2021
- Report posted 16 August 2021; its examples: BERT, DALL-E, GPT-3.
- Current meaning
- 2023
- Taken up by industry and government: Amazon Bedrock, a UK taskforce.
- Also called
- GPAI model
- Short for general-purpose AI model, the EU AI Act’s legal term.
In plain words
In AI, a foundation model is a large model trained once on broad data and then adapted to many tasks, from chatbots to image tools.
Many AI systems are built for one job, such as filtering spam. A foundation model works the other way around. It is trained once on a vast and varied body of data, and many different products are then built on top of it. Apple, for example, adapts the foundation models inside Apple Intelligence for writing, summarizing notifications and other tasks.
Stanford researchers named the idea in 2021. They considered large language model but judged it too narrow, since their focus was not only language. They chose foundation because a foundation is built first and is unfinished on its own.1
Many foundation models are large language models, such as the models behind ChatGPT. Others work with other kinds of data. Announcing its Bedrock service in 2023, Amazon listed the image generator Stable Diffusion as a foundation model alongside language models.2 Microsoft trained Aurora, a foundation model of the atmosphere, on more than a million hours of weather and climate simulations.3
Governments took up the term in 2023. The UK funded a Foundation Model Taskforce, then renamed it the Frontier AI Taskforce that September. A US executive order defined a dual-use foundation model. The EU’s AI Act settled on a different legal term, general-purpose AI model.4
The name drew criticism from the start. Gary Marcus and Ernest Davis called it a misleading relabeling of pretrained language models. The computer vision researcher Jitendra Malik said the models had no foundations at all.
How it works
Building a foundation model starts with pretraining. The model learns from a vast collection of data that no one has labeled, such as web text, images or weather simulations. It trains itself by predicting parts of that data, for instance the next word in a sentence. What it learns is stored in its weights, the numbers adjusted during training.
The second stage is adaptation. Fine-tuning continues training on a small set of examples for one task. BERT, a 2018 model, needed only one extra output layer to set new records on eleven language tasks this way. Other models are adapted by prompting alone, with no further training. When the prompt includes no worked examples, this is called zero-shot prompting. Meta designed Segment Anything to work like this, outlining objects in kinds of images it never saw in training.
The Stanford report named two effects of this approach. The first is emergence: scale produces abilities that nobody trained for directly. The second is homogenization: when many products share a few models, a defect in one foundation model is inherited by everything adapted from it.5
How a model is released is a separate choice. Meta released Segment Anything publicly, with the dataset of more than a billion masks it was trained on.6 A model whose weights anyone can download is called an open-weight model. Other foundation models are offered as a service through an API, as on Amazon Bedrock.
Origin and first known use
Rishi Bommasani and colleagues at Stanford’s Center for Research on Foundation Models coined the term in a report posted on 16 August 2021. They had weighed existing names. Large language model seemed too narrow, self-supervised model too tied to one training method, and pretrained model implied that the important work came after pretraining. They chose foundation for something built first and unfinished alone, and preferred it to foundational, which suggested fundamental principles. Critics called the name misleading from the start. Governments adopted it in 2023, but the EU’s final AI Act uses general-purpose AI model.
- 2018 BERT, a model for language understanding, is pretrained on unlabeled text, then fine-tuned with one extra output layer, setting new records on eleven tasks.Devlin et al., “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding”, 2018 arXiv 1810.04805 →: source for 2018
- 2021 Stanford researchers coin foundation model for models trained on broad data at scale and adaptable to many tasks. Their examples include BERT, DALL-E and GPT-3.Bommasani et al., “On the Opportunities and Risks of Foundation Models”, Stanford CRFM, 16 August 2021 arXiv 2108.07258 →: source for 2021
- 2021 Gary Marcus and Ernest Davis object to the new name and report Jitendra Malik telling Stanford’s workshop that the models have no foundations.Marcus and Davis, “Has AI found a new Foundation?”, The Gradient, 11 September 2021 thegradient.pub →: source for 2021
- 2023 Amazon announces Bedrock, a service offering foundation models from AI21 Labs, Anthropic, Stability AI and Amazon through one API.Swami Sivasubramanian, “Announcing New Tools for Building with Generative AI on AWS”, AWS Machine Learning Blog, 13 April 2023 aws.amazon.com →: source for 2023
- 2023 The UK government puts £100 million into a Foundation Model Taskforce, describing foundation models as including large language models such as ChatGPT and Google Bard.Department for Science, Innovation and Technology and Prime Minister’s Office, “Initial £100 million for expert taskforce to help UK build and adopt next generation of safe AI”, GOV.UK, 24 April 2023 gov.uk →: source for 2023
- 2023 In September the UK renames its Foundation Model Taskforce the Frontier AI Taskforce, to focus on risks that grow as systems become more capable.Department for Science, Innovation and Technology, “Frontier AI Taskforce: first progress report”, GOV.UK, 7 September 2023 gov.uk →: source for 2023
- 2023 Executive Order 14110 defines a dual-use foundation model by broad training data, tens of billions of parameters and serious security risks. It was revoked in January 2025.Executive Order 14110, “Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence”, signed 30 October 2023, Federal Register, 1 November 2023 federalregister.gov →: source for 2023
- 2024 The EU AI Act, Regulation 2024/1689 of 13 June 2024, regulates general-purpose AI models. It defines them by significant generality and competence at a wide range of distinct tasks.Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 on artificial intelligence eur-lex.europa.eu →: source for 2024
In the wild
-
“We call these models foundation models to underscore their critically central yet incomplete character.”
Bommasani et al., Stanford CRFM report arXiv 2108.07258 →: Bommasani et al., Stanford CRFM report, 16 Aug 2021Archived →: copy of Bommasani et al., Stanford CRFM report, 16 Aug 2021 -
“Foundation models risk exacerbating ML’s ethical challenges”
-
“…we think that relabeling “pretrained language models” as foundation models is misleading.”
Gary Marcus and Ernest Davis, in The Gradient thegradient.pub →: Gary Marcus and Ernest Davis, in The Gradient, 11 Sep 2021Archived →: copy of Gary Marcus and Ernest Davis, in The Gradient, 11 Sep 2021 -
“…a foundation model for image segmentation: a promptable model that is trained on diverse data and that can adapt to specific tasks…”
Meta AI, Segment Anything announcement ai.meta.com →: Meta AI, Segment Anything announcement, 5 Apr 2023Archived →: copy of Meta AI, Segment Anything announcement, 5 Apr 2023 -
“…very large models that are pre-trained on vast amounts of data and commonly referred to as Foundation Models (FMs).”
Swami Sivasubramanian, AWS Machine Learning Blog aws.amazon.com →: Swami Sivasubramanian, AWS Machine Learning Blog, 13 Apr 2023Archived →: copy of Swami Sivasubramanian, AWS Machine Learning Blog, 13 Apr 2023 -
“Foundation models - including large language models like ChatGPT and Google Bard - are a category of artificial intelligence systems trained on huge volumes of data…”
UK Department for Science, Innovation and Technology, press release gov.uk →: UK Department for Science, Innovation and Technology, press release, 24 Apr 2023 -
“…“dual-use foundation model” means an AI model that is trained on broad data; generally uses self-supervision; contains at least tens of billions of parameters…”
The White House, Executive Order 14110 federalregister.gov →: The White House, Executive Order 14110, 1 Nov 2023Archived →: copy of The White House, Executive Order 14110, 1 Nov 2023 -
“…foundation models, large systems capable to competently perform a wide range of distinctive tasks, such as generating video, text, images…”
Council of the EU, press release on the AI Act deal consilium.europa.eu →: Council of the EU, press release on the AI Act deal, 9 Dec 2023Archived →: copy of Council of the EU, press release on the AI Act deal, 9 Dec 2023 -
“The foundation models built into Apple Intelligence have been fine-tuned for user experiences such as writing and refining text, prioritizing and summarizing notifications…”
Apple Machine Learning Research, blog post machinelearning.apple.com →: Apple Machine Learning Research, blog post, 10 Jun 2024Archived →: copy of Apple Machine Learning Research, blog post, 10 Jun 2024
Not to be confused with
-
Large language model
A large language model is a foundation model that works with language. Foundation model is the wider category, covering image, video, audio and weather models too. Its coiners rejected large language model as too narrow.
-
Generative AI
Generative AI describes what a system does: make new content. Foundation model describes how a model is built and reused. Many foundation models are generative, but BERT, an early example, was built to understand text rather than write it.
-
General-purpose AI model
The EU AI Act’s legal term for a similar class. Where foundation model stresses broad training and adaptation, the legal test is generality: competently performing a wide range of distinct tasks.
-
Open-weight model
An open-weight model is one whose trained weights anyone can download and run. That describes how a model is released, not how it is built. A foundation model can be released either way.
Questions
What is a foundation model in simple terms?
It is a large AI model trained once on a vast and varied body of data, then adapted for many different jobs. Adaptation can mean further training on a few examples, called fine-tuning, or simply a prompt. One model can sit under many products. Examples include the language models behind ChatGPT, the image generator Stable Diffusion and Microsoft’s weather model Aurora.
What is the difference between a foundation model and an LLM?
A large language model is one kind of foundation model: the kind that works with language. Foundation model is the wider category, which also covers models for images, video, audio and weather. The Stanford researchers who coined the term in 2021 considered calling these models large language models. They rejected that name as too narrow, because their focus was not only language.
Who coined the term foundation model, and why?
Rishi Bommasani and colleagues at Stanford’s Center for Research on Foundation Models coined it in a report posted on 16 August 2021. They chose foundation because a foundation is built first and is unfinished on its own, needing more building to be useful. Critics objected within weeks. Gary Marcus and Ernest Davis called the new name a misleading relabeling of pretrained language models.
What is a dual-use foundation model?
It is a legal term from US Executive Order 14110, signed on 30 October 2023. The order defined it as a model trained on broad data, generally by self-supervision, with at least tens of billions of parameters. It also performs, or could easily be made to perform, tasks that pose serious security risks. The order was revoked in January 2025.
What does the EU AI Act call a foundation model?
The final text of the EU AI Act, dated 13 June 2024, uses the term general-purpose AI model. The European Commission explains it as a model that displays significant generality and competently performs a wide range of distinct tasks. The political deal struck in December 2023 still used foundation models, for large systems able to perform a wide range of distinctive tasks.
Watch
- Workshop on Foundation Models (Welcome and Introduction)Percy Liang presents foundation models at Stanford’s August 2021 workshop, after an introduction by Fei-Fei Li.
- Workshop on Foundation Models (Session II: Technological Foundations)Talks and a panel where Jitendra Malik argued against treating these models as the foundation of AI.
Cite this entry
“Foundation model.” SUPERR_dictionary, Superrduperr, 24 Sept. 2026, superrduperr.com/
Superrduperr. (2026, September 24). Foundation model. In SUPERR_dictionary. https://superrduperr.com/
Superrduperr. “Foundation model.” SUPERR_dictionary. September 24, 2026. https://superrduperr.com/