weights
The numbers inside an AI model, adjusted during training, that set how much each input counts at every step of its calculation and together store what it has learned.
- The 175 billion learned numbers inside OpenAI’s GPT-3
- Meta’s LLaMA model files, leaked as a torrent in 2023
- 1,000 electrically adjustable resistors in the 1960s Madaline I machine
- Current meaning
- 2019
- The trained model as one asset: OpenAI withholds GPT-2’s weights.
- Also called
- Parameters
- Strictly, parameters also include the bias added at each unit.
In plain words
In AI, weights are the numbers a model adjusts during training, which set how it turns inputs into outputs and together make up the trained model.1
A neural network is built from simple units joined by connections. Each connection carries a number, its weight, which sets how much one unit’s signal counts at the next. A model’s weights are all of these numbers together. Training adjusts them until the model’s outputs fit its examples. What the model learns is held in its weights, which are separate from the text or image it is processing at any moment.2
Modern large language models have billions of weights: OpenAI’s GPT-3 had 175 billion in 2020.3 The word parameters is often used for the same numbers. Strictly, parameters also include biases: one extra learned number per unit, added to its total.
Because the weights are the trained model, whoever holds them can run the model or train it further. In 2019 OpenAI withheld the full GPT-2 over concerns about malicious applications, then released it in stages. In 2023 Meta’s LLaMA, offered to researchers case by case, was posted online as a torrent. Anthropic said in 2025 that stolen weights could let someone use its models without the protections it deploys.4 A model whose weights are published for anyone to download is an open-weight model.
The weights are shaped by the training data but are not a library of it. Getty Images v Stability AI, a 2025 English case, concerned the image generator Stable Diffusion. Its judgment says the trained model does not store its training data. Models can still memorize some examples: researchers got GPT-2 to reproduce hundreds of verbatim passages from its training text.5 Weights also differ from an embedding, the list of numbers that stands for one input. They differ as well from hyperparameters, such as the learning rate, which people set to control training.
How it works
Each unit in a network takes in numbers from the layer before it. It multiplies each input by the weight on its connection and adds the results, plus a bias. It then passes the total through a simple function to the next layer. Figure W1 shows one unit.
Training sets the weights, which start as random values. The network makes a prediction, and its error is measured against the right answer. Backpropagation, described in 1986, then adjusts every weight to shrink that error. Repeated over vast numbers of examples, this is how pretraining and fine-tuning work.
In a transformer, the weights sit in large grids of numbers called matrices. GPT-3’s 175 billion weights fill just under 28,000 of them.2 After training, the weights are saved as a file and stay fixed during inference. The file’s size is the number of weights times the bytes each one takes. At 2 bytes per weight, the 176-billion-parameter BLOOM model needs 352 GB.6 Quantization stores each weight in fewer bits, which shrinks the file.
- Input 1 × weight
- 1.0 × 0.8 = 0.8
- Input 2 × weight
- 0.5 × -1.2 = -0.6
- Input 3 × weight
- 2.0 × 0.3 = 0.6
- Plus bias
- 0.1
- Total passed on
- 0.9
Origin and first known use
From Old English wiht or gewiht, heaviness, related to weigh. By the 1520s the word also meant importance. Statistics gave it a numerical use: weighted average is recorded by 1845, and a statistical sense of the verb by 1901. Neural network research applied it to the strength of a connection. In the early 1960s the Madaline I learning machine held 1,000 weights, each a resistor adjusted by electroplating. In 1986 backpropagation showed how to adjust the weights of networks with hidden units. From 2019, when OpenAI withheld GPT-2, the plural also named a whole trained model, to be released or held back.
- 1845 Weighted average is recorded in English by 1845. A statistical sense of the verb weight follows by 1901.Online Etymology Dictionary, “weight” Etymonline →: source for 1845
- 1960s In the early 1960s a Madaline I machine for pattern recognition is built with 1,000 weights. Each is a memistor, a resistor adjusted by electroplating.Widrow and Lehr, “30 Years of Adaptive Neural Networks: Perceptron, Madaline, and Backpropagation”, Proceedings of the IEEE 78, no. 9, September 1990 stanford.edu →: source for 1960s
- 1986 Rumelhart, Hinton and Williams describe backpropagation, which repeatedly adjusts the weights of a network’s connections to shrink the gap between its actual and desired outputs.Rumelhart, Hinton and Williams, “Learning representations by back-propagating errors”, Nature 323, 9 October 1986 nature.com →: source for 1986
- 2019 OpenAI withholds the full GPT-2 over concerns about malicious applications, saying it is not releasing the model weights. It releases larger versions in stages.OpenAI, “Better language models and their implications”, 14 February 2019 openai.com →: source for 2019
- 2023 A torrent of Meta’s LLaMA, which Meta had offered to researchers case by case, is posted on 4chan. Vice reports the leak on 7 March.Joseph Cox, report on the LLaMA leak, Vice, 7 March 2023 vice.com →: source for 2023
- 2023 Executive Order 14110 defines a model weight as a numerical parameter that helps determine a model’s outputs. The order was revoked in January 2025.Executive Order 14110, “Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence”, signed 30 October 2023, Federal Register, 1 November 2023 federalregister.gov →: source for 2023
- 2024 RAND researchers identify 38 distinct ways attackers could steal a frontier model’s weights, and define five security levels for protecting them.Nevo, Lahav, Karpur, Bar-On, Bradley and Alstott, “Securing AI Model Weights: Preventing Theft and Misuse of Frontier Models”, RAND, RR-A2849-1, 30 May 2024 rand.org →: source for 2024
- 2025 In Getty Images v Stability AI, the English High Court says the trained model does not store training data. Getty does not allege it stores copies of its works.Getty Images (US) Inc and others v Stability AI Ltd [2025] EWHC 2863 (Ch), judgment of Mrs Justice Joanna Smith, 4 November 2025 nationalarchives.gov.uk →: source for 2025
In the wild
-
“We describe a new learning procedure, back-propagation, for networks of neurone-like units. The procedure repeatedly adjusts the weights of the connections in the network…”
Rumelhart et al., Nature 1986 nature.com →: Rumelhart et al., Nature 1986, 9 Oct 1986Archived →: copy of Rumelhart et al., Nature 1986, 9 Oct 1986 -
“…we’re releasing the largest version (1.5B parameters) of GPT‑2 along with code and model weights to facilitate detection of outputs of GPT‑2 models.”
-
“Facebook’s Powerful Large Language Model Leaks Online”
-
“…protect model weights — the learnable parameters that encode the core intelligence of an AI — from theft by a variety of potential attackers.”
Nevo et al., RAND research report rand.org →: Nevo et al., RAND research report, 30 May 2024Archived →: copy of Nevo et al., RAND research report, 30 May 2024 -
“Many developers provide limited or no public access to the inner workings of their advanced models, including their weights.”
NTIA, report on widely available model weights ntia.gov →: NTIA, report on widely available model weights, 30 Jul 2024Archived →: copy of NTIA, report on widely available model weights, 30 Jul 2024 -
“When potential exfiltration of model weights is detected through unusual bandwidth usage, security systems can block the suspicious traffic.”
-
“As the network is exposed to the training data, the weights are iteratively updated to better satisfy an optimization criterion specified by engineers.”
Mrs Justice Joanna Smith, judgment in Getty Images v Stability AI Judgment →: Mrs Justice Joanna Smith, judgment in Getty Images v Stability AI, 4 Nov 2025Archived →: copy of Mrs Justice Joanna Smith, judgment in Getty Images v Stability AI, 4 Nov 2025 -
“In contrast, parameters are the variables, like the weights and bias, that are part of the model itself.”
Google, Machine Learning Crash Course developers.google.com →: Google, Machine Learning Crash Course, 3 Dec 2025Archived →: copy of Google, Machine Learning Crash Course, 3 Dec 2025
Not to be confused with
-
Hyperparameters
Hyperparameters are settings people choose to control training, such as the learning rate or batch size. Weights are values the model itself calculates during training.
-
Training data
Training data is the collection of examples a model learns from. Weights are the numbers training leaves behind. A 2025 English judgment says Stable Diffusion does not store its training data, though models can memorize some examples.
-
Embedding
An embedding is the list of numbers that stands for one input. Weights are all the numbers a model learns. A language model’s embedding matrix, which holds one embedding per token, is part of its weights.
-
Open-weight model
An open-weight model is one whose weights are published for anyone to download and run. The term describes how a model is released. Every neural network has weights; many developers keep them private.
Questions
What are weights in an AI model?
Weights are the numbers inside a model that training adjusts. In a neural network, each connection between two units carries a weight that sets how much one unit’s signal counts at the next. Together they hold what the model has learned. Large models have billions: OpenAI’s GPT-3 had 175 billion in 2020. A 2023 US executive order, since revoked, defined a model weight as a numerical parameter that helps determine a model’s outputs.
What is the difference between weights and parameters?
In everyday use there is little difference, and many sources treat the words as one. RAND, for example, describes model weights as a model’s learnable parameters. Strictly, parameters are all the numbers a model learns in training: the weights plus a bias at each unit. Hyperparameters are different. They are settings, such as the learning rate, that people choose to control training.
Why do AI companies guard their model weights?
Because the weights are the trained model, whoever holds them can run it. Anthropic said in 2025 that stolen weights could let someone use its models without the protections it deploys. It described watching for unusual outbound traffic that could signal weights being copied out. A 2024 RAND report identified 38 distinct ways attackers could try to steal a frontier model’s weights, and defined five security levels for protecting them.
Do model weights contain the training data?
Not as a stored copy, according to a 2025 English judgment. In Getty Images v Stability AI, the High Court said the trained image model does not store its training data. Training shapes the weights instead. Models can still memorize some examples, though. In 2020 researchers got GPT-2 to reproduce hundreds of verbatim passages from its training data, including names, phone numbers and code.
How much storage do model weights need?
Multiply the number of weights by the bytes each one takes. At half precision, which uses 16 bits, each weight takes 2 bytes. The 176-billion-parameter BLOOM model therefore needs 352 GB for its weights. Quantization stores each weight in fewer bits, which shrinks the total.
Watch
- But what is a neural network? | Deep learning chapter 1An animated walk through a small network, its 13,002 weights and biases, and what each weight does.
- Gradient descent, how neural networks learn | Deep Learning Chapter 2How training starts from random weights and biases and nudges them to lower a cost function.
Cite this entry
“Weights.” SUPERR_dictionary, Superrduperr, 24 Sept. 2026, superrduperr.com/
Superrduperr. (2026, September 24). Weights. In SUPERR_dictionary. https://superrduperr.com/
Superrduperr. “Weights.” SUPERR_dictionary. September 24, 2026. https://superrduperr.com/