Start a project
Foundations Reviewed 24 Sep 2026 · 17 sources

quantization

/ˌkwɑːn.təˈzeɪ.ʃən/ noun Plural quantizations · verb quantize · adjective quantized
Definition

A technique that stores an AI model’s weights in fewer bits each, so the model needs less memory and runs faster, usually at some cost in accuracy.

Examples
  • A 70-billion-weight model stored at 4 bits instead of 16
  • Google’s Inception image model, shrunk from 91 MB to 23 MB
  • OpenAI’s gpt-oss-120b, with most weights in 4.25-bit MXFP4
  • Meta’s quantized Llama 3.2 1B and 3B models
Two rulers, one above the other. Dots on the finely marked upper ruler are each moved by an arrow to the nearest of a few widely spaced marks on the lower ruler. Beside them, a tall memory block shrinks to a quarter of its height.
Inkling · Each weight rounded to the nearest of a few allowed valuesFig. Q1
Current meaning
2011
A trained network cut to 8 bits to save memory and run faster.
Opposite
Full precision
Weights kept in the 16-bit or 32-bit form used in training.
§01

In plain words

In AI, quantization is storing a model’s weights in fewer bits, often 8 or 4 instead of 16, so the model uses less memory and runs faster.

A trained AI model is, underneath, a very large set of numbers called weights. Each weight is usually stored in 16 or 32 bits. Quantization rounds every weight to a shorter number, often 8 or 4 bits, so the whole model takes less memory. The design of the model and the number of weights stay the same; only their precision drops. A 2017 Google post compared it to rain: to tell light rain from heavy rain, nobody needs a count of the drops. Quantizing Google’s Inception image model shrank it from 91 MB to 23 MB.1

It matters most at inference, when a finished model answers requests and memory is often the limit. At 16 bits, a large language model with 70 billion weights, such as Meta’s Llama 3 70B, needs at least 140 GB of fast memory.2 At 4 bits its weights need a quarter of that. In 2023 the llama.cpp project set out to run Meta’s LLaMA on a MacBook using 4-bit quantization. Labs now ship quantized models themselves. OpenAI released its open-weight gpt-oss models in 2025 with most weights in MXFP4, a format of about 4.25 bits per weight.

The cost is accuracy. Rounding discards detail, and the loss tends to grow as the bits shrink. At 8 bits, results are often close to the original. The damage can also hide from automatic tests. In a 2024 study of multilingual models, Japanese results fell 1.7% on automatic tests but 16% in human ratings.3 Languages written in non-Latin scripts were hurt most.

Quantization is often confused with distillation, which trains a smaller model to imitate a larger one, and with pruning, which deletes weights. A quantized model keeps all of its weights and stores each one more coarsely.

§02

How it works

A quantizer maps each weight onto a short list of allowed values: 256 of them at 8 bits, 16 at 4 bits. A scale, stored with each group of weights, sets the range the list covers, and each weight is rounded to the nearest value. Google described its 8-bit scheme as approximating any value between a preset minimum and maximum.

Quantizing a finished model is called post-training quantization. GPTQ, published in 2022, used approximate second-order information, an estimate of how sensitive the output is to each weight, to round very large models to 3 or 4 bits. The results ran about 3.25 times faster than at 16 bits on high-end GPUs.4 The alternative, quantization-aware training, simulates the rounding while the model is still learning, so it can adapt. Meta used it for small Llama models that ran two to four times faster.5 QLoRA, from 2023, applies fine-tuning to a model whose 4-bit weights stay frozen, training only small add-on layers.

Language models add a complication. A few internal features, called outliers, dominate attention and the model’s predictions. LLM.int8() handles them separately: it keeps those dimensions at 16 bits and multiplies more than 99.9% of values in 8 bits.6

Fig. Q2 · Memory for 70 billion weights, in GBIn GB · Illustrative, not measured
  1. 32-bit 280
  2. 16-bit 140
  3. 8-bit 70
  4. 4-bit 35
Weights alone at each precision: 70 billion weights times bits per weight, divided by 8. Computed, not measured. The 16-bit value matches The Register’s figure for Llama 3 70B. A running model needs extra memory.
§03

Origin and first known use

From quantize, formed from quantum, Latin for “how much”. Merriam-Webster dates the verb to 1920, in physics, for dividing something such as energy into small, measurable steps. Signal engineers took it up for turning a continuous signal into a set of discrete levels. In 1948 W. R. Bennett analyzed such quantized signals, which let speech be sent as code. Neural network researchers applied the word to their own arithmetic by the early 1990s, while designing low-precision chips. The current sense, shrinking a trained model’s weights to save memory and time, dates from about 2011.

  1. 1920 Merriam-Webster dates the verb quantize to 1920, in physics: to subdivide something, such as energy, into small but measurable increments.
    Merriam-Webster, “quantize” merriam-webster.com →: source for 1920
  2. 1948 Bennett analyzes quantized signals, which allow only certain discrete magnitudes instead of a continuous range. Quantizing makes it possible to send speech as code.
    W. R. Bennett, “Spectra of Quantized Signals”, Bell System Technical Journal 27(3), July 1948 archive.org →: source for 1948
  3. 1993 In an early neural network use, Simard and Graf adapt backpropagation for low-resolution chips without multipliers. Each computed sum is quantized by checking one bit.
    Simard and Graf, “Backpropagation without Multiplication”, NIPS 1993 neurips.cc →: source for 1993
  4. 2011 Google researchers quantize most of a neural network to 8 bits for speed on ordinary CPUs. Its memory shrinks 3 to 4 times, at no cost in accuracy.
    Vanhoucke, Senior and Mao, “Improving the speed of neural networks on CPUs”, NIPS 2011 Deep Learning and Unsupervised Feature Learning Workshop research.google →: source for 2011
  5. 2017 Jacob and colleagues target mobile devices: quantized inference in integer-only arithmetic, plus a training procedure that preserves accuracy after quantization.
    Jacob et al., “Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference” arXiv 1712.05877 →: source for 2017
  6. 2022 Frantar and colleagues’ GPTQ cuts 175-billion-parameter models to 3 or 4 bits per weight, which the authors say lets one GPU run such a model for the first time.
    Frantar, Ashkboos, Hoefler and Alistarh, “GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers”, ICLR 2023 arXiv 2210.17323 →: source for 2022
  7. 2023 QLoRA fine-tunes a 65-billion-parameter model on a single 48 GB GPU by training small adapters on top of a frozen, 4-bit quantized model.
    Dettmers, Pagnoni, Holtzman and Zettlemoyer, “QLoRA: Efficient Finetuning of Quantized LLMs” arXiv 2305.14314 →: source for 2023
  8. 2025 OpenAI releases gpt-oss with over 90% of its weights in MXFP4, about 4.25 bits each, so the larger model fits on one 80 GB GPU.
    OpenAI, “gpt-oss-120b & gpt-oss-20b Model Card”, 5 August 2025 arXiv 2508.10925 →: source for 2025
§04

In the wild

10 sightings · 2017–2025 · oldest first
  1. Blog12 May 2017

    “Quantization is an optimization technique that uses an 8-bit integer to approximate an arbitrary value between a preset minimum and a maximum value.”

    Kaz Sato and Cliff Young, Google Cloud blog cloud.google.com →: Kaz Sato and Cliff Young, Google Cloud blog, 12 May 2017Archived →: copy of Kaz Sato and Cliff Young, Google Cloud blog, 12 May 2017
  2. Paper15 Dec 2017

    “We propose a quantization scheme that allows inference to be carried out using integer-only arithmetic…”

    Jacob et al., arXiv preprint arXiv 1712.05877 →: Jacob et al., arXiv preprint, 15 Dec 2017Archived →: copy of Jacob et al., arXiv preprint, 15 Dec 2017
  3. Paper15 Aug 2022

    “…we develop a two-part quantization procedure, LLM.int8().”

    Dettmers et al., NeurIPS 2022 arXiv 2208.07339 →: Dettmers et al., NeurIPS 2022, 15 Aug 2022Archived →: copy of Dettmers et al., NeurIPS 2022, 15 Aug 2022
  4. Paper31 Oct 2022

    “…GPTQ can quantize GPT models with 175 billion parameters in approximately four GPU hours, reducing the bitwidth down to 3 or 4 bits per weight…”

    Frantar et al., ICLR 2023 arXiv 2210.17323 →: Frantar et al., ICLR 2023, 31 Oct 2022Archived →: copy of Frantar et al., ICLR 2023, 31 Oct 2022
  5. Blog11 Mar 2023

    “4-bit quantization is a technique for reducing the size of models so they can run on less powerful hardware.”

    Simon Willison, on his blog simonwillison.net →: Simon Willison, on his blog, 11 Mar 2023Archived →: copy of Simon Willison, on his blog, 11 Mar 2023
  6. Paper23 May 2023

    “QLoRA backpropagates gradients through a frozen, 4-bit quantized pretrained language model into Low Rank Adapters…”

    Dettmers et al., arXiv preprint arXiv 2305.14314 →: Dettmers et al., arXiv preprint, 23 May 2023Archived →: copy of Dettmers et al., arXiv preprint, 23 May 2023
  7. Paper3 Jul 2024

    “…languages are disparately affected by quantization, with non-Latin script languages impacted worst…”

    Marchisio et al., Findings of EMNLP 2024 arXiv 2407.03211 →: Marchisio et al., Findings of EMNLP 2024, 3 Jul 2024Archived →: copy of Marchisio et al., Findings of EMNLP 2024, 3 Jul 2024
  8. Headline14 Jul 2024

    “Honey, I shrunk the LLM! A beginner’s guide to quantization – and testing it”

    The Register theregister.com →: The Register, 14 Jul 2024Archived →: copy of The Register, 14 Jul 2024
  9. Blog24 Oct 2024

    “We employ Quantization-Aware Training (QAT) to simulate the effects of quantization during the training of Llama 3.2 models…”

    Meta, quantized Llama 3.2 announcement ai.meta.com →: Meta, quantized Llama 3.2 announcement, 24 Oct 2024Archived →: copy of Meta, quantized Llama 3.2 announcement, 24 Oct 2024
  10. Blog5 Aug 2025

    “The weights for both gpt-oss-120b and gpt-oss-20b are freely available for download on Hugging Face and come natively quantized in MXFP4.”

    OpenAI, gpt-oss announcement openai.com →: OpenAI, gpt-oss announcement, 5 Aug 2025Archived →: copy of OpenAI, gpt-oss announcement, 5 Aug 2025
§05

Not to be confused with

  • Distillation

    Distillation trains a separate, smaller model to imitate a larger one, so the result has fewer weights. Quantization keeps the original model and every weight, and stores each weight in fewer bits.

    See distillation →

  • Pruning

    Pruning deletes weights, or whole parts of a network, that contribute little to the output. Quantization keeps every weight and rounds each one to a coarser value.

  • Model compression

    The umbrella term for all ways of shrinking a model, including pruning, distillation and quantization. Quantization is the method that lowers the precision of each number, rather than removing weights or training a new model.

§06

Questions

  • What is quantization in AI?

    Quantization is a way to shrink a trained AI model by storing its weights, the numbers it learned in training, with fewer bits. A weight kept in 16 or 32 bits is rounded to a nearby value that fits in 8 or 4. The model keeps its structure but needs less memory and usually runs faster. Google’s Inception image model shrank from 91 MB to 23 MB this way.

  • Does quantization make a model less accurate?

    Usually a little, and more as the bits shrink. At 8 bits the loss is often negligible: LLM.int8() ran models of up to 175 billion parameters without measured degradation, and GPTQ reported negligible loss at 3 or 4 bits. Automatic tests can understate the harm. In a 2024 multilingual study, Japanese results fell 1.7% on automatic tests but 16% in human ratings.

  • How much memory does a quantized model save?

    Memory scales with the bits per weight. A model with 70 billion weights needs at least 140 GB at 16 bits, which is two bytes per weight. At 8 bits that halves, and at 4 bits it falls to a quarter. OpenAI’s gpt-oss-120b stores most weights in MXFP4, about 4.25 bits each, and fits on a single 80 GB GPU. A running model also needs memory for other data.

  • What is the difference between quantization and distillation?

    Distillation trains a new, smaller model to imitate a larger one, so the result has fewer weights. Quantization keeps the same model and all of its weights, and stores each weight in fewer bits. Pruning, a third method, deletes weights that contribute little. All three aim to make a model cheaper to run, and model compression is the umbrella term for them.

  • What is quantization-aware training?

    Quantization-aware training simulates the effects of rounding while a model is still being trained, so the model learns to work at low precision. The alternative, post-training quantization, rounds a model after training has finished, as GPTQ does. Meta used quantization-aware training for quantized versions of its Llama 3.2 1B and 3B models, which ran two to four times faster than the originals.

§07

Watch

§09

Cite this entry

“Quantization.” SUPERR_dictionary, Superrduperr, 24 Sept. 2026, superrduperr.com/dictionary/quantization.

First published 24 Sep 2026 · Last reviewed 24 Sep 2026 · Report an error →