Term · Compute & AI Hardware
FP8 (8-bit floating point)
An 8-bit float defined by OCP in two variants, E4M3 favouring precision and E5M2 favouring range, used with per-tensor scaling or, in the OCP MX formats, a shared scale per block of 32 values, for training and inference on recent accelerators.
- Also known as
- E4M3, E5M2
- Type
- Term
- Domain
- Compute & AI Hardware, Numerics and precision
Notes
Hopper and MI300 class parts made FP8 training practical, roughly doubling throughput over BF16 on the same silicon.
Overview
FP8 is an 8-bit floating-point number format used in AI accelerators to store and multiply neural network values at a quarter of the size of 32-bit floats, with two standard encodings: E4M3, which keeps more precision, and E5M2, which covers a wider range. The formats were proposed in a 2022 paper by NVIDIA, Arm and Intel engineers and published as an Open Compute Project specification in 2023. FP8 is used for both training and inference, nearly always together with scale factors that fit each tensor into its narrow range.
How it works
Each FP8 value has one sign bit, a biased exponent and a few mantissa bits. E4M3 has four exponent bits and three mantissa bits with an exponent bias of 7; E5M2 has five exponent bits and two mantissa bits with a bias of 15. E5M2 follows IEEE 754 conventions, including infinities, and behaves like a shortened 16-bit half-precision float. E4M3 gives up infinities and keeps only one NaN pattern, which lets it reach a largest value of 448 instead of 240. E5M2 reaches 57,344. The authors recommend E4M3 for weights and activations and E5M2 for gradients, which need the extra range.
Eight bits cannot cover the spread of values in every tensor, so software multiplies each tensor by a scale factor before converting it, chosen so that its largest magnitude lands near the top of the format, and divides it out after the matrix multiply. The OCP specification defines conversion from wider formats, requiring round to nearest even and both a saturating mode (overflow becomes the largest value) and a non-saturating mode, but leaves the choice of scale factors to software.
The later OCP Microscaling (MX) specification takes the idea further: MXFP8 stores blocks of 32 FP8 elements that share one 8-bit power-of-two scale, so scaling adapts within a tensor rather than once per tensor.
The gain is efficiency: narrower multipliers and half the bytes of a 16-bit format, which cuts memory traffic. The cost is extra care over scaling and accuracy checks for each model.
Examples
- NVIDIA's Hopper architecture, which NVIDIA describes as implementing FP8 natively.
- Mixed-precision training that keeps weights and activations in E4M3 and gradients in E5M2, as recommended in the 2022 FP8 paper.
- Post-training quantisation of a 16-bit-trained BERT model to E4M3 for inference, which the paper reports kept accuracy where INT8 lost it.
- MXFP8 tensors under the OCP MX specification, with each block of 32 values sharing an E8M0 scale.
Why it matters
FP8 support is a prominent feature of recent data centre AI accelerators, because it can raise throughput and lower memory use for large models. Numerics engineers, accelerator architects, ML compiler developers and training engineers decide where FP8 is safe, how scale factors are managed, and how results are checked against 16-bit baselines.
In the chip lifecycle, number formats are chosen at architecture, the multiply-accumulate datapaths are designed and verified bit-exactly against reference models, software libraries and compilers add support around first silicon, and accuracy is validated model by model before deployment.
It matters most to: Distributed Training Engineer, ML Compiler Engineer, Model Optimization Engineer, NPU / AI Accelerator Architect, Numerics and Low-Precision Format Engineer
Common confusions
- Not the same as INT8
- INT8 holds evenly spaced integers and relies entirely on scale factors for range. FP8 has its own exponent, so its values are spaced more finely near zero and more widely at large magnitudes. FP8 vs INT8
- Not the same as BF16
- BF16 is a 16-bit format with the same 8-bit exponent as a 32-bit float. FP8 halves the size again and needs scaling to stay in range.
Related standards
- OCP OFP8 Revision 1.0 (8-bit floating point interchange formats E4M3 and E5M2)
- OCP MX Version 1.0 (microscaling block-scaled formats, including MXFP8)
Sources
- OCP 8-bit Floating Point Specification (OFP8) Revision 1.0 (approved 20 June 2023) (opens in a new tab), Open Compute Project, 2026-10-22
- OCP Microscaling Formats (MX) Specification Version 1.0 (September 2023) (opens in a new tab), Open Compute Project, 2026-10-22
- FP8 Formats for Deep Learning (Micikevicius et al., arXiv:2209.05433, September 2022) (opens in a new tab), arXiv, 2026-10-22
- NVIDIA, Arm, and Intel Publish FP8 Specification for Standardization as an Interchange Format for AI (Shar Narasimhan, 14 September 2022) (opens in a new tab), NVIDIA Technical Blog, 2026-10-22
Related terms
Part of the Compute & AI Hardware glossary, with every term in the topic and the lifecycles, tools and standards that use them.
Where it appears on the Hub
Used on 5 pages of the Hub. In each group, the pages that mention it most come first.
Suggest an improvement
Suggest an improvement to this entry
A clearer definition, an example, a correction or a source an editor can check. Editors review every suggestion before the entry changes.
Last reviewed . Written by The Semiconductor Hub editorial team. The full glossary