Roofline performance model
Plot attainable performance against arithmetic intensity for a chip's compute and memory limits, and place your kernels and models on it.
AI, systems and startupFree to useHow it worksSourcesWhere it helps
Fills the inputs below with typical figures. Replace any of them with your own.
- Ridge point at BF16 or FP16
- 295.4 FLOP per byte
- 989.5 TFLOP/s sustained peak over 3.35 TB/s
- Memory bound kernels
- 4 of 6
- 2 compute bound
- First kernel, attainable
- 989.5 TFLOP/s
- 100.0% of the sustained peak
- Roofline, HBM (solid line, circles)
- Kernels, attainable (dashed line, squares)
Show the data as a table
| Arithmetic intensity (FLOP per byte, log scale) | Roofline, HBM | Kernels, attainable |
|---|---|---|
| 0.1 | 0.335 | |
| 0.167 | 0.558 | 0.558 |
| 0.178 | 0.596 | |
| 0.316 | 1.06 | |
| 0.562 | 1.88 | |
| 1 | 3.35 | 3.35 |
| 1 | 3.35 | |
| 1.78 | 5.96 | |
| 3.16 | 10.6 | |
| 5.62 | 18.8 | |
| 10 | 33.5 | |
| 17.8 | 59.6 | |
| 31.6 | 106 | |
| 56.2 | 188 | |
| 62.1 | 208 | 208 |
| 100 | 335 | |
| 178 | 596 | |
| 287 | 962 | 962 |
| 295 | 990 | |
| 316 | 990 | |
| 562 | 990 | |
| 1,000 | 990 | |
| 1,780 | 990 | |
| 2,050 | 990 | 990 |
| 2,730 | 990 | 990 |
| 3,160 | 990 | |
| 5,620 | 990 | |
| 10,000 | 990 |
Both axes are logarithmic. The lines join the kernel markers in order of intensity so that the markers can be read; each marker is one kernel.
Sign in to save your inputs and come back to them later.
| Kernel | Precision | Intensity (FLOP per byte) | Ridge point (FLOP per byte) | Attainable | Limited by | Share of peak | Time at the roofline | Measured over attainable |
|---|---|---|---|---|---|---|---|---|
| GEMM, large (8192 x 8192 x 8192) | BF16 or FP16 | 2,731 | 295.4 | 989.5 TFLOP/s | Compute bound | 100.0% | 1.11 ms | none entered |
| GEMV in LLM decode (4096 x 4096 weights, batch 1) | BF16 or FP16 | 0.9995 | 295.4 | 3.348 TFLOP/s | Memory bound (HBM) | 0.3% | 10 µs | none entered |
| Attention, fused (sequence 4096, head size 128) | BF16 or FP16 | 2,048 | 295.4 | 989.5 TFLOP/s | Compute bound | 100.0% | 8.68 µs | none entered |
| Attention, unfused (sequence 4096, head size 128) | BF16 or FP16 | 62.06 | 295.4 | 207.9 TFLOP/s | Memory bound (HBM) | 21.0% | 41.3 µs | none entered |
| Convolution, 3 x 3 (batch 32, 64 channels, 56 x 56) | BF16 or FP16 | 287.2 | 295.4 | 962 TFLOP/s | Memory bound (HBM) | 97.2% | 7.69 µs | none entered |
| Elementwise add (100 million BF16 values) | BF16 or FP16 | 0.1667 | 295.4 | 0.5583 TFLOP/s | Memory bound (HBM) | 0.1% | 179 µs | none entered |
| Precision | Sustained peak | Ridge point (FLOP per byte) |
|---|---|---|
| FP64 | 67 TFLOP/s | 20 |
| FP32 | 67 TFLOP/s | 20 |
| TF32 | 494.5 TFLOP/s | 147.6 |
| BF16 or FP16 | 989.5 TFLOP/s | 295.4 |
| FP8 | 1,979 TFLOP/s | 590.7 |
| INT8 | 1,979 TFLOP/s | 590.7 |
| Kernel | Bandwidth x 0.5 | Bandwidth x 1 | Bandwidth x 2 | Bandwidth x 4 |
|---|---|---|---|---|
| GEMM, large (8192 x 8192 x 8192) | 990 TFLOP/s | 990 TFLOP/s | 990 TFLOP/s | 990 TFLOP/s |
| GEMV in LLM decode (4096 x 4096 weights, batch 1) | 1.67 TFLOP/s | 3.35 TFLOP/s | 6.7 TFLOP/s | 13.4 TFLOP/s |
| Attention, fused (sequence 4096, head size 128) | 990 TFLOP/s | 990 TFLOP/s | 990 TFLOP/s | 990 TFLOP/s |
| Attention, unfused (sequence 4096, head size 128) | 104 TFLOP/s | 208 TFLOP/s | 416 TFLOP/s | 832 TFLOP/s |
| Convolution, 3 x 3 (batch 32, 64 channels, 56 x 56) | 481 TFLOP/s | 962 TFLOP/s | 990 TFLOP/s | 990 TFLOP/s |
| Elementwise add (100 million BF16 values) | 0.279 TFLOP/s | 0.558 TFLOP/s | 1.12 TFLOP/s | 2.23 TFLOP/s |
| Kernel | Peak x 0.5 | Peak x 1 | Peak x 2 | Peak x 4 |
|---|---|---|---|---|
| GEMM, large (8192 x 8192 x 8192) | 495 TFLOP/s | 990 TFLOP/s | 1,980 TFLOP/s | 3,960 TFLOP/s |
| GEMV in LLM decode (4096 x 4096 weights, batch 1) | 3.35 TFLOP/s | 3.35 TFLOP/s | 3.35 TFLOP/s | 3.35 TFLOP/s |
| Attention, fused (sequence 4096, head size 128) | 495 TFLOP/s | 990 TFLOP/s | 1,980 TFLOP/s | 3,960 TFLOP/s |
| Attention, unfused (sequence 4096, head size 128) | 208 TFLOP/s | 208 TFLOP/s | 208 TFLOP/s | 208 TFLOP/s |
| Convolution, 3 x 3 (batch 32, 64 channels, 56 x 56) | 495 TFLOP/s | 962 TFLOP/s | 962 TFLOP/s | 962 TFLOP/s |
| Elementwise add (100 million BF16 values) | 0.558 TFLOP/s | 0.558 TFLOP/s | 0.558 TFLOP/s | 0.558 TFLOP/s |
A higher ridge point means more of your kernels sit on the memory side. Preset data sheet peaks, with the sustained shares above. No result means the data sheet gives no peak at that precision.
- NVIDIA H100 SXM295.4
- NVIDIA B200 (HGX, 1000 W)292.2
- AMD Instinct MI300X246.7
- Google TPU v5p166
- Intel Gaudi 3 (OAM)453.5
Show the data as a table
| Item | Ridge point (FLOP per byte) |
|---|---|
| NVIDIA H100 SXM | 295.4 |
| NVIDIA B200 (HGX, 1000 W) | 292.2 |
| AMD Instinct MI300X | 246.7 |
| Google TPU v5p | 166 |
| Intel Gaudi 3 (OAM) | 453.5 |
| Kernel | NVIDIA H100 SXM | NVIDIA B200 (HGX, 1000 W) | AMD Instinct MI300X | Google TPU v5p | Intel Gaudi 3 (OAM) |
|---|---|---|---|---|---|
| GEMM, large (8192 x 8192 x 8192) | 990 TFLOP/s | 2,250 TFLOP/s | 1,310 TFLOP/s | 459 TFLOP/s | 1,680 TFLOP/s |
| GEMV in LLM decode (4096 x 4096 weights, batch 1) | 3.35 TFLOP/s | 7.7 TFLOP/s | 5.3 TFLOP/s | 2.76 TFLOP/s | 3.7 TFLOP/s |
| Attention, fused (sequence 4096, head size 128) | 990 TFLOP/s | 2,250 TFLOP/s | 1,310 TFLOP/s | 459 TFLOP/s | 1,680 TFLOP/s |
| Attention, unfused (sequence 4096, head size 128) | 208 TFLOP/s | 478 TFLOP/s | 329 TFLOP/s | 172 TFLOP/s | 230 TFLOP/s |
| Convolution, 3 x 3 (batch 32, 64 channels, 56 x 56) | 962 TFLOP/s | 2,210 TFLOP/s | 1,310 TFLOP/s | 459 TFLOP/s | 1,060 TFLOP/s |
| Elementwise add (100 million BF16 values) | 0.558 TFLOP/s | 1.28 TFLOP/s | 0.883 TFLOP/s | 0.461 TFLOP/s | 0.617 TFLOP/s |
How it works
The roofline model shows the most a chip can do for a kernel, given two limits: how fast it can compute and how fast it can move data. A kernel's arithmetic intensity, the operations it performs for each byte it moves, decides which limit it meets first. Use it to see whether a kernel or a whole layer is memory bound or compute bound, how far it sits from the limit, and what a change in bandwidth or precision would do.
- Convert to SI units. Peaks are entered in GFLOP/s, TFLOP/s or PFLOP/s (TOPS for integer rates) and bandwidth in GB/s or TB/s, then converted to operations and bytes per second. The sustained shares scale each ceiling down to what benchmarks reach rather than the data sheet peak.
peak = entered peak x unit x sustained compute share; bandwidth = entered bandwidth x unit x sustained bandwidth share - Find each kernel's arithmetic intensity. Enter the operations and the bytes a kernel moves to or from memory, or enter its intensity directly. The bytes are those that reach the memory level being modelled, after caches have filtered the traffic.
intensity = operations / bytes moved (FLOP per byte) - Apply the roofline. Attainable performance is the lower of the compute peak and the rate memory can feed the kernel. On a log chart this is a sloped line for memory followed by a flat line for compute.
attainable = min(peak, intensity x bandwidth) - Find the ridge point. The ridge point is the intensity where the two limits meet. A kernel left of it is memory bound; a kernel right of it is compute bound. Each precision has its own peak and so its own ridge point.
ridge point = peak / bandwidth (FLOP per byte) - Add a second memory level. For a hierarchical roofline, give the bandwidth of an on-chip level such as SRAM or L2, and for each kernel the bytes moved at that level. The kernel is held by whichever limit is lowest: compute, HBM or the second level.
attainable = min(peak, intensity HBM x bandwidth HBM, intensity L2 x bandwidth L2) - Read the efficiency and the time. A measured rate divided by the attainable rate shows how much of the roofline a kernel reaches. If the operations are given, the time at the attainable rate is the lower bound on the kernel's run time.
efficiency = measured / attainable; time = operations / attainable
Assumptions
- Peaks are theoretical dense rates (no structured sparsity), taken from vendor data sheets and product pages. NVIDIA prints some peaks with sparsity; the presets here use the dense half.
- Bytes moved are the traffic that reaches HBM. A kernel whose data fits in cache has an intensity against HBM that is higher than its intensity against the cache.
- Memory and compute overlap perfectly, so the kernel runs at the lower of the two limits. Real kernels add latency, launch and synchronisation costs on top.
- The kernel presets use the usual operation counts: 2MNK for a matrix multiply, 4 s^2 d for one attention head, and one operation per value for elementwise work. The sizes and byte counts are illustrative.
Limits and warnings
- The model gives an upper bound, not a prediction. A kernel can sit well below its roof because of occupancy, tile shapes, instruction mix or communication.
- Only the limits entered are modelled: network, PCIe or storage limits, and power or clock throttling, need their own ceilings.
- One peak per precision is used. Mixed-precision kernels, and kernels whose operations run on different units (vector and matrix), should be split into rows.
- Vendors count operations in different ways (a fused multiply-add as two operations is the usual choice). Compare chips only on figures counted the same way.
- Second-level bandwidths are often not published; where the data sheet gives none, the preset leaves the level empty.
Sources
The methods and default figures rest on these public sources. The wording here is our own; read the source for its full text.
- Roofline: an insightful visual performance model for multicore architectures (opens in a new tab), Communications of the ACM, volume 52, issue 4 (Williams, Waterman, Patterson), April 2009. The method and the ridge point. The URL is the Berkeley technical report that preceded the article.
- NVIDIA H100 Tensor Core GPU (opens in a new tab), NVIDIA, page read 25 October 2026. H100 SXM peaks (printed with sparsity) and 3.35 TB/s memory bandwidth.
- ThinkSystem NVIDIA HGX B200 180GB 1000W GPU Product Guide (LP2226) (opens in a new tab), Lenovo Press, last updated 13 January 2026. B200 dense and sparse peaks, 180 GB HBM3e at 7.7 TB/s.
- NVIDIA DGX B200 (opens in a new tab), NVIDIA, page read 25 October 2026. System totals for eight B200 GPUs: 1,440 GB of HBM3e and 64 TB/s, with FP8 shown with sparsity.
- AMD Instinct MI300X Accelerator data sheet (opens in a new tab), Advanced Micro Devices, 2025. MI300X peaks by data type and 5.3 TB/s memory bandwidth.
- TPU v5p (opens in a new tab), Google Cloud documentation, page updated 22 October 2026, read 25 October 2026. 459 TFLOP/s BF16, 95 GiB HBM and 2,765 GB/s per chip.
- Intel Gaudi 3 AI Accelerator White Paper (opens in a new tab), Intel, March 2025 (V1 Rev. 2). Gaudi 3 BF16 and FP8 matrix rates, 128 GB HBM2e at 3.7 TB/s, 96 MB SRAM at 12.8 TB/s.
Where it helps
The lifecycle stages this tool serves: 5 stages in 2 lifecycles.
More ai, systems and startup tools
Method notev1Formula, ranges, sources and limits
Finds the ridge point of an accelerator at a chosen precision and the attainable performance of the default set of kernels from peak compute, memory bandwidth and arithmetic intensity.
Formula
peak = entered peak x unit x sustained compute share; bandwidth = entered bandwidth x unit x sustained bandwidth share; intensity = operations / bytes moved (FLOP per byte); attainable = min(peak, intensity x bandwidth); ridge point = peak / bandwidth (FLOP per byte); attainable = min(peak, intensity HBM x bandwidth HBM, intensity L2 x bandwidth L2); efficiency = measured / attainable; time = operations / attainable
| Input | Unit | Range it holds for | Comes from |
|---|---|---|---|
| Unit of the compute peaks | choice | gflops, tflops, pflops | Your figure |
| Peak FP64 | entered unit | from 0 | Your figure |
| Peak FP32 | entered unit | from 0 | Your figure |
| Peak TF32 | entered unit | from 0 | Your figure |
| Peak BF16 or FP16 | entered unit | from 0 | Your figure |
| Peak FP8 | entered unit | from 0 | Your figure |
| Peak INT8 | entered unit | from 0 | Your figure |
| Unit of the bandwidths | choice | gbs, tbs | Your figure |
| HBM bandwidth | entered unit | from 0 | Your figure |
| Second-level bandwidth | entered unit, 0 for none | from 0 | Your figure |
| Sustained share of peak compute | % | 1 to 100 | Your figure |
| Sustained share of memory bandwidth | % | 1 to 100 | Your figure |
| Precision for the ridge point and the chart | choice | fp64, fp32, tf32, bf16, fp8, int8 | Your figure |
| Unit of kernel operations | choice | mflop, gflop, tflop | Your figure |
| Unit of kernel bytes | choice | kb, mb, gb | Your figure |
Outputs: Ridge point at the chart precision (FLOP per byte); Sustained peak at the chart precision (TFLOP/s); Sustained HBM bandwidth (TB/s); Attainable rate of the first kernel (TFLOP/s); Memory bound kernels (count).
Assumptions
- Peaks are theoretical dense rates (no structured sparsity), taken from vendor data sheets and product pages. NVIDIA prints some peaks with sparsity; the presets here use the dense half.
- Bytes moved are the traffic that reaches HBM. A kernel whose data fits in cache has an intensity against HBM that is higher than its intensity against the cache.
- Memory and compute overlap perfectly, so the kernel runs at the lower of the two limits. Real kernels add latency, launch and synchronisation costs on top.
- The kernel presets use the usual operation counts: 2MNK for a matrix multiply, 4 s^2 d for one attention head, and one operation per value for elementwise work. The sizes and byte counts are illustrative.
Uncertainty: The arithmetic is exact. The result is an upper bound that is as good as the peaks and byte counts entered; measured kernels commonly reach 50 to 90 percent of the roofline.
Known limits
- The model gives an upper bound, not a prediction. A kernel can sit well below its roof because of occupancy, tile shapes, instruction mix or communication.
- Only the limits entered are modelled: network, PCIe or storage limits, and power or clock throttling, need their own ceilings.
- One peak per precision is used. Mixed-precision kernels, and kernels whose operations run on different units (vector and matrix), should be split into rows.
- Vendors count operations in different ways (a fused multiply-add as two operations is the usual choice). Compare chips only on figures counted the same way.
- Second-level bandwidths are often not published; where the data sheet gives none, the preset leaves the level empty.
Where the method comes from
Follows Williams, Waterman and Patterson, Roofline: an insightful visual performance model for multicore architectures (Communications of the ACM, April 2009) (opens in a new tab). The platform's version departs from it as follows.
- Peaks come from vendor data sheets, scaled by a sustained share, instead of microbenchmarks run on the machine Why: Most people sizing an accelerator cannot run microbenchmarks on it first, so a data sheet figure with an adjustable share is the practical start.
- A second memory level can be added as a hierarchical roofline, with its own bytes for each kernel Why: Accelerators with large on-die SRAM or caches have a second bandwidth limit that the single DRAM line does not show.
Reference example, checked by test: Worked example in this method note: the H100 SXM preset (dense BF16 peak 989.5 TFLOP/s, HBM 3.35 TB/s), whose ridge point is 989.5e12 / 3.35e12 = 295.4 FLOP per byte; the first kernel is an 8192 cubed BF16 matrix multiply with an intensity of about 2,731, so it is compute bound at the peak. Ridge point at the chart precision 295.373; Sustained peak at the chart precision 989.5; Sustained HBM bandwidth 3.35; Attainable rate of the first kernel 989.5; Memory bound kernels 4.
Versions
- v1 (2026-10-25): First version, built from the tool idea.
Terms on this page
- Arithmetic intensity
- The ratio of arithmetic operations performed to bytes moved from memory for a given computation.
- Chip
- The everyday word for a finished integrated circuit, used loosely for both the bare die and the packaged part sold to a customer.
- Kernel
- In GPU programming, a function launched across a grid of many threads on the device.
- Roofline model
- A visual performance model that plots attainable throughput against arithmetic intensity, capped by a horizontal peak-compute roof and a sloping memory-bandwidth roof.
- Bandwidth
- In meeting jargon, a person's or team's available working capacity rather than any data rate.
- TOPS
- A throughput figure for AI accelerators, counting trillions of multiply-accumulate-class operations per second, usually quoted at INT8 precision.
- Datasheet
- The authoritative customer-facing document giving a part's electrical characteristics, absolute maximum ratings, timing, pinout, package drawing and ordering information.
- Cache
- A small fast memory holding recently or likely used data close to the processor, normally SRAM arranged in levels L1, L2 and L3.
- SRAM
- A volatile memory that holds each bit in a cross-coupled inverter latch, keeping its value as long as power is applied and needing no refresh.
- Level 2
- Automation in which the system controls both steering and acceleration/braking within its design limits while the driver must continuously supervise and remain responsible for object and event dete...
- HBM
- A JEDEC standard DRAM built as a vertical stack of core dice on a base die, connected by through-silicon vias and presenting a very wide, relatively slow interface.
- Sparsity
- The property of a tensor having many zero entries, which hardware can exploit by skipping the corresponding multiplications and by storing only non-zeros.
Need help acting on this?
Tell us what you're after and leave your email: we'll connect you with the right person on the Hub.