Scale-up

Notes

Rack-scale systems exist precisely to enlarge the scale-up domain from eight accelerators to dozens or more.

Overview

Scale-up, in AI and accelerated computing hardware, means making one tightly coupled compute domain larger by connecting more accelerators over short, very high-bandwidth links, so that they can share data almost as if they were one large device. It is the counterpart of scale-out, which links many such domains over a wider data centre network. How far a system can scale up is set by the reach, bandwidth and switch capacity of its scale-up interconnect.

How it works

Inside a scale-up domain, accelerators talk over a dedicated fabric rather than through the general network. NVIDIA, for example, describes NVLink as a scale-up networking fabric: its fifth generation gives each GPU 1,800 GB/s of NVLink bandwidth, and NVLink switches join 8 or 72 GPUs into one domain. Other companies have formed the UALink Consortium to define an open standard for accelerator-to-accelerator links of the same kind.

Because links inside the domain are fast, the most communication-heavy work is placed there. Splitting the parameters of individual layers across accelerators, known as tensor parallelism, is the usual case: NVIDIA's Megatron Bridge guidance, for example, says it works best within a single high-bandwidth NVLink node and recommends other forms of parallelism between nodes. Work that exchanges data less intensively can run across the slower but far larger scale-out network, which is often Ethernet or InfiniBand; the Ultra Ethernet Consortium, for instance, targets that scale-out role.

Scale-up has limits. The domain can only be as large as its links can reach at full speed, and each added accelerator needs more switch ports, power and cooling, so system makers look to rack-scale designs and new link technologies to enlarge it.

Examples

• A rack-scale system in which NVLink switches connect 72 GPUs into one NVLink domain.
• A large model whose individual layers are split across the GPUs of one scale-up domain using tensor parallelism.
• Accelerator makers and cloud companies working in the UALink Consortium on an open accelerator-to-accelerator interconnect.
• A training cluster that uses NVLink inside each rack for scale-up and an Ethernet fabric between racks for scale-out.

Why it matters

AI systems architects, interconnect designers, cluster infrastructure engineers and data centre planners care about scale-up because the size of the domain decides which models can be split efficiently and how much time accelerators spend waiting for data. It also shapes rack power, cooling and cabling.

In the chip lifecycle, scale-up capability is fixed during accelerator and switch architecture, when the number of links and their speed are chosen, then verified in interconnect and system bring-up, and finally exercised in production clusters, where software maps work onto the domain.

It matters most to: AI Cluster Infrastructure Engineer, AI Systems Performance Architect, Distributed Training Engineer, GPU Architect, Interconnect / NoC Architect

Common confusions

Not the same as scale-out

Scale-out connects many separate nodes or domains over a data centre network, trading bandwidth and latency for size. Scale-up enlarges one domain with much faster, shorter links. Scale-up vs scale-out

Not the same as scaling up in general IT

In general computing, scaling up often means moving to a bigger single server, as opposed to adding more servers. In AI hardware the term refers specifically to the accelerator domain joined by a dedicated fabric.

Not the same as scale-up in manufacturing

In process and materials work, scale-up means moving a process from the lab to volume production. It has nothing to do with interconnects.

Key figures

• 1,800 GB/s per GPU NVLink bandwidth that NVIDIA quotes for its fifth-generation NVLink, which joins 8 or 72 GPUs into one scale-up domain. (Source: NVLink and NVLink Switch, 2026-10-22)

Sources

1. NVLink and NVLink Switch (opens in a new tab), NVIDIA, 2026-10-22
2. Megatron Bridge: Parallelisms Guide (opens in a new tab), NVIDIA, 2026-10-22
3. UALink Consortium (opens in a new tab), UALink Consortium, 2026-10-22
4. Ultra Ethernet Consortium (opens in a new tab), Ultra Ethernet Consortium (Linux Foundation), 2026-10-22

Related terms

• Scale-out
• NVLink
• Rack scale
• Tensor parallelism

Where it appears on the Hub

Used on 9 pages of the Hub. In each group, the pages that mention it most come first.