Cluster network sizer

How it works

An AI training cluster needs a network that can move gradients and activations between thousands of accelerators without becoming the bottleneck. This tool sizes a fat-tree (Clos) fabric from the number of accelerators, the ports and speed of each, and the switch radix: how many leaf, spine and core switches, how many links and optical transceivers, what the network draws, how much bandwidth crosses its middle, and how long an all-reduce takes.

1. Count the endpoints and the usable radix. Every accelerator port that joins the network is an endpoint. A switch port that is faster than the link can be split (breakout), for example one 800 Gb/s port into four 200 Gb/s links; a slower one must be joined in groups. The result is the radix: how many links one switch can take at the link speed.endpoints = accelerators x ports per accelerator; radix = switch ports x floor(switch speed / link speed)
2. Split each leaf between servers and uplinks. A leaf switch gives some of its ports to accelerators (down) and the rest to the next tier (up). With no oversubscription the two are equal. An oversubscription ratio of r to 1 means r times more downlink than uplink capacity. In a rail-optimised design the leaf switches of one rail serve only that rail's port in every server, so each rail is counted as a group.up = floor(radix / (1 + r)); down = radix − up; leaf switches = ceiling(endpoints in the group / down), summed over the groups
3. Count the spine tier. The uplinks of all leaf switches land on spine switches. In a two-tier fabric the spine uses every port for leaf links, and every spine must reach every leaf, so there can be no more leaf switches than the radix. This is the leaf-spine arrangement of a fat tree, as Al-Fares, Loukissas and Vahdat describe for commodity switches.leaf-spine links = ceiling(endpoints x up / down); two-tier spine switches = ceiling(leaf-spine links / radix)
4. Add a core tier for three tiers. In a three-tier fabric the spine also splits its ports between leaf links (down) and core links (up) at its own oversubscription ratio, and the core joins the pods. A non-blocking fabric of switches with k ports reaches k^3 / 4 endpoints.spine switches = ceiling(leaf-spine links / spine down); spine-core links = ceiling(leaf-spine links x spine up / spine down); core switches = ceiling(spine-core links / radix)
5. Count cables, transceivers and power. Each link is one cable. A link on pluggable optics needs two transceivers, one at each end; an active optical cable has its optics built into both ends; passive copper has none. Network power is the switches plus the optics, with a typical figure per module chosen from the line speed unless you enter your own.transceivers = 2 x optical links; power = switches x switch power + transceivers x module power + AOC ends x end power
6. Find the bandwidth across the fabric. Each endpoint injects at the link speed. The bandwidth across the middle of a full fat tree is half the endpoint total; oversubscription at each tier divides it, and the same ratio limits what one accelerator can sustain when its traffic crosses the fabric.bisection bandwidth = endpoints x link speed / 2 / (leaf ratio x spine ratio); sustained per accelerator = ports x link speed / (leaf ratio x spine ratio)
7. Time the all-reduce with the alpha-beta model. A message of m bytes takes alpha seconds to start and m / beta seconds to send, where beta is the bandwidth of one accelerator across the network times a transport efficiency. A ring all-reduce over p accelerators runs a reduce-scatter and an all-gather of p − 1 steps each. Halving and doubling (Rabenseifner) takes about lg p steps, and recursive doubling suits small messages. The hierarchical ring reduces inside each server first, runs the ring across servers on each accelerator's share, then gathers inside the server. Alpha is the interface latency plus the switch traversals of the longest path.ring: T = 2 (p − 1) alpha + 2 (p − 1) / p x m / beta; halving and doubling: T = 2 lg(p) alpha + 2 (p − 1) / p x m / beta; recursive doubling: T = lg(p) alpha + lg(p) x m / beta

Sources

The methods and default figures rest on these public sources. The wording here is our own; read the source for its full text.

• A scalable, commodity data center network architecture (opens in a new tab), ACM SIGCOMM 2008 (Al-Fares, Loukissas, Vahdat; University of California San Diego), August 2008. The fat tree built from identical commodity switches, and the k^3 / 4 capacity. The link is a copy hosted by Princeton University.
• Optimization of collective communication operations in MPICH (opens in a new tab), The International Journal of High Performance Computing Applications, volume 19, issue 1, pages 49 to 66 (Thakur, Rabenseifner, Gropp), Spring 2005. The alpha-beta cost of the ring and recursive-doubling algorithms and of Rabenseifner's reduce-scatter and gather. The all-reduce forms here add the reduce-scatter and all-gather costs in the same way.
• NVIDIA DGX SuperPOD: Next Generation Scalable Infrastructure for AI Leadership Reference Architecture, featuring NVIDIA DGX H100 (opens in a new tab), NVIDIA, last updated 19 November 2025. Switch and cable counts for 4 to 56 scalable units: for example 32 leaf and 16 spine switches for 1,024 GPUs, and 256 leaf, 256 spine and 128 core for 8,192.
• QM97XX 1U NDR 400Gbps InfiniBand Switch Systems User Manual: Specifications (opens in a new tab), NVIDIA, last updated 25 June 2025. A 32-cage, 64-port 400 Gb/s switch with 747 W typical power with passive cables.
• Cisco 400G QSFP Transceiver Modules Data Sheet (opens in a new tab), Cisco, updated 15 September 2026. Module power: 9 W maximum for the 400G DR4.
• Cisco OSFP 800G Transceiver Modules Data Sheet (opens in a new tab), Cisco, updated 5 January 2026. Module power: 12.9 to 14.2 W typical and 14.5 to 17 W maximum across the 800G modules listed.