1. Purpose and scope
Large training jobs and large model inference spread work across many accelerators, so the speed of the links between them and the efficiency of collective operations often limit delivered performance as much as the chip itself. This stage designs how accelerators connect within a server and across racks, how collective communication uses that topology, and what customers must build.
The output is a cluster reference design that customers and partners can follow, and a collective communication plan the software teams implement. Both are proven on hardware during bring-up and benchmarking.
Several kinds of link are involved. Within a server, accelerators may be joined by dedicated high-bandwidth links or through the host interface; across servers, by a switched network, often based on Ethernet or another data centre fabric. Each choice affects bandwidth, latency, cost, power and how much the customer must build, so the stage compares them against the communication patterns of the target workloads rather than in general.
The stage is also where fault tolerance at scale is considered. Large clusters see link and node failures as a routine event, so the design states how jobs survive them: spare links, checkpointing support in the software stack, and telemetry that finds failing parts before they stop a job.
12 more sections for registered members.