A chip built to run machine learning workloads, developed together with its compiler, runtime and libraries and the system it runs in: from workload and model analysis and performance modelling through architecture, design, tape-out, boards, power, cooling and scale-out networking to bring-up with the software stack, benchmarking, customer workload qualification and software releases that continue long after launch.
AI accelerator lifecycle: 14 stages, 18 gates of its own and 5 hand-offs to the Product lifecycle, from Workload & Model Analysis to Software Releases, Sustaining & End of Support.
14 stages, 18 gates of its own and 5 hand-offs to the Product lifecycle, 3 of the gates technical readiness reviews (TR). Each stop opens its stage; each sign opens its gate. Point at a stop or tab to it to read what the stage is for. What the codes mean
In collaboration with The Digital Electronics Blog (opens in a new tab)
A dashed sign with a TR tag is a technical readiness review gate.
An outlined sign with an arrow is a hand-off: the gate is owned by the Product lifecycle, where this lifecycle hands its work on.
In collaboration with The Digital Electronics Blog (opens in a new tab)
Choosing the machine learning workloads the accelerator must run well, such as training or inference of particular model families, and measuring what they need: operations, data types, memory capacity and bandwidth, communication between chips and the batch sizes and latencies customers expect.
Concept approval gate, owned by the Product lifecycle (Concept approval)Target workload set agreed gate
Building analytical and simulation models that predict how candidate designs run the target workloads, and exploring compute, memory, interconnect and numeric choices together with the compiler team before any design is fixed.
An outlined sign with an arrow is a hand-off: the gate is owned by the Product lifecycle, where this lifecycle hands its work on.
Defining the compute units, on-chip memory, memory hierarchy, network on chip, chip-to-chip links, host interface and the interface software sees, within an agreed power and thermal envelope.
Writing the design code for the compute arrays, memory controllers, network on chip and interfaces, and building its verification in simulation, formal checks and emulation alongside it, against the architecture and the performance model, up to the RTL freeze.
Building the compiler that maps models onto the hardware, the runtime and drivers that run them, and the kernel libraries and framework integration developers use, on simulators and emulation before silicon, so that hardware and software problems are found while the design can still change.
Verification sign-off gatePre-silicon software readiness gate
Implementing a large, power-dense die, often with high-bandwidth memory beside it in an advanced package, and closing timing, power integrity, physical checks and the package design together until they are signed off.
Finishing the signed-off die database, re-checking every late change, and releasing the database and the package design to manufacture.
Designing the accelerator card or module, its power delivery and its cooling, air or liquid, and building the bring-up boards while the silicon is being manufactured.
Designing how many accelerators work as one: the links between chips and between servers, the topology, the collective communication the software uses, and a reference design for racks and clusters.
Powering up first silicon on its boards, bringing up firmware, drivers, runtime and compiler on real hardware, running the first models end to end and recording every difference from the design as an issue or an erratum.
Measuring performance, efficiency and accuracy on the target workloads and on public benchmarks, comparing silicon with the performance model, and tuning compiler, kernels and runtime until the targets are met or the gaps are understood.
Putting early access systems with lead customers, porting their models, and qualifying that their workloads run correctly, at the expected accuracy and performance, on the released software stack and the production system.
Early access to lead customers gate, owned by the Product lifecycle (Engineering samples)Customer workloads qualified gate
Qualifying the chip, package and system for reliability, releasing production and system-level test, publishing the software release and its documentation, and releasing the product for volume orders.
Qualification passed gateTR3, technical readiness reviewProduction software released gateProduction release gate, owned by the Product lifecycle (Production release)
Shipping regular software releases that add model support and performance, fixing field issues, keeping supply and quality in control, and finally ending support with a published notice and a migration path.
Maintenance software release gateEnd of support gate, owned by the Product lifecycle (End of support)
The full lifecycle detail, with its charts, flows, tables and every section, is for registered members. Registration is free and takes a minute. You come back to this page afterwards.