MAC (Multiply-Accumulate)

Notes

One MAC is conventionally counted as two operations, which is why a MAC array's TOPS figure is twice its MAC rate.

Overview

A MAC, short for multiply-accumulate, is the arithmetic step that multiplies two numbers and adds the product to a running total held in an accumulator. Matrix multiplications, convolutions and digital filters are built almost entirely from long chains of MACs, so the number of MAC units in a chip, and how often each can fire, largely sets its peak throughput for AI and signal processing. In networking, the same three letters mean media access control, which is unrelated.

How it works

A MAC unit contains a multiplier feeding an adder whose output loops back into a register. Each cycle it takes a new pair of operands, such as a weight and an activation, multiplies them and adds the result to the accumulator. After the last pair in a dot product, the accumulator holds the answer. A matrix multiply of an M by K matrix with a K by N matrix needs M times N times K of these steps, and performance tools usually count each one as two operations, a multiply and an add.

Accelerators replicate the unit by the thousand. Google's TPU matrix unit, for example, arranges multiply-accumulators in a systolic array, 128 by 128 in older generations and 256 by 256 in newer ones, so data flows from cell to cell and each cell performs one MAC per cycle. GPUs group MAC hardware into tensor or matrix cores, and digital signal processors were the first processors to include dedicated MAC units, for filtering.

Precision is a key design choice. The inputs are often narrow formats such as bfloat16, FP16 or INT8 to save area and energy, while the accumulator is kept wider, for instance FP32, so that rounding errors do not build up over long sums. Peak figures assume every MAC unit is busy every cycle; in practice memory bandwidth and data movement usually limit how many are kept fed.

Examples

• A TPU matrix unit of 128 by 128 multiply-accumulators performing 16K MAC operations per cycle.
• GPU tensor core operations that take FP16 inputs and accumulate in FP32.
• A DSP running a finite impulse response filter, which needs one MAC per filter tap for each output sample.

Why it matters

MAC count, clock frequency and data width together give the headline TOPS or TFLOPS figures quoted for accelerators, so architects, datapath designers and performance modellers all reason in MACs. Each MAC also costs area and energy, so the format of the multiplier and the width of the accumulator are traded against accuracy and power.

Related terms

• Matrix multiply unit
• TOPS
• DSP
• Systolic array

Where it appears on the Hub

Used on 8 pages of the Hub. In each group, the pages that mention it most come first.