ZipFusion: Fusing Lossless KV Decompression with Attention for Efficient LLM Decoding

The key-value (KV) cache places growing demands on GPU memory capacity and bandwidth as large language models serve longer contexts and more concurrent requests. Its footprint limits concurrency, while repeated KV reads during attention can make decoding memory-bandwidth bound. Prior lossless KV cache compression methods reduce transfers between devices but restore the uncompressed KV cache in GPU memory before attention. They therefore leave these repeated KV reads unchanged and require additi…

Source: arXiv cs.AR (Hardware Architecture), published 2026-10-27. The summary is the Hub's own; the full story is at the source. Report a rights or accuracy concern