UMLoad: Fast On-Demand Model Loading on Unified Memory Architecture
By The Semiconductor Hub news desk ·
Link post
On-demand model loading dominates response latency for interactive on-device AI, yet unified memory architectures (UMAs), now common on edge devices, should make it cheap by placing weights in memory the GPU already sees. However, frameworks still use the loading flow built for discrete GPUs, leaving significant inefficiencies. This paper presents UMLoad, a system-level model-loading accelerator that decouples a framework's logical allocations from their physical page backing. UMLoad uses on-de…
Read at arXiv cs.AR (Hardware Architecture)Source: arXiv cs.AR (Hardware Architecture), published . The summary is the Hub's own; the full story is at the source. Report a rights or accuracy concern
Comments (0)
Sign in to comment.