SemCache: Semantics-Aware Caching for Efficient GPU Offloading

In this book, we introduce SemCache, a semantics-aware GPU cache that automatically manages CPU-GPU communication in addition to optimizing communication by eliminating redundant transfers using caching. It uses library semantics to determine the appropriate caching granularity for a given offloaded library. Our caching technique is efficient; it only tracks matrices instead of tracking every memory access at fine granularity. We applied SemCache to Basic Linear Algebra Subprograms (BLAS) library to provide a GPU drop-in replacement library. SemCache++ extends SemCache to support offloading to multiple GPUs. SemCache++ is used to build the first multi-GPU drop-in replacement library that (a) uses the virtual memory to automatically manage and optimize multi-GPU communication and (b) requires no program rewriting or annotations. SemCache++ also enables new features like asynchronous transfers, parallel execution and overlapping communication with computation. Experimental results show that our system can dramatically reduce redundant communication for real-world computational science application and deliver significant improvements, beating exciting GPU-based implementations.

Autorentext

Dr. Nabeel Al-Saber currently works with Qualcomm as a senior software Engineer. He received a B.S. in Computer Engineering from Jordan University in 2006, and an M.S. from the New Jersey Institute of Technology and a Ph.D. in Computer Engineering from the Purdue University in 2015. He has multiple publications in different research areas.