A complete and efficient CUDA-sharing solution for HPC clusters

Antonio Peña, Carlos Reaño, Federico Silla, Rafael Mayo, Enrique Quintana Orti, J. Duato

Research output: Contribution to journalArticlepeer-review

64 Citations (Scopus)
205 Downloads (Pure)


In this paper we detail the key features, architectural design, and implementation of rCUDA, an advanced framework to enable remote and transparent GPGPU acceleration in HPC clusters. rCUDA allows decoupling GPUs from nodes, forming pools of shared accelerators, which brings enhanced flexibility to cluster configurations. This opens the door to configurations with fewer accelerators than nodes, as well as permits a single node to exploit the whole set of GPUs installed in the cluster. In our proposal, CUDA applications can seamlessly interact with any GPU in the cluster, independently of its physical location. Thus, GPUs can be either distributed among compute nodes or concentrated in dedicated GPGPU servers, depending on the cluster administrator’s policy. This proposal leads to savings not only in space but also in energy, acquisition, and maintenance costs. The performance evaluation in this paper with a series of benchmarks and a production application clearly demonstrates the viability of this proposal. Concretely, experiments with the matrix–matrix product reveal excellent performance compared with regular executions on the local GPU; on a much more complex application, the GPU-accelerated LAMMPS, we attain up to 11x speedup employing 8 remote accelerators from a single node with respect to a 12-core CPU-only execution. GPGPU service interaction in compute nodes, remote acceleration in dedicated GPGPU servers, and data transfer performance of similar GPU virtualization frameworks are also evaluated.
Original languageEnglish
Pages (from-to)574-588
JournalParallel Computing
Issue number10
Early online date08 Oct 2014
Publication statusPublished - 01 Dec 2014


Dive into the research topics of 'A complete and efficient CUDA-sharing solution for HPC clusters'. Together they form a unique fingerprint.

Cite this