Pipelined compression in remote GPU virtualization systems using rCUDA: early experiences

The amount of Internet of Things (IoT) devices has been increasing in the last years. These are usually low-performance devices with slow network connections. A common improvement is therefore to perform some computations at the edge of the network (e.g. preprocessing data), thereby reducing the amount of data sent through the network. To enhance the computing capabilities of edge devices, remote virtual Graphics Processing Units (GPUs) can be used. Thus, edge devices can leverage GPUs installed in remote computers. However, this solution requires exchanging data with the remote GPU across the network, which as mentioned is typically slow. In this paper we present a novel approach to improve communication performance of edge devices using rCUDA remote GPU virtualization framework. We implement within this framework on-The-fly pipelined data compression, which is done transparently to applications. We use four popular machine learning samples to carry out an initial performance exploration. The analysis is done using a slow 10 Mbps network to emulate the conditions of these devices. Early results show potential improvements provided some current issues are addressed.