Projects per year
This paper introduces hybrid address spaces as a fundamental design methodology for implementing scalable runtime systems on many-core architectures without hardware support for cache coherence. We use hybrid address spaces for an implementation of MapReduce, a programming model for large-scale data processing, and the implementation of a remote memory access (RMA) model. Both implementations are available on the Intel SCC and are portable to similar architectures. We present the design and implementation of HyMR, a MapReduce runtime system whereby different stages and the synchronization operations between them alternate between a distributed memory address space and a shared memory address space, to improve performance and scalability. We compare HyMR to a reference implementation and we find that HyMR improves performance by a factor of 1.71× over a set of representative MapReduce benchmarks. We also compare HyMR with Phoenix++, a state-of-art implementation for systems with hardware-managed cache coherence in terms of scalability and sustained to peak data processing bandwidth, where HyMR demon- strates improvements of a factor of 3.1× and 3.2× respectively. We further evaluate our hybrid remote memory access (HyRMA) programming model and assess its performance to be superior of that of message passing.
|Number of pages||18|
|Journal||Journal of Systems and Software|
|Early online date||05 Jul 2014|
|Publication status||Published - Nov 2014|
FingerprintDive into the research topics of 'Hybrid address spaces: A methodology for implementing scalable high-level programming models on non-coherent many-core architectures'. Together they form a unique fingerprint.
- 1 Active
R6410CSC: NanoStreams: A Hardware and Software Stack for Real-Time Analytics on Fast Data Streams
Nikolopoulos, D., Spence, I. & Woods, R.
01/08/2013 → …