Hybrid address spaces: A methodology for implementing scalable high-level programming models on non-coherent many-core architectures

Anastasios Papagiannis, Dimitrios Nikolopoulos

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)
280 Downloads (Pure)

Abstract

This paper introduces hybrid address spaces as a fundamental design methodology for implementing scalable runtime systems on many-core architectures without hardware support for cache coherence. We use hybrid address spaces for an implementation of MapReduce, a programming model for large-scale data processing, and the implementation of a remote memory access (RMA) model. Both implementations are available on the Intel SCC and are portable to similar architectures. We present the design and implementation of HyMR, a MapReduce runtime system whereby different stages and the synchronization operations between them alternate between a distributed memory address space and a shared memory address space, to improve performance and scalability. We compare HyMR to a reference implementation and we find that HyMR improves performance by a factor of 1.71× over a set of representative MapReduce benchmarks. We also compare HyMR with Phoenix++, a state-of-art implementation for systems with hardware-managed cache coherence in terms of scalability and sustained to peak data processing bandwidth, where HyMR demon- strates improvements of a factor of 3.1× and 3.2× respectively. We further evaluate our hybrid remote memory access (HyRMA) programming model and assess its performance to be superior of that of message passing.
Original languageEnglish
Pages (from-to)47-64
Number of pages18
JournalJournal of Systems and Software
Volume97
Early online date05 Jul 2014
DOIs
Publication statusPublished - Nov 2014

Fingerprint Dive into the research topics of 'Hybrid address spaces: A methodology for implementing scalable high-level programming models on non-coherent many-core architectures'. Together they form a unique fingerprint.

Cite this