MapReduce job optimization: a mapping study

Qinghua Lu; Liming Zhu; He Zhang; Dongyao Wu; Zheng Li; Xiwei Xu

doi:10.1109/CCBD.2015.33

MapReduce job optimization: a mapping study

Qinghua Lu
, Liming Zhu
, He Zhang
, Dongyao Wu
, Zheng Li
, Xiwei Xu

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Citations (Scopus)

Abstract

MapReduce has become the standard model for supporting big data analytics. In particular, MapReduce job optimization has been widely considered to be crucial in the implementations of big data analytics. However, there is still a lack of guidelines especially for practitioners to understand how the MapReduce jobs can be optimized. This paper aims to systematic identify and taxonomically classify the existing work on job optimization. We conducted a mapping study on 47 selected papers that were published between 2004 and 2014. We classified and compared the selected papers based on a 5WH-based characterization framework. This study generates a knowledge base of current job optimization solutions and also identifies a set of research gaps and opportunities. This study concludes that job optimization is still in an early stage of maturity. More attentions need to be paid to the cross-data center, cluster or rack job optimization to improve communication efficiency.

Original language	English
Title of host publication	Proceedings of the 2015 International Conference on Cloud Computing and Big Data (CCBD)
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	81-88
Number of pages	8
ISBN (Electronic)	9781467383509
DOIs	https://doi.org/10.1109/CCBD.2015.33
Publication status	Published - 08 Apr 2016
Externally published	Yes
Event	International Conference on Cloud Computing and Big Data, CCBD 2015 - Shanghai, China Duration: 04 Nov 2015 → 06 Nov 2015

Publication series

Name	Proceedings of the International Conference on Cloud Computing and Big Data (CCBD)

Conference

Conference	International Conference on Cloud Computing and Big Data, CCBD 2015
Country/Territory	China
City	Shanghai
Period	04/11/2015 → 06/11/2015

Bibliographical note

Funding Information:
This project is supported by National Natural Science Foundation of China (Grant No. 61402533).

Publisher Copyright:
© 2015 IEEE.

Keywords

big data
job optimization
mapping study
MapReduce
systematic literature review

ASJC Scopus subject areas

Computer Networks and Communications
Information Systems

Access to Document

10.1109/CCBD.2015.33Licence: Unspecified

Cite this

Lu, Q., Zhu, L., Zhang, H., Wu, D., Li, Z., & Xu, X. (2016). MapReduce job optimization: a mapping study. In Proceedings of the 2015 International Conference on Cloud Computing and Big Data (CCBD) (pp. 81-88). Article 7450534 (Proceedings of the International Conference on Cloud Computing and Big Data (CCBD)). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CCBD.2015.33

@inproceedings{8ee3d7596a2a4eb184d9cdc05d5f636e,

title = "MapReduce job optimization: a mapping study",

abstract = "MapReduce has become the standard model for supporting big data analytics. In particular, MapReduce job optimization has been widely considered to be crucial in the implementations of big data analytics. However, there is still a lack of guidelines especially for practitioners to understand how the MapReduce jobs can be optimized. This paper aims to systematic identify and taxonomically classify the existing work on job optimization. We conducted a mapping study on 47 selected papers that were published between 2004 and 2014. We classified and compared the selected papers based on a 5WH-based characterization framework. This study generates a knowledge base of current job optimization solutions and also identifies a set of research gaps and opportunities. This study concludes that job optimization is still in an early stage of maturity. More attentions need to be paid to the cross-data center, cluster or rack job optimization to improve communication efficiency.",

keywords = "big data, job optimization, mapping study, MapReduce, systematic literature review",

author = "Qinghua Lu and Liming Zhu and He Zhang and Dongyao Wu and Zheng Li and Xiwei Xu",

note = "Funding Information: This project is supported by National Natural Science Foundation of China (Grant No. 61402533). Publisher Copyright: {\textcopyright} 2015 IEEE.; International Conference on Cloud Computing and Big Data, CCBD 2015 ; Conference date: 04-11-2015 Through 06-11-2015",

year = "2016",

month = apr,

day = "8",

doi = "10.1109/CCBD.2015.33",

language = "English",

series = "Proceedings of the International Conference on Cloud Computing and Big Data (CCBD)",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "81--88",

booktitle = "Proceedings of the 2015 International Conference on Cloud Computing and Big Data (CCBD)",

address = "United States",

}

Lu, Q, Zhu, L, Zhang, H, Wu, D, Li, Z & Xu, X 2016, MapReduce job optimization: a mapping study. in Proceedings of the 2015 International Conference on Cloud Computing and Big Data (CCBD)., 7450534, Proceedings of the International Conference on Cloud Computing and Big Data (CCBD), Institute of Electrical and Electronics Engineers Inc., pp. 81-88, International Conference on Cloud Computing and Big Data, CCBD 2015, Shanghai, China, 04/11/2015. https://doi.org/10.1109/CCBD.2015.33

MapReduce job optimization: a mapping study. / Lu, Qinghua; Zhu, Liming; Zhang, He et al.
Proceedings of the 2015 International Conference on Cloud Computing and Big Data (CCBD). Institute of Electrical and Electronics Engineers Inc., 2016. p. 81-88 7450534 (Proceedings of the International Conference on Cloud Computing and Big Data (CCBD)).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - MapReduce job optimization: a mapping study

AU - Lu, Qinghua

AU - Zhu, Liming

AU - Zhang, He

AU - Wu, Dongyao

AU - Li, Zheng

AU - Xu, Xiwei

PY - 2016/4/8

Y1 - 2016/4/8

N2 - MapReduce has become the standard model for supporting big data analytics. In particular, MapReduce job optimization has been widely considered to be crucial in the implementations of big data analytics. However, there is still a lack of guidelines especially for practitioners to understand how the MapReduce jobs can be optimized. This paper aims to systematic identify and taxonomically classify the existing work on job optimization. We conducted a mapping study on 47 selected papers that were published between 2004 and 2014. We classified and compared the selected papers based on a 5WH-based characterization framework. This study generates a knowledge base of current job optimization solutions and also identifies a set of research gaps and opportunities. This study concludes that job optimization is still in an early stage of maturity. More attentions need to be paid to the cross-data center, cluster or rack job optimization to improve communication efficiency.

AB - MapReduce has become the standard model for supporting big data analytics. In particular, MapReduce job optimization has been widely considered to be crucial in the implementations of big data analytics. However, there is still a lack of guidelines especially for practitioners to understand how the MapReduce jobs can be optimized. This paper aims to systematic identify and taxonomically classify the existing work on job optimization. We conducted a mapping study on 47 selected papers that were published between 2004 and 2014. We classified and compared the selected papers based on a 5WH-based characterization framework. This study generates a knowledge base of current job optimization solutions and also identifies a set of research gaps and opportunities. This study concludes that job optimization is still in an early stage of maturity. More attentions need to be paid to the cross-data center, cluster or rack job optimization to improve communication efficiency.

KW - big data

KW - job optimization

KW - mapping study

KW - MapReduce

KW - systematic literature review

U2 - 10.1109/CCBD.2015.33

DO - 10.1109/CCBD.2015.33

M3 - Conference contribution

AN - SCOPUS:84969506247

T3 - Proceedings of the International Conference on Cloud Computing and Big Data (CCBD)

SP - 81

EP - 88

BT - Proceedings of the 2015 International Conference on Cloud Computing and Big Data (CCBD)

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - International Conference on Cloud Computing and Big Data, CCBD 2015

Y2 - 4 November 2015 through 6 November 2015

ER -

MapReduce job optimization: a mapping study

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Fingerprint

Cite this