The need for multivocal literature reviews in software engineering: Complementing systematic literature reviews with grey literature

Vahid Garousi, Michael Felderer, Mika V. Mäntylä

Research output: Chapter in Book/Report/Conference proceedingConference contribution

49 Citations (Scopus)
61 Downloads (Pure)

Abstract

Systematic Literature Reviews (SLR) may not provide insight into the "state of the practice" in SE, as they do not typically include the "grey" (non-published) literature. A Multivocal Literature Review (MLR) is a form of a SLR which includes grey literature in addition to the published (formal) literature. Only a few MLRs have been published in SE so far. We aim at raising the awareness for MLRs in SE by addressing two research questions (RQs): (1) What types of knowledge are missed when a SLR does not include the multivocal literature in a SE field? and (2) What do we, as a community, gain when we include the multivocal literature and conduct MLRs? To answer these RQs, we sample a few example SLRs and MLRs and identify the missing and the gained knowledge due to excluding or including the grey literature. We find that (1) grey literature can give substantial benefits in certain areas of SE, and that (2) the inclusion of grey literature brings forward certain challenges as evidence in them is often experience and opinion based. Given these conflicting viewpoints, the authors are planning to prepare systematic guidelines for performing MLRs in SE.

Original languageEnglish
Title of host publicationProceedings of the 20th International Conference on Evaluation and Assessment in Software Engineering, EASE 2016
PublisherAssociation for Computing Machinery
Volume01-03-June-2016
ISBN (Electronic)9781450336918
DOIs
Publication statusPublished - 01 Jun 2016
Externally publishedYes
Event20th International Conference on Evaluation and Assessment in Software Engineering, EASE 2016 - Limerick, Ireland
Duration: 01 Jun 201603 Jun 2016

Conference

Conference20th International Conference on Evaluation and Assessment in Software Engineering, EASE 2016
CountryIreland
CityLimerick
Period01/06/201603/06/2016

Keywords

  • Empirical software engineering
  • Grey literature
  • MLR
  • Multivocal Literature Reviews
  • Research methodology
  • SLR
  • Systematic literature reviews

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction
  • Computer Vision and Pattern Recognition
  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'The need for multivocal literature reviews in software engineering: Complementing systematic literature reviews with grey literature'. Together they form a unique fingerprint.

  • Cite this

    Garousi, V., Felderer, M., & Mäntylä, M. V. (2016). The need for multivocal literature reviews in software engineering: Complementing systematic literature reviews with grey literature. In Proceedings of the 20th International Conference on Evaluation and Assessment in Software Engineering, EASE 2016 (Vol. 01-03-June-2016). [a26] Association for Computing Machinery. https://doi.org/10.1145/2915970.2916008