Do software engineering practitioners cite software testing research in their online articles? A larger scale replication

Ashley Williams, Austen Rainer

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Background: Software engineering (SE) research continues to study the degree to which practitioners perceive research as relevant to practice. Such studies typically comprise surveys of practitioner opinions. In a preliminary, and relatively small scale, study of online articles we previously found few explicit citations to software testing research. Our previous study provided an in situ complement to the typical survey study, however the findings of the previous study were limited by the size of our sample. Objective: To further investigate whether and how practitioners cite software testing research in the grey literature, by using a larger and more diverse dataset. Method: We analyse four distinct datasets totalling over 400,000 online articles with approx. 2M external citations. Two datasets were generated by crawling predefined domains and two were generated by applying heuristics, developed in prior research, in Google searches. Citations are classified and then analysed. Results: We find a (very) low percentage of citations to research. Conclusion: Our replication corroborates our preliminary study and findings from others. In relative terms, topic–specific searches appear to return results that contain articles with more citations. Our results and method provide a basis for benchmarking.

Original languageEnglish
Title of host publicationProceedings of 23rd International Conference on Evaluation and Assessment in Software Engineering (EASE'19)
PublisherAssociation for Computing Machinery
Pages292-297
Number of pages6
ISBN (Electronic)9781450371452
DOIs
Publication statusPublished - 15 Apr 2019
Externally publishedYes
Event23rd Evaluation and Assessment in Software Engineering Conference, EASE 2019 - Copenhagen, Denmark
Duration: 14 Apr 201917 Apr 2019

Conference

Conference23rd Evaluation and Assessment in Software Engineering Conference, EASE 2019
CountryDenmark
CityCopenhagen
Period14/04/201917/04/2019

Keywords

  • Evidence
  • Grey literature
  • Research impact
  • Research relevance

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Software

Fingerprint Dive into the research topics of 'Do software engineering practitioners cite software testing research in their online articles? A larger scale replication'. Together they form a unique fingerprint.

  • Cite this

    Williams, A., & Rainer, A. (2019). Do software engineering practitioners cite software testing research in their online articles? A larger scale replication. In Proceedings of 23rd International Conference on Evaluation and Assessment in Software Engineering (EASE'19) (pp. 292-297). Association for Computing Machinery. https://doi.org/10.1145/3319008.3319708