Abstract
Background: Software engineering (SE) research continues to study the degree to which practitioners perceive research as relevant to practice. Such studies typically comprise surveys of practitioner opinions. In a preliminary, and relatively small scale, study of online articles we previously found few explicit citations to software testing research. Our previous study provided an in situ complement to the typical survey study, however the findings of the previous study were limited by the size of our sample. Objective: To further investigate whether and how practitioners cite software testing research in the grey literature, by using a larger and more diverse dataset. Method: We analyse four distinct datasets totalling over 400,000 online articles with approx. 2M external citations. Two datasets were generated by crawling predefined domains and two were generated by applying heuristics, developed in prior research, in Google searches. Citations are classified and then analysed. Results: We find a (very) low percentage of citations to research. Conclusion: Our replication corroborates our preliminary study and findings from others. In relative terms, topic–specific searches appear to return results that contain articles with more citations. Our results and method provide a basis for benchmarking.
Original language | English |
---|---|
Title of host publication | Proceedings of 23rd International Conference on Evaluation and Assessment in Software Engineering (EASE'19) |
Publisher | Association for Computing Machinery |
Pages | 292-297 |
Number of pages | 6 |
ISBN (Electronic) | 9781450371452 |
DOIs | |
Publication status | Published - 15 Apr 2019 |
Externally published | Yes |
Event | 23rd Evaluation and Assessment in Software Engineering Conference, EASE 2019 - Copenhagen, Denmark Duration: 14 Apr 2019 → 17 Apr 2019 |
Conference
Conference | 23rd Evaluation and Assessment in Software Engineering Conference, EASE 2019 |
---|---|
Country | Denmark |
City | Copenhagen |
Period | 14/04/2019 → 17/04/2019 |
Keywords
- Evidence
- Grey literature
- Research impact
- Research relevance
ASJC Scopus subject areas
- Human-Computer Interaction
- Software