Using Citation Behavior to Rethink Academic Impact in Software Engineering

Simon Poulding, Kai Petersen, Robert Feldt, Vahid Garousi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

Although citation counts are often considered a measure of academic impact, they are criticized for failing to evaluate impact as intended. In this paper we propose that software engineering citations may be classified according to how the citation is used by the author of the citing paper, and that through this classification of citation behaviour it is possible to achieve a more refined understanding of the cited paper's impact. Our objective in this work is to conduct an initial evaluation using the citation behaviour taxonomy proposed by Bornmann and Daniel. We independently classified citations to ten highly-cited papers published at the International Symposium on Empirical Software Engineering and Measurement (ESEM). The degree to which classifications were consistent between researchers was analyzed in order to assess the clarity of Bornmann and Daniel's taxonomy. We found poor to fair agreement between researchers even though the taxonomy was perceived as relatively easy to apply for the majority of citations. We were nevertheless able to identify clear differences in the profile of citation behaviors between the cited papers. We conclude that an improved taxonomy is required if classification is to be reliable, and that a degree of automation would improve reliability as well as reduce the time taken to make a classification.

Original languageEnglish
Title of host publication2015 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2015 - Proceedings
PublisherIEEE Computer Society
Pages140-143
Volume2015-November
ISBN (Electronic)9781467378994
DOIs
Publication statusPublished - 09 Nov 2015
Externally publishedYes
EventACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2015 - Beijing, China
Duration: 22 Oct 201523 Oct 2015

Conference

ConferenceACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2015
CountryChina
CityBeijing
Period22/10/201523/10/2015

Keywords

  • Bibliometrics
  • Context
  • Data collection
  • Reliability
  • Software engineering
  • Software measurement
  • Taxonomy

ASJC Scopus subject areas

  • Computer Science Applications
  • Software

Fingerprint Dive into the research topics of 'Using Citation Behavior to Rethink Academic Impact in Software Engineering'. Together they form a unique fingerprint.

  • Cite this

    Poulding, S., Petersen, K., Feldt, R., & Garousi, V. (2015). Using Citation Behavior to Rethink Academic Impact in Software Engineering. In 2015 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2015 - Proceedings (Vol. 2015-November, pp. 140-143). [7321216] IEEE Computer Society. https://doi.org/10.1109/ESEM.2015.7321216