Academic search engines: constraints, bugs, and recommendations

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)
141 Downloads (Pure)

Abstract

Academic search engines (i.e., digital libraries and indexers) play an increasingly important role in systematic reviews however these engines do not seem to effectively support such reviews, e.g., researchers confront usability issues with the engines when conducting their searches. To investigate whether the usability issues are bugs (i.e., faults in the search engines) or constraints, and to provide recommendations to search-engine providers and researchers on how to tackle these issues. Using snowball-sampling from tertiary studies, we identify a set of 621 secondary studies in software engineering. By physically re-attempting the searches for all of these 621 studies, we effectively conduct regression testing for 42 search engines. We identify 13 bugs for eight engines, and also identify other constraints. We provide recommendations for tackling these issues. There is still a considerable gap between the search-needs of researchers and the usability of academic search engines. It is not clear whether search-engine developers are aware of this gap. Also, the evaluation, by academics, of academic search engines has not kept pace with the development, by search-engine providers, of those search engines. Thus, the gap between evaluation and development makes it harder to properly understand the gap between the search-needs of researchers and search-features of the search engines.

Original languageEnglish
Title of host publicationA-TEST 2022: Proceedings of the 13th International Workshop on Automating Test Case Design, Selection and Evaluation
PublisherAssociation for Computing Machinery
Pages25–32
Number of pages8
ISBN (Electronic)9781450394529
DOIs
Publication statusPublished - 09 Nov 2022
EventThe 13th International Workshop on Automating TEST Case Design, Selection, and Evaluation, A-TEST@ESEC/SIGSOFT FSE 2022 - Singapore, Singapore, Singapore
Duration: 17 Nov 202218 Nov 2022
https://a-test.org/

Publication series

NameInternational Workshop on Automating Test Case Design, Selection and Evaluation Proceedings

Workshop

WorkshopThe 13th International Workshop on Automating TEST Case Design, Selection, and Evaluation, A-TEST@ESEC/SIGSOFT FSE 2022
Country/TerritorySingapore
CitySingapore
Period17/11/202218/11/2022
Internet address

Fingerprint

Dive into the research topics of 'Academic search engines: constraints, bugs, and recommendations'. Together they form a unique fingerprint.

Cite this