No one tool to rule them all: prokaryotic gene prediction tool annotations are highly dependent on the organism of study

Nicholas J Dimonaco, Wayne Aubrey, Kim Kenobi, Amanda Clare, Christopher J Creevey

Research output: Contribution to journalArticlepeer-review

111 Downloads (Pure)


Abstract Motivation The biases in CoDing Sequence (CDS) prediction tools, which have been based on historic genomic annotations from model organisms, impact our understanding of novel genomes and metagenomes. This hinders the discovery of new genomic information as it results in predictions being biased towards existing knowledge. To date, users have lacked a systematic and replicable approach to identify the strengths and weaknesses of any CDS prediction tool and allow them to choose the right tool for their analysis. Results We present an evaluation framework (ORForise) based on a comprehensive set of 12 primary and 60 secondary metrics that facilitate the assessment of the performance of CDS prediction tools. This makes it possible to identify which performs better for specific use-cases. We use this to assess 15 ab initio- and model-based tools representing those most widely used (historically and currently) to generate the knowledge in genomic databases. We find that the performance of any tool is dependent on the genome being analysed, and no individual tool ranked as the most accurate across all genomes or metrics analysed. Even the top-ranked tools produced conflicting gene collections, which could not be resolved by aggregation. The ORForise evaluation framework provides users with a replicable, data-led approach to make informed tool choices for novel genome annotations and for refining historical annotations. Availability and implementation Code and datasets for reproduction and customisation are available at Supplementary information Supplementary data are available at Bioinformatics online.
Original languageEnglish
Pages (from-to)1198-1207
Number of pages10
Issue number5
Early online date07 Dec 2021
Publication statusPublished - 07 Feb 2022

Bibliographical note

© The Author(s) 2021. Published by Oxford University Press.


  • Biochemistry
  • Computational Mathematics
  • Computational Theory and Mathematics
  • Computer Science Applications
  • Molecular Biology
  • Statistics and Probability


Dive into the research topics of 'No one tool to rule them all: prokaryotic gene prediction tool annotations are highly dependent on the organism of study'. Together they form a unique fingerprint.

Cite this