Don’t judge a book or health app by its cover: user ratings and downloads are not linked to quality

Maciej Hyzy, Raymond Bond, Maurice Mulvenna, Lu Bai, Anna-Lena Frey, Jorge Carracedo, Robert Daly, Simon Leigh

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)
16 Downloads (Pure)


To analyse the relationship between health app quality with user ratings and the number of downloads of corresponding health apps.
Utilising a dataset of 881 Android-based health apps, assessed via the 300-point objective Organisation for the Review of Care and Health Applications (ORCHA) assessment tool, we explored whether subjective user-level indicators of quality (user ratings and downloads) correlate with objective quality scores in the domains of user experience, data privacy and professional/clinical assurance. For this purpose, we applied spearman correlation and multiple linear regression models.
For user experience, professional/clinical assurance and data privacy scores, all models had very low adjusted R squared values (<.02). Suggesting that there is no meaningful link between subjective user ratings or the number of health app downloads and objective quality measures. Spearman correlations suggested that prior downloads only had a very weak positive correlation with user experience scores (Spearman = .084, p=.012) and data privacy scores (Spearman = .088, p = .009). There was a very weak negative correlation between downloads and professional/clinical assurance score (Spearman = -.081, p = .016). Additionally, user ratings demonstrated a very weak correlation with no statistically significant correlations observed between user ratings and the scores (all p > 0.05). For ORCHA scores multiple linear regression had adjusted R-squared = -.002.
This study highlights that widely available proxies which users may perceive to signify the quality of health apps, namely user ratings and downloads, are inaccurate predictors for estimating quality. This indicates the need for wider use of quality assurance methodologies which can accurately determine the quality, safety, and compliance of health apps. Findings suggest more should be done to enable users to recognise high-quality health apps, including digital health literacy training and the provision of nationally endorsed “libraries”.
Original languageEnglish
Article numbere0298977
Number of pages11
JournalPLoS One
Issue number3
Publication statusPublished - 04 Mar 2024


Dive into the research topics of 'Don’t judge a book or health app by its cover: user ratings and downloads are not linked to quality'. Together they form a unique fingerprint.

Cite this