K. D. Ahlquist, Lauren A. Sugden, Sohini Ramachandran
Abstract
Machine learning tools have proven useful across biological disciplines, allowing researchers to draw conclusions from large datasets, and opening up new opportunities for interpreting complex and heterogeneous biological data. Alongside the rapid growth of machine learning, there have also been growing pains: some models that appear to perform well have later been revealed to rely on features of the data that are artifactual or biased; this feeds into the general criticism that machine learning models are designed to optimize model performance over the creation of new biological insights.
Introduction
Interpretability has become an increasingly important area of research in machine learning, although the term “interpretability” is not always well-defined When machine learning approaches are applied to biological problems, two major elements of interpretability are desirable: 1) the ability to connect results generated by machine learning applications with existing biological theory and understanding of biological mechanisms, and 2) the ability to identify and characterize the limitations of a given machine learning algorithm, so that it is applied correctly and appropriately in the biological context of interest.
Material and Methods:
This framework can be generalized to other applications by simply changing the classification categories (adaptive and neutral), and with adjustment of the Bayesian prior π according to the specific application. Attributes denoted by X must be continuous variables. Eq 1 (Results) adapts the numerator of SWIF(r), which approximates the likelihood of the observed data conditioned on each class using both the marginal distributions of the attributes and the two-dimensional joint distributions of the attributes, from which conditional distributions are derived.
Discussion
In this study we introduce the SWIF(r) reliability score (SRS), which assesses the trustworthiness of the classification for a specific instance using the model underlying the SWIF(r) classifier. We demonstrate the utility of the SRS when faced with common challenges in machine learning including: 1) an unknown class present in testing data that was not present in training data, 2) systemic mismatch between training and testing data, and 3) instances of testing data that have missing values for some attributes.
Acknowledgments:
The authors thank Samuel Pattillo Smith for assistance with UKBiobank data.
Citation: Ahlquist KD, Sugden LA, Ramachandran S (2023) Enabling interpretable machine learning for biological data with reliability scores. PLoS Comput Biol 19(5): e1011175. https://doi.org/10.1371/journal.pcbi.1011175
Editor: Luis Pedro Coelho, Fudan University, CHINA
Received: February 24, 2022; Accepted: May 10, 2023; Published: May 26, 2023.
Copyright: © 2023 Ahlquist et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files. Data and code are available on Github: https://github.com/ramachandran-lab/SRS_paper.
Funding: This work was supported by the US National Institutes of Health R01GM118652 (KDA, LAS, SR) and R35GM139628 (KDA, SR), and the Wimmer Family Foundation (LAS). KDA was also supported as a trainee by NIH T32GM007601. The funders had no role in software design, data collection or analysis, decision to publish or the preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
