Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence

By Gary S. Collins, P. Dhiman, C. A. Andaur Navarro, Ji Ma, Lotty Hooft, Johannes B Reitsma, P. Logullo, Andrew Beam, Lily H. Peng, B. Van calster and 3 more. BMJ Open, 2021

676 citations recorded by the source corpus.

Introduction The Transparent Reporting of a multivariable prediction model of Individual Prognosis Or Diagnosis (TRIPOD) statement and the Prediction model Risk Of Bias ASsessment Tool (PROBAST) were both published to improve the reporting and critical appraisal of prediction model studies for diagnosis and prognosis. This paper describes the processes and methods that will be used to develop an extension to the TRIPOD statement (TRIPOD-artificial intelligence, AI) and the PROBAST (PROBAST-AI) tool for prediction model studies that applied machine learning techniques. Methods and analysis TRIPOD-AI and PROBAST-AI will be developed following published guidance from the EQUATOR Network, and will comprise five stages. Stage 1 will comprise two systematic reviews (across all medical fields and specifically in oncology) to examine the quality of reporting in published machine-learning-based prediction model studies. In stage 2, we will consult a diverse group of key stakeholders using a Delphi process to identify items to be considered for inclusion in TRIPOD-AI and PROBAST-AI. Stage 3 will be virtual consensus meetings to consolidate and prioritise key items to be included in TRIPOD-AI and PROBAST-AI. Stage 4 will involve developing the TRIPOD-AI checklist and the PROBAST-AI tool, and writing the accompanying explanation and elaboration papers. In the final stage, stage 5, we will disseminate TRIPOD-AI and PROBAST-AI via journals, conferences, blogs, websites (including TRIPOD, PROBAST and EQUATOR Network) and social media. TRIPOD-AI will provide researchers working on prediction model studies based on machine learning with a reporting guideline that can help them report key details that readers need to evaluate the study quality and interpret its findings, potentially reducing research waste. We anticipate PROBAST-AI will help researchers, clinicians, systematic reviewers and policymakers critically appraise the design, conduct and analysis of machine learning based prediction model studies, with a robust standardised tool for bias evaluation. Ethics and dissemination Ethical approval has been granted by the Central University Research Ethics Committee, University of Oxford on 10-December-2020 (R73034/RE001). Findings from this study will be disseminated through peer-review publications. PROSPERO registration number CRD42019140361 and CRD42019161764.

Subfields: Health Informatics, Radiology, Nuclear Medicine and Imaging, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/dbde9692749150152354690e0706c3032b9c6be0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Health Informatics

In Radiology, Nuclear Medicine and Imaging

In Statistics and Probability

Citations in the corpus

PROBAST+AI: an updated quality, risk of bias, and applicability assessment tool for prediction models using regression or artificial intelligence methods (2025, neutral): “The protocol for updating PROBAST-2019 has been published 41 and is also available on the Open Science Framework.”
Agentic AI: Autonomous Intelligence for Complex Goals—A Comprehensive Survey (2025, neutral): “One such framework primarily employed is the risk evaluation and management protocol [64], which leads to the review of the risks posed by the actions of the AI, and subsequently, the control measures are put in place.”
TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods (2024, background): “The protocol, which also describes the development of a quality assessment and risk-of-bias tool for prediction models developed using machine learning methods (PROBAST+AI), was published in 2021.”
FUTURE-AI: international consensus guideline for trustworthy and deployable artificial intelligence in healthcare (2023, positive): “AI traceability will also enable continuous auditing of AI models, 47 identify risks and limitations, and update the AI models when needed.”