Improving the quality of evidence of clinical trials for AI interventions
This project is developing the world’s first guidelines to make sure that AI interventions for health are supported by the best possible evidence.
Artificial intelligence (AI) is a relatively new area for clinical study. And although there is lots of excitement about the potential of AI for improving health and care, its application in clinical practice is still unpredictable, with few interventions making it from published paper to real-world implementation. Part of the problem is that there are no specific, universally agreed minimum standards for clinical trials of AI health interventions, which are crucial for making sure that studies follow good scientific practice and that reporting is complete, comprehensive and transparent.
This makes it hard for researchers to properly design and report their studies, and to address potential sources of bias. It also makes it difficult for the health community to assess the quality of the findings and how (or even whether) an AI intervention might work in a particular setting. The risk is that we adopt new health technologies and products whose safety and efficacy aren’t adequately supported by evidence.
This project brings together experts from around the globe to develop the world’s first reporting standards for clinical trials of health interventions involving AI. This collaborative, international effort is building consensus among clinicians, data scientists, healthcare regulators and national bodies, journal editors, patients and others about the AI-specific considerations for clinical trials, using as a starting point the existing SPIRIT and CONSORT guidelines for clinical trials (of drugs and other therapies). Based on this, it has already developed two checklists – SPIRIT-AI and CONSORT-AI – that can be used by researchers, peer reviewers and journal editors conducting and appraising clinical trials for AI systems.
The SPIRIT-AI and CONSORT-AI guidelines are the first internationally recognised standards for reporting AI clinical trials and their protocols. They will help to put authors, peer reviewers and journal editors on a level playing field to make sure AI clinical trials are supported by the highest level of evidence.
The impact and outcomes
The creation of the SPIRIT-AI and CONSORT-AI standards will be a crucial first step in helping to bridge the gap between clinical AI trial data and real-world care. Not only will the standards help researchers by providing clear guidance to follow, but they will also provide a benchmark against which the health and care community can assess future studies to make sure that we adopt safe and effective technologies and approaches to deliver the best possible care.
Like the existing guidelines, the hope is that the criteria will be endorsed and adopted by the world’s most influential journals, researchers and experts, to ultimately become a global standard for clinical AI trials everywhere.
Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, and the EQUATOR Network
This work was funded by a Wellcome Trust Institutional Strategic Support Fund: Digital Health Pilot Grant, Research England (part of UK Research and Innovation), Health Data Research UK and the Alan Turing Institute
and Melanie Calvert and Samantha Cruz Rivera.
New research questions reliability of studies into Lassa fever drug
28 February 2023
Researchers identified and re-analysed a series of studies to assess the reliability of the evidence backing the use of ribavirin as a treatment for Lassa fever.
Early Warning (NEWS2) Scores to Predict Risk of Severe COVID-19
16 February 2023
Researched used nationwide patient data to evaluate the effectiveness of the NEWS2 score for predicting who was at greatest risk of intensive care.
COVID-19 mortality risk for inflammatory arthritis patients: a cohort study using SAIL Databank
24 November 2022
A study of inflammatory arthritis (IA) patients found that shielding reduced the incidence of COVID-19. IA was not associated with an increased risk of dying within 28 days, but being vulnerable...