Artificial intelligence (AI) is a relatively new area for clinical study. And although there is lots of excitement about the potential of AI for improving health and care, its application in clinical practice is still unpredictable, with few interventions making it from published paper to real-world implementation. Part of the problem is that there are no specific, universally agreed minimum standards for clinical trials of AI health interventions, which are crucial for making sure that studies follow good scientific practice and that reporting is complete, comprehensive and transparent.
This makes it hard for researchers to properly design and report their studies, and to address potential sources of bias. It also makes it difficult for the health community to assess the quality of the findings and how (or even whether) an AI intervention might work in a particular setting. The risk is that we adopt new health technologies and products whose safety and efficacy aren’t adequately supported by evidence.
This project brings together experts from around the globe to develop the world’s first reporting standards for clinical trials of health interventions involving AI. This collaborative, international effort is building consensus among clinicians, data scientists, healthcare regulators and national bodies, journal editors, patients and others about the AI-specific considerations for clinical trials, using as a starting point the existing SPIRIT and CONSORT guidelines for clinical trials (of drugs and other therapies). Based on this, it has already developed two checklists – SPIRIT-AI and CONSORT-AI – that can be used by researchers, peer reviewers and journal editors conducting and appraising clinical trials for AI systems.
The SPIRIT-AI and CONSORT-AI guidelines are the first internationally recognised standards for reporting AI clinical trials and their protocols. They will help to put authors, peer reviewers and journal editors on a level playing field to make sure AI clinical trials are supported by the highest level of evidence.
The impact and outcomes
The creation of the SPIRIT-AI and CONSORT-AI standards will be a crucial first step in helping to bridge the gap between clinical AI trial data and real-world care. Not only will the standards help researchers by providing clear guidance to follow, but they will also provide a benchmark against which the health and care community can assess future studies to make sure that we adopt safe and effective technologies and approaches to deliver the best possible care.
Like the existing guidelines, the hope is that the criteria will be endorsed and adopted by the world’s most influential journals, researchers and experts, to ultimately become a global standard for clinical AI trials everywhere.
Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, and the EQUATOR Network
This work was funded by a Wellcome Trust Institutional Strategic Support Fund: Digital Health Pilot Grant, Research England (part of UK Research and Innovation), Health Data Research UK and the Alan Turing Institute
and Melanie Calvert and Samantha Cruz Rivera.
Do home adaptation interventions help to reduce emergency fall admissions? A national longitudinal data-linkage study of 657,536 older adults living in Wales (UK) between 2010 and 2017
21 April 2022
Falls are a significant concern for older people and the NHS, however there is little evidence for the effectiveness of services to make adaptations at home. Researchers in Wales have used health...
GP consultation rates for sequelae after acute covid-19 in patients managed in the community or hospital in the UK: population based study
20 April 2022
The long-term effects in some people with COVID-19 are still poorly understood. This new research used large-scale data to understand how these effects differ based on the severity of the initial...
The Office for National Statistics and BREATHE Announce Partnership to improve respiratory health
10 March 2022
BREATHE, our Health Data Research Hub for Respiratory Health, have today announced their partnership with the Office for National Statistics (ONS) to improve respiratory health in the UK through...