The challenge

Artificial intelligence (AI) is a relatively new area for clinical study. And although there is lots of excitement about the potential of AI for improving health and care, its application in clinical practice is still unpredictable, with few interventions making it from published paper to real-world implementation. Part of the problem is that there are no specific, universally agreed minimum standards for clinical trials of AI health interventions, which are crucial for making sure that studies follow good scientific practice and that reporting is complete, comprehensive and transparent.

This makes it hard for researchers to properly design and report their studies, and to address potential sources of bias. It also makes it difficult for the health community to assess the quality of the findings and how (or even whether) an AI intervention might work in a particular setting. The risk is that we adopt new health technologies and products whose safety and efficacy aren’t adequately supported by evidence.

The solution

This project brings together experts from around the globe to develop the world’s first reporting standards for clinical trials of health interventions involving AI. This collaborative, international effort is building consensus among clinicians, data scientists, healthcare regulators and national bodies, journal editors, patients and others about the AI-specific considerations for clinical trials, using as a starting point the existing SPIRIT and CONSORT guidelines for clinical trials (of drugs and other therapies). Based on this, it has already developed two checklists – SPIRIT-AI and CONSORT-AI – that can be used by researchers, peer reviewers and journal editors conducting and appraising clinical trials for AI systems.

The SPIRIT-AI and CONSORT-AI guidelines are the first internationally recognised standards for reporting AI clinical trials and their protocols. They will help to put authors, peer reviewers and journal editors on a level playing field to make sure AI clinical trials are supported by the highest level of evidence.

—Xiao Liu

The impact and outcomes

The creation of the SPIRIT-AI and CONSORT-AI standards will be a crucial first step in helping to bridge the gap between clinical AI trial data and real-world care. Not only will the standards help researchers by providing clear guidance to follow, but they will also provide a benchmark against which the health and care community can assess future studies to make sure that we adopt safe and effective technologies and approaches to deliver the best possible care.

Like the existing guidelines, the hope is that the criteria will be endorsed and adopted by the world’s most influential journals, researchers and experts, to ultimately become a global standard for clinical AI trials everywhere.

Partners

Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, and the EQUATOR Network

This work was funded by a Wellcome Trust Institutional Strategic Support Fund: Digital Health Pilot Grant, Research England (part of UK Research and Innovation), Health Data Research UK and the Alan Turing Institute

Project Leads

and Melanie Calvert and Samantha Cruz Rivera.