Not all AI health tools with regulatory authorization are clinically validated (original) (raw)

Devices that lack adequate clinical validation pose risks for patient care. A new validation standard is proposed to evaluate FDA authorization as an indication of clinical effectiveness in medical AI.

This is a preview of subscription content, access via your institution

Change history

A Correction to this paper has been published: https://doi.org/10.1038/s41591-024-03293-z

References

  1. Murphy, K. et al. BMC Med. Ethics 22, 14 (2021).
    Article PubMed PubMed Central Google Scholar
  2. Kim, J. Y. et al. In FAccT '23: Proc. 2023 ACM Conference on Fairness, Accountability, and Transparency 1396–1417 (2023).
  3. Geis, J. R. et al. Can. Assoc. Radiol. J. 70, 329–334 (2019).
    Article PubMed Google Scholar
  4. Evans, B. J. J. Law Biosci. 10, lsad001 (2023).
    Article PubMed PubMed Central Google Scholar
  5. Vellido, A. Kidney Dis. 5, 11–17 (2018).
    Article Google Scholar
  6. Center for Devices and Radiological Health. FDA https://go.nature.com/4dDbUct (6 September 2023).
  7. US Food and Drug Administration. https://go.nature.com/3YN8Civ (14 October 2016).
  8. US Food and Drug Administration. https://go.nature.com/3LYd32i (September 2023).
  9. US Food and Drug Administration. https://go.nature.com/3SOXHRw (accessed 17 September 2023).
  10. Lyell, D., Wang, Y., Coiera, E. & Magrabi, F. J. Am. Med. Inform. Assoc. 30, 1227–1236 (2023).
    Article PubMed PubMed Central Google Scholar
  11. US Food and Drug Administration. https://go.nature.com/3YwMt87 (accessed 14 June 2023).
  12. Kalantaridis, C. & Küttim, M. J. Technol. Transfer 46, 1487–1513 (2021).
    Article Google Scholar
  13. MIT News Office. https://go.nature.com/3LVzBkb (7 September 2017).
  14. Sandhu, S. et al. Patterns 4, 100710 (2023).
    Article PubMed PubMed Central Google Scholar
  15. Tierney, W. M., Meslin, E. M. & Kroenke, K. J. Gen. Intern. Med. 31, 228–233 (2016).
    Article PubMed Google Scholar
  16. Harris, P., Winney, I., Ashhurst-Smith, C., O'Brien, M. & Graves, S. Pathology 44, 583–585 (2012).
    Article PubMed Google Scholar
  17. Wu, E. et al. Nat. Med. 27, 582–584 (2021).
    Article CAS PubMed Google Scholar

Download references

Acknowledgements

For their consultation, comments and mentorship, the authors thank M. R. Patel, W. Ratliff, Z. Hussain, J. Castillo, I. Yüksel, H. Chen, P. Guo, H. McInnis, J. E. Hancock, B. J. Evans and A. Merlo.

Author information

Author notes

  1. These authors contributed equally: Adonis Abdullah, Ying Fang.

Authors and Affiliations

  1. UNC School of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
    Sammy Chouffani El Fassi, Adonis Abdullah, Ying Fang & Naya Kayali
  2. Duke Heart Center, Duke University School of Medicine, Durham, NC, USA
    Sammy Chouffani El Fassi
  3. University of South Carolina School of Medicine, Columbia, SC, USA
    Ying Fang
  4. Department of Pharmacology, University of Oxford, Oxford, UK
    Sarabesh Natarajan
  5. Columbia University Vagelos College of Physicians and Surgeons, Columbia University, New York, NY, USA
    Sarabesh Natarajan
  6. Department of Biomedical Engineering, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
    Awab Bin Masroor
  7. ECU School of Dental Medicine, East Carolina University, Greenville, NC, USA
    Naya Kayali
  8. University of Miami Miller School of Medicine, University of Miami, Miami, FL, USA
    Simran Prakash
  9. Department of Social Medicine, School of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
    Gail E. Henderson

Authors

  1. Sammy Chouffani El Fassi
  2. Adonis Abdullah
  3. Ying Fang
  4. Sarabesh Natarajan
  5. Awab Bin Masroor
  6. Naya Kayali
  7. Simran Prakash
  8. Gail E. Henderson

Corresponding author

Correspondence toSammy Chouffani El Fassi.

Ethics declarations

Competing interests

The authors declare no competing interests.

Rights and permissions

About this article

Cite this article

Chouffani El Fassi, S., Abdullah, A., Fang, Y. et al. Not all AI health tools with regulatory authorization are clinically validated.Nat Med 30, 2718–2720 (2024). https://doi.org/10.1038/s41591-024-03203-3

Download citation