PDF icon PDF herunterladen
Open Access
CC BY-NC-ND 4.0 · Appl Clin Inform 2024; 15(02): 306-312
DOI: 10.1055/a-2281-7092
Research Article

A Survey of Clinicians' Views of the Utility of Large Language Models

Matthew Spotnitz
1   Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, New York, United States
,
Betina Idnay
1   Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, New York, United States
,
Emily R. Gordon
1   Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, New York, United States
2   Department of Dermatology, Vagelos College of Physicians and Surgeons, Columbia University Irving Medical Center, New York, New York, United States
,
Rebecca Shyu
1   Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, New York, United States
,
Gongbo Zhang
1   Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, New York, United States
,
Cong Liu
1   Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, New York, United States
,
James J. Cimino
1   Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, New York, United States
3   Department of Biomedical Informatics and Data Science, Informatics Institute, Heersink School of Medicine, University of Alabama at Birmingham, Birmingham, Alabama, United States
,
Chunhua Weng
1   Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, New York, United States
› Institutsangaben
Funding This work was supported by National Library of Medicine (NLM) grants R01LM014344 and R01LM009886 to C.W., National Human Genome Institute grant R01HG012655 to C.L., and by National Center for Advancing Clinical and Translational Science grant UL1TR001873 to Columbia University Irving Medical Center. B.I. and R.S. acknowledge support from NLM grant T15LM007079.
› Weitere Informationen
Auch verfügbar aufeRef
  Lizenzen und Reprints
Preview

Abstract

Objectives Large language models (LLMs) like Generative pre-trained transformer (ChatGPT) are powerful algorithms that have been shown to produce human-like text from input data. Several potential clinical applications of this technology have been proposed and evaluated by biomedical informatics experts. However, few have surveyed health care providers for their opinions about whether the technology is fit for use.

Methods We distributed a validated mixed-methods survey to gauge practicing clinicians' comfort with LLMs for a breadth of tasks in clinical practice, research, and education, which were selected from the literature.

Results A total of 30 clinicians fully completed the survey. Of the 23 tasks, 16 were rated positively by more than 50% of the respondents. Based on our qualitative analysis, health care providers considered LLMs to have excellent synthesis skills and efficiency. However, our respondents had concerns that LLMs could generate false information and propagate training data bias.

Our survey respondents were most comfortable with scenarios that allow LLMs to function in an assistive role, like a physician extender or trainee.

Conclusion In a mixed-methods survey of clinicians about LLM use, health care providers were encouraging of having LLMs in health care for many tasks, and especially in assistive roles. There is a need for continued human-centered development of both LLMs and artificial intelligence in general.

Keywords

artificial intelligence - clinical decision support - clinical informatics - clinical information systems

Protection of Human Subjects

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects, and was reviewed by Columbia University Irving Medical Center Institutional Review Board (AAAU7954).


Supplementary Material



Publikationsverlauf

Eingereicht: 01. Dezember 2023

Angenommen: 15. Februar 2024

Accepted Manuscript online:
05. März 2024

Artikel online veröffentlicht:
17. April 2024

© 2024. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany