26
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Clinical artificial intelligence quality improvement: towards continual monitoring and updating of AI algorithms in healthcare

      review-article

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Machine learning (ML) and artificial intelligence (AI) algorithms have the potential to derive insights from clinical data and improve patient outcomes. However, these highly complex systems are sensitive to changes in the environment and liable to performance decay. Even after their successful integration into clinical practice, ML/AI algorithms should be continuously monitored and updated to ensure their long-term safety and effectiveness. To bring AI into maturity in clinical care, we advocate for the creation of hospital units responsible for quality assurance and improvement of these algorithms, which we refer to as “AI-QI” units. We discuss how tools that have long been used in hospital quality assurance and quality improvement can be adapted to monitor static ML algorithms. On the other hand, procedures for continual model updating are still nascent. We highlight key considerations when choosing between existing methods and opportunities for methodological innovation.

          Related collections

          Most cited references64

          • Record: found
          • Abstract: found
          • Article: found
          Is Open Access

          MIMIC-III, a freely accessible critical care database

          MIMIC-III (‘Medical Information Mart for Intensive Care’) is a large, single-center database comprising information relating to patients admitted to critical care units at a large tertiary care hospital. Data includes vital signs, medications, laboratory measurements, observations and notes charted by care providers, fluid balance, procedure codes, diagnostic codes, imaging reports, hospital length of stay, survival data, and more. The database supports applications including academic and industrial research, quality improvement initiatives, and higher education coursework.
            Bookmark
            • Record: found
            • Abstract: found
            • Article: not found

            A guide to deep learning in healthcare

            Here we present deep-learning techniques for healthcare, centering our discussion on deep learning in computer vision, natural language processing, reinforcement learning, and generalized methods. We describe how these computational techniques can impact a few key areas of medicine and explore how to build end-to-end systems. Our discussion of computer vision focuses largely on medical imaging, and we describe the application of natural language processing to domains such as electronic health record data. Similarly, reinforcement learning is discussed in the context of robotic-assisted surgery, and generalized deep-learning methods for genomics are reviewed.
              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network

              Computerized electrocardiogram (ECG) interpretation plays a critical role in the clinical ECG workflow1. Widely available digital ECG data and the algorithmic paradigm of deep learning2 present an opportunity to substantially improve the accuracy and scalability of automated ECG analysis. However, a comprehensive evaluation of an end-to-end deep learning approach for ECG analysis across a wide variety of diagnostic classes has not been previously reported. Here, we develop a deep neural network (DNN) to classify 12 rhythm classes using 91,232 single-lead ECGs from 53,549 patients who used a single-lead ambulatory ECG monitoring device. When validated against an independent test dataset annotated by a consensus committee of board-certified practicing cardiologists, the DNN achieved an average area under the receiver operating characteristic curve (ROC) of 0.97. The average F1 score, which is the harmonic mean of the positive predictive value and sensitivity, for the DNN (0.837) exceeded that of average cardiologists (0.780). With specificity fixed at the average specificity achieved by cardiologists, the sensitivity of the DNN exceeded the average cardiologist sensitivity for all rhythm classes. These findings demonstrate that an end-to-end deep learning approach can classify a broad range of distinct arrhythmias from single-lead ECGs with high diagnostic performance similar to that of cardiologists. If confirmed in clinical settings, this approach could reduce the rate of misdiagnosed computerized ECG interpretations and improve the efficiency of expert human ECG interpretation by accurately triaging or prioritizing the most urgent conditions.
                Bookmark

                Author and article information

                Contributors
                jean.feng@ucsf.edu
                Journal
                NPJ Digit Med
                NPJ Digit Med
                NPJ Digital Medicine
                Nature Publishing Group UK (London )
                2398-6352
                31 May 2022
                31 May 2022
                2022
                : 5
                : 66
                Affiliations
                [1 ]GRID grid.266102.1, ISNI 0000 0001 2297 6811, Department of Epidemiology and Biostatistics, , University of California, ; San Francisco, CA USA
                [2 ]GRID grid.266102.1, ISNI 0000 0001 2297 6811, Bakar Computational Health Sciences Institute, , University of California San Francisco, ; San Francisco, CA USA
                [3 ]GRID grid.47840.3f, ISNI 0000 0001 2181 7878, Department of Biostatistics, , University of California, ; Berkeley, CA USA
                [4 ]GRID grid.266102.1, ISNI 0000 0001 2297 6811, Department of Anesthesia, , University of California, ; San Francisco, CA USA
                [5 ]GRID grid.239395.7, ISNI 0000 0000 9011 8547, Institute for Medical Engineering and Science, Massachusetts Institute of Technology, , Department of Medicine, Beth Israel Deaconess Medical Center; Department of Biostatistics, Harvard T.H. Chan School of Public Health, ; Boston, MA 02115 USA
                Author information
                http://orcid.org/0000-0003-2041-3104
                http://orcid.org/0000-0002-8474-591X
                http://orcid.org/0000-0002-5427-482X
                http://orcid.org/0000-0001-6712-6626
                Article
                611
                10.1038/s41746-022-00611-y
                9156743
                35641814
                5ad2727b-3e38-4990-966d-40d71b5da94b
                © The Author(s) 2022

                Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

                History
                : 16 November 2021
                : 29 April 2022
                Funding
                Funded by: FundRef https://doi.org/10.13039/100005831, Foundation for Anesthesia Education and Research (FAER);
                Funded by: FundRef https://doi.org/10.13039/100000070, U.S. Department of Health & Human Services | NIH | National Institute of Biomedical Imaging and Bioengineering (NIBIB);
                Award ID: R01EB017205
                Award Recipient :
                Categories
                Perspective
                Custom metadata
                © The Author(s) 2022

                public health,statistics
                public health, statistics

                Comments

                Comment on this article

                scite_
                0
                0
                0
                0
                Smart Citations
                0
                0
                0
                0
                Citing PublicationsSupportingMentioningContrasting
                View Citations

                See how this article has been cited at scite.ai

                scite shows how a scientific paper has been cited by providing the context of the citation, a classification describing whether it supports, mentions, or contrasts the cited claim, and a label indicating in which section the citation was made.

                Similar content95

                Cited by55

                Most referenced authors3,284