Advertisement

Ethical Considerations Related to Using Machine Learning-Based Prediction of Mortality in the Pediatric Intensive Care Unit

  • Kelly N. Michelson
    Correspondence
    Corresponding Author: Kelly Michelson, MD MPH 225 East Chicago Avenue, Box 73 Chicago, IL 60611 Phone: 312-227-1606 Fax: 312-227-9753
    Affiliations
    Center for Bioethics and Medical Humanities Institute for Augmented Intelligence in Medicine (I.AIM) and Institute for Public Health and Medicine (IPHAM) Department of Pediatrics, Division of Pediatric Critical Care Medicine Northwestern Feinberg School of Medicine
    Search for articles by this author
  • Craig M. Klugman
    Affiliations
    Department of Health Sciences, DePaul University
    Search for articles by this author
  • Abel N. Kho
    Affiliations
    Center for Health Information Partnerships Institute for Augmented Intelligence in Medicine (I.AIM) and Institute for Public Health and Medicine (IPHAM)Departments of Medicine and Preventive Medicine Northwestern Feinberg School of Medicine Sara Gerke, Dipl-Jur Univ, MAPenn State Dickinson Law
    Search for articles by this author
Published:January 13, 2022DOI:https://doi.org/10.1016/j.jpeds.2021.12.069
      Machine Learning (ML) shows promise for developing prediction models that could improve care in the pediatric intensive care unit (PICU). Advocates claim these systems enhance prognostic accuracy and can adapt to changing clinical practices by adding more and new large-scale child health data. accurate predictive models using ML could benefit decision-making and care delivery, and in turn, outcomes for patients and families. Despite their potential, some of these models may replicate the biases of their training datasets or may be biased in other ways (e.g., label bias or contextual bias), and are built without the capacity to explain how they reach decisions (so-called “black boxes”). Moreover, implicit trust or mistrust in technology may influence patients’, families’, and clinicians’ views of software-generated opinions as more objective and valid than they really are. This essay provides an overview of ethical concerns posed by the advent of ML-based models for mortality prediction in the PICU. We discuss the benefits and risks related to this emerging technology, including consideration of technical questions, care delivery, family experience and decision making, clinician-family relationships, as well as legal and organizational issues.

      Key Words

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic and Personal
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to The Journal of Pediatrics
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Siddique S.
        • Chow J.C.L.
        Machine Learning in Healthcare Communication.
        Encyclopedia. 2021; 1: 220-239
        • Wiens J.
        • Shenoy E.S.
        Machine Learning for Healthcare: On the Verge of a Major Shift in Healthcare Epidemiology.
        Clin Infect Dis. 2018; 66: 149-153
        • McKinney S.M.
        • Sieniek M.
        • Godbole V.
        • Godwin J.
        • Antropova N.
        • Ashrafian H.
        • et al.
        International evaluation of an AI system for breast cancer screening.
        Nature. 2020; 577: 89-94
        • Poplin R.
        • Varadarajan A.V.
        • Blumer K.
        • Liu Y.
        • McConnell M.V.
        • Corrado G.S.
        • et al.
        Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning.
        Nat Biomed Eng. 2018; 2: 158-164
        • Hyland S.L.
        • Faltys M.
        • Hüser M.
        • Lyu X.
        • Gumbsch T.
        • Esteban C.
        • et al.
        Early prediction of circulatory failure in the intensive care unit using machine learning.
        Nat Med. 2020; 26: 364-373
        • Kim S.Y.
        • Kim S.
        • Cho J.
        • Kim Y.S.
        • Sol I.S.
        • Sung Y.
        • et al.
        A deep learning model for real-time mortality prediction in critically ill children.
        Crit Care. 2019; 23: 279
        • Visser I.H.E.
        • Hazelzet J.A.
        • Albers M.J.I.J.
        • Verlaat C.W.M.
        • Hogenbirk K.
        • van Woensel J.B.
        • et al.
        Mortality prediction models for pediatric intensive care: comparison of overall and subgroup specific performance.
        Intensive Care Med. 2013; 39: 942-950
        • Tyagi P.
        • Tullu M.S.
        • Agrawal M.
        Comparison of Pediatric Risk of Mortality III, Pediatric Index of Mortality 2, and Pediatric Index of Mortality 3 in Predicting Mortality in a Pediatric Intensive Care Unit.
        J Pediatr Intensive Care. 2018; 7: 201-206
      1. Winter MC, Day TE, Ledbetter DR, Aczon MD, Newth CJL, Wetzel RC, et al. Machine Learning to Predict Cardiac Death Within 1 Hour After Terminal Extubation. Pediatr Crit Care Med J Soc Crit Care Med World Fed Pediatr Intensive Crit Care Soc. 2021;22:161–171.

        • Obermeyer Z.
        • Powers B.
        • Vogeli C.
        • Mullainathan S.
        Dissecting racial bias in an algorithm used to manage the health of populations.
        Science. 2019; 366: 447-453
        • Babic B.
        • Gerke S.
        • Evgeniou T.
        • Cohen I.G.
        Beware explanations from AI in health care.
        Science. 2021; 373: 284-286
      2. Michelson KN, Patel R, Haber-Barker N, Emanuel L, Frader J. End-of-life care decisions in the PICU: roles professionals play. Pediatr Crit Care Med J Soc Crit Care Med World Fed Pediatr Intensive Crit Care Soc. 2013;14:e34-44.

      3. Michelson KN, Emanuel L, Carter A, Brinkman P, Clayman ML, Frader J. Pediatric intensive care unit family conferences: one mode of communication for discussing end-of-life care decisions. Pediatr Crit Care Med J Soc Crit Care Med World Fed Pediatr Intensive Crit Care Soc. 2011;12:e336-343.

        • Glass H.C.
        • Costarino A.T.
        • Stayer S.A.
        • Brett C.
        • Cladis F.
        • Davis P.J.
        Outcomes for Extremely Premature Infants.
        Anesth Analg. 2015; 120: 1337-1351
        • Babic B.
        • Gerke S.
        • Evgeniou T.
        • Cohen I.G.
        Algorithms on regulatory lockdown in medicine.
        Science. 2019; 366: 1202-1204
        • Davidson J.E.
        • Aslakson R.A.
        • Long A.C.
        • Puntillo K.A.
        • Kross E.K.
        • Hart J.
        • et al.
        Guidelines for Family-Centered Care in the Neonatal, Pediatric, and Adult ICU.
        Crit Care Med. 2017; 45: 103-128
        • Gaube S.
        • Suresh H.
        • Raue M.
        • Merritt A.
        • Berkowitz S.J.
        • Lermer E.
        • et al.
        Do as AI say: susceptibility in deployment of clinical decision-aids.
        Npj Digit Med. 2021; 4: 1-8
        • Longoni C.
        • Bonezzi A.
        • Morewedge C.K.
        Resistance to Medical Artificial Intelligence.
        J Consum Res. 2019; 46: 629-650
        • Asan O.
        • Bayrak A.E.
        • Choudhury A.
        Artificial Intelligence and Human Trust in Healthcare: Focus on Clinicians.
        J Med Internet Res. 2020; 22: e15154
      4. Brohan | M. AI may save one small hospital $20 million [Internet]. Digital Commerce 360. 2018 [cited 2021 Jul 26]. Available from: https://www.digitalcommerce360.com/2018/09/14/ai-may-save-one-small-hospital-20-million/

      5. The cost of AI in radiology: is it really worth it? [Internet]. AI Blog. 2019 [cited 2021 Jul 26]. Available from: https://ai.myesr.org/healthcare/the-cost-of-ai-in-radiology-is-it-really-worth-it/

        • Hassan A.E.
        New Technology Add-On Payment (NTAP) for Viz LVO: a win for stroke care.
        J NeuroInterventional Surg. 2021; 13: 406-408
      6. Wetsman N. Hospitals are selling treasure troves of medical data — what could go wrong? [Internet]. The Verge. 2021 [cited 2021 Jul 26]. Available from: https://www.theverge.com/2021/6/23/22547397/medical-records-health-data-hospitals-research

      7. HIPAA and the Leak of “Deidentified” EHR Data | NEJM [Internet]. [cited 2021 Aug 13]. Available from: https://www.nejm.org/doi/full/10.1056/NEJMp2102616

        • Gerke S.
        • Minssen T.
        • Cohen G.
        Ethical and legal challenges of artificial intelligence-driven healthcare.
        Artif Intell Healthc. 2020; : 295-336
      8. Patients aren’t being told about the AI systems advising their care [Internet]. STAT. 2020 [cited 2021 Aug 3]. Available from: https://www.statnews.com/2020/07/15/artificial-intelligence-patient-consent-hospitals/

      9. Price WN II, Gerke S, Cohen IG. Potential Liability for Physicians Using Artificial Intelligence. JAMA. 2019;322:1765–1766

        • Price W.N.
        • Gerke S.
        • Cohen I.G.
        How Much Can Potential Jurors Tell Us About Liability for Medical Artificial Intelligence?.
        J Nucl Med. 2021; 62: 15-16
        • Tobia K.
        • Nielsen A.
        • Stremitzer A.
        When Does Physician Use of AI Increase Liability?.
        J Nucl Med. 2021; 62: 17-21
      10. Maliha G, Gerke S, Cohen IG, Parikh RB. Artificial Intelligence and Liability in Medicine: Balancing Safety and Innovation. Milbank Q [Internet]. [cited 2021 Aug 3];n/a(n/a). Available from: https://onlinelibrary.wiley.com/doi/abs/10.1111/1468-0009.12504

      11. Maliha G, Gerke S, Parikh RB, Cohen IG. To Spur Growth in AI, We Need a New Approach to Legal Liability. Harvard Business Review [Internet]. 2021 Jul 13 [cited 2021 Aug 3]; Available from: https://hbr.org/2021/07/to-spur-growth-in-ai-we-need-a-new-approach-to-legal-liability

      12. Price WN, Cohen IG. Privacy in the age of medical big data. Nat Med. 2019;25:37–43.

      13. California Consumer Privacy Act (CCPA) [Internet]. State of California - Department of Justice - Office of the Attorney General. 2018 [cited 2021 Aug 3]. Available from: https://oag.ca.gov/privacy/ccpa

      14. LIS > Bill Tracking > SB1392 > 2021 session [Internet]. [cited 2021 Aug 3]. Available from: https://lis.virginia.gov/cgi-bin/legp604.exe?211+sum+SB1392

      15. State Laws Related to Digital Privacy [Internet]. [cited 2021 Aug 13]. Available from: https://www.ncsl.org/research/telecommunications-and-information-technology/state-laws-related-to-internet-privacy.aspx