Medicine and Society
Feb 2019

Making Policy on Augmented Intelligence in Health Care

Elliott Crigger, PhD and Christopher Khoury, MSc, MBA
AMA J Ethics. 2019;21(2):E188-191. doi: 10.1001/amajethics.2019.188.

Abstract

In June 2018, the American Medical Association adopted new policy to provide a broad framework for the evolution of artificial intelligence (AI) in health care that is designed to help ensure that AI realizes the benefits it promises for patients, physicians, and the health care community.

Responding to Industry Activity on Artificial Intelligence

In June 2018, the American Medical Association (AMA) adopted a new policy, H-480.940, “Augmented Intelligence in Health Care,”1 to provide a broad framework for the evolution of artificial intelligence (AI) in health care that is designed to help ensure that AI realizes the benefits it promises for patients, physicians, and the health care community. In parallel, a wave of scientific and investment activity is cresting, focused on AI and its applications in health care and medicine. Advances in computing power, storage, sensors, and multidisciplinary research have laid the groundwork for the increased development and use of AI techniques in health care.

AI is the ability of a computer to function appropriately and with foresight in its environment, that is, to complete tasks in a manner typically associated with a rational human being. Augmented intelligence is an alternative conceptualization that focuses on AI’s assistive role, emphasizing a design approach and implementation that enhances human intelligence rather than replaces it. Collectively, these areas of scientific research and health care industry activity represent thousands of peer-reviewed studies and, by one estimate, over $2.7 billion in investment across 121 digital health companies and 206 funding deals between 2011 and 2017, just within the United States.2

The background report that informs this new policy focused on 2 fundamental normative conditions for appropriate integration of AI into health care.3 First, health care AI should be understood as a tool to augment professional clinical judgment, not a technology to replace or override it. Second, the development of health care AI tools should attend carefully to the design and evaluation of individual applications, to issues of patient privacy, and to thoughtful clinical implementation.

Ethical Dimensions of AI in Health Care Practice

Design challenges loom large. An AI-derived algorithm is only as good as the data with which it works.

The research, patient care, and insurance records available as training data sets for health care AI can be highly variable.… Clinical trials systematically include or exclude participants with certain characteristics; patient charts and insurance records capture information only from those individuals who have access to the health care system.3

Rarely do such records contain information about social determinants of health, for example. AI systems can, invisibly and unintentionally, reproduce or magnify the biases of their human designers or training data sets in ways that risk exacerbating existing health disparities—such as when data reflect only the conditions of individuals who have access to health care to begin with.4

Conversely, algorithms properly designed and deployed can help compensate for or minimize human bias. AI algorithms must also be evaluated using criteria that are “clinically relevant and evaluation should be representative of how the algorithm will be applied in practice.”3 Predictive algorithms, for example, should be able to predict events sufficiently in advance to meaningfully influence care decisions and patient outcomes.

Addressing concerns about privacy and security are 2 further challenges for the evolution of health care AI. Existing practices of notifying patients and obtaining consent for data use are not adequate, nor are strategies for de-identifying data effective in the context of large, complex data sets when machine learning algorithms can re-identify a record from as few as 3 data points.5 Algorithms can also be vulnerable to cyberattack.6 Evolving technical responses to the challenge of ensuring data security and integrity, such as blockchain-style technologies,7 are promising, but traditional expectations for health care privacy might no longer be attainable. Rigorous oversight of data use and transfer will be critical to protecting patients’ interests.

To realize its promise, health care AI must be deployed in ways that promote quality of care and minimize potentially disruptive effects. Physicians will need to learn to work effectively with AI systems,3 just as medical students and residents are now trained to work effectively with electronic health records.8 If physicians are to base clinical recommendations on AI, “they will need to understand AI methods and systems sufficiently to be able to trust an algorithm’s predictions.”3 Even as technical solutions to the problem of trust evolve, such as algorithms that “explain” to users why a particular prediction has been made, the health care organizations that implement AI systems should vigilantly monitor the operation of those systems to identify and address adverse consequences.

Legal experts and commercial developers of AI tools that aid in diagnosis must also begin to address questions of liability when incorrect diagnoses are made either by humans using augmented intelligence tools or by AI tools directly. Questions also remain about the evolving role of the patient-physician relationship and fiduciary compact in an algorithm-enabled health care environment.9

The AMA’s adoption of H-480.940 suggests the ethical importance of these questions in calling for development of thoughtfully designed, high-quality, clinically validated health care AI that does the following1:

a) is designed and evaluated in keeping with best practices in user-centered design, particularly for physicians and other members of the health care team;
b) is transparent;
c) conforms to leading standards for reproducibility;
d) identifies and takes steps to address bias and avoids introducing or exacerbating health care disparities including when testing or deploying new AI tools on vulnerable populations; and
e) safeguards patients’ and other individuals’ privacy interests and preserves the security and integrity of personal information.

Values of ethical relevance considered in this policy include professionalism, transparency, justice, safety, and privacy.

References

  1. American Medical Association. Augmented intelligence in health care H-480.940. https://policysearch.ama-assn.org/policyfinder/detail/augmented%20intelligence?uri=%2FAMADoc%2FHOD.xml-H-480.940.xml. Modified 2018. Accessed November 14, 2018.
  2. Zweig M, Tran D. The AI/ML use cases investors are betting on in healthcare. Rock Health. https://rockhealth.com/reports/the-ai-ml-use-cases-investors-are-betting-on-in-healthcare/. Accessed November 30, 2018.

  3. American Medical Association. Augmented intelligence in health care: report 41 of the AMA Board of Trustees. https://static1.squarespace.com/static/58d0113a3e00bef537b02b70/t/5b6aed0a758d4610026a719c/1533734156501/AI_2018_Report_AMA.pdf. Accessed November 14, 2018.

  4. AI Now Institute. The AI Now report: the social and economic implications of artificial intelligence technologies in the near-term. New York, NY: AI Now Institute; 2016. https://ainowinstitute.org/AI_Now_2016_Report.pdf. Updated September 22, 2016. Accessed January 26, 2018.

  5. Osoba O, Welser W IV. An intelligence in our image: the risks of bias and errors in artificial intelligence. Santa Monica, CA: Rand Corporation; 2017. https://www.rand.org/pubs/research_reports/RR1744.html. Accessed February 19, 2018.

  6. Finlayson SG, Won Chung H, Kohan IS, Beam AL. Adversarial attacks against medical deep learning systems. arXiv. https://arxiv.org/abs/1804.05296. Published April 15, 2018. Updated May 21, 2018. Accessed August 9, 2018.

  7. JASON. Artificial intelligence for health and health care. McClean, VA: MITRE Corporation; 2017. https://www.healthit.gov/sites/default/files/jsr-17-task-002_aiforhealthandhealthcare12122017.pdf. Accessed February 19, 2018.

  8. Miliard M. AMA, Regenstrief launch EHR training platform for medical students. Healthcare IT News. April 19, 2017. https://www.healthcareitnews.com/news/ama-regenstrief-launch-ehr-training-platform-medical-students. Accessed December 5, 2018.

  9. Char Ds, Shah NH, Magnus D. Implementing machine learning in health care—addressing ethical challenges. N Engl J Med. 2018;378(11):981-983.

Citation

AMA J Ethics. 2019;21(2):E188-191.

DOI

10.1001/amajethics.2019.188.

Conflict of Interest Disclosure

The author(s) had no conflicts of interest to disclose. 

The viewpoints expressed in this article are those of the author(s) and do not necessarily reflect the views and policies of the AMA.