ETHICAL AI

This new article from Dr. Shauly and his research team addresses the opportunities and inherent risks of integrating generative AI (genAI) technologies like large language models into surgical care. They propose five ethical principles, including data transparency, patient autonomy, safety and accountability, equity, and sustainability, which are adapted from World Health Organization guidelines to govern genAI adoption in plastic surgery. The article details ethical challenges such as algorithmic bias, lack of transparency in AI decision-making, and the potential for patient data breaches, emphasizing that no genAI has yet received Food and Drug Administration (FDA) approval for surgical use. Through hypothetical scenarios and a code of conduct, the text stresses the importance of rigorous testing, informed consent, and human oversight to ensure genAI ethically enhances, rather than compromises, patient trust and care.

Nip Talk: Gen-AI In Medicine

Comprehensive Study Guide

Key Terms

AI Hallucination: The production of logical, well-defined, and plausible incorrect responses by chatbots like LLMs. These counterfactual outputs are often articulated with confidence.
Algorithm Life Cycle: The entire process of an AI model’s existence, including diverse data collection, development, trials, and validation, which should incorporate input from varied backgrounds to ensure equity.
Autonomous AI: AI-based tools capable of analyzing data and providing diagnostic decisions without the need for human interpretation. An example is a system that diagnoses diabetic retinopathy directly from retinal images.
Explainable AI: A concept in which the decisions made by AI can be justified and made intelligible to users, driven by the need to ensure physician-patient trust and safety. It takes two forms: inherent and post hoc.
Generative AI (GenAI): A subfield of Artificial Intelligence (AI) that uses large language models (LLMs) to generate realistic images, text, and videos to assist in automating tasks.
Inherent Explainability: A form of explainability pertaining to AI models with clear input-output data where the relationships between independent and dependent variables can be quantified, such as in a predictive linear regression.
Janus Interface: A technique involving the modification of a Large Language Model (LLM) that allows users to circumvent security measures and potentially access confidential, personally identifiable information.
Large Language Models (LLMs): The underlying technology for genAI systems like ChatGPT, Bard, and Llama. They can efficiently handle complex concepts and provide multi-modal responses to a wide array of inquiries and prompts.
Nonautonomous AI: AI agents, such as ChatGPT and Bard, that are prompted to serve as assistants for surgeons. They are designed to assist, not replace, human decision-makers.
Post Hoc Explainability: A method used for complex AI models that lack simple input-output relationships. It aims to dissect the model’s decision-making process by using a confluence of decision variables to “reverse-engineer” the output.
Responsibility Gap: A situation where it is unclear who would be held liable for ill-informed or harmful clinical decisions augmented by genAI, potentially placing the burden on surgeons who did not develop the technology.
Software-As-A-Medical-Device (SAAMD): The category under which the FDA regulates AI and machine learning technologies. This regulatory strategy focuses on continuous monitoring and evaluation of real-world performance.
Technological Divide: A situation where populational inequities in healthcare are exacerbated by advancements in medical technology, creating disparities in access and outcomes.

ETHICAL AI

Nip Talk: Gen-AI In Medicine

Comprehensive Study Guide

Key Terms

AI Hallucination: The production of logical, well-defined, and plausible incorrect responses by chatbots like LLMs. These counterfactual outputs are often articulated with confidence.
Algorithm Life Cycle: The entire process of an AI model’s existence, including diverse data collection, development, trials, and validation, which should incorporate input from varied backgrounds to ensure equity.
Autonomous AI: AI-based tools capable of analyzing data and providing diagnostic decisions without the need for human interpretation. An example is a system that diagnoses diabetic retinopathy directly from retinal images.
Explainable AI: A concept in which the decisions made by AI can be justified and made intelligible to users, driven by the need to ensure physician-patient trust and safety. It takes two forms: inherent and post hoc.
Generative AI (GenAI): A subfield of Artificial Intelligence (AI) that uses large language models (LLMs) to generate realistic images, text, and videos to assist in automating tasks.
Inherent Explainability: A form of explainability pertaining to AI models with clear input-output data where the relationships between independent and dependent variables can be quantified, such as in a predictive linear regression.
Janus Interface: A technique involving the modification of a Large Language Model (LLM) that allows users to circumvent security measures and potentially access confidential, personally identifiable information.
Large Language Models (LLMs): The underlying technology for genAI systems like ChatGPT, Bard, and Llama. They can efficiently handle complex concepts and provide multi-modal responses to a wide array of inquiries and prompts.
Nonautonomous AI: AI agents, such as ChatGPT and Bard, that are prompted to serve as assistants for surgeons. They are designed to assist, not replace, human decision-makers.
Post Hoc Explainability: A method used for complex AI models that lack simple input-output relationships. It aims to dissect the model’s decision-making process by using a confluence of decision variables to “reverse-engineer” the output.
Responsibility Gap: A situation where it is unclear who would be held liable for ill-informed or harmful clinical decisions augmented by genAI, potentially placing the burden on surgeons who did not develop the technology.
Software-As-A-Medical-Device (SAAMD): The category under which the FDA regulates AI and machine learning technologies. This regulatory strategy focuses on continuous monitoring and evaluation of real-world performance.
Technological Divide: A situation where populational inequities in healthcare are exacerbated by advancements in medical technology, creating disparities in access and outcomes.