As members of the ACS Health Information Technology (HIT) Committee, we appreciate Dr. Elsey's insights and share his concerns regarding the potential risks associated with artificial intelligence (AI) in healthcare. The proper adoption and risk mitigation of these technologies will indeed require a concerted effort from multiple stakeholders, including governmental agencies, academic institutions, and professional societies like the ACS.
Partial engagement or poorly executed approaches could result in missed opportunities and potential harm to both the surgical profession and society. Trustworthiness of AI technologies, which relies on factors such as digital literacy and AI literacy among healthcare professionals and patients alike, is a key area of concern. The fear surrounding AI is reasonable, and this unease can be mitigated by educating surgeons on AI principles and fostering a deeper understanding of these technologies within our community.
Today, AI is often viewed as possessing seemingly limitless potential. AI advocates and skeptics highlight this in their discussions of the risks and benefits that may result from widespread AI adoption. However, expectations surrounding these risks and benefits may be tempered when one considers the significant limitations of current AI technologies.
“Enchanted determinism,” a cognitive bias that arises when a lack of understanding of technical principles leads one to view the technology as magical, has certainly impacted perceptions of AI applications for healthcare.1 Painting AI as limitless stimulates creativity around potential use cases but also risks distracting from immediate problems and issues that plague healthcare AI, such as lack of quality data, inequalities in access to healthcare that bias data, unequal access to datasets, and inappropriate or misleading use of metrics to measure algorithmic performance.2 The “last mile” problem in healthcare AI will be difficult to overcome and will limit meaningful applications of AI unless clinicians, supported by our representative societies such as the ACS, become AI literate.3
Effective and safe implementation of AI technologies into clinical workflows will require tremendous effort among all stakeholders. The most successful translational advances in healthcare AI have combined the expertise of clinicians and computer scientists.4 For these types of collaborations to occur, clinicians must possess the ability to engage in a meaningful dialogue with the developers of AI tools.
Like the wave of digital health technologies that came before it, AI demands its own unique set of competencies, termed AI literacy. Basic skills in areas such as statistics, data science, and computer science are foundational to the ways AI tools function; however, clinicians have traditionally demonstrated low performance in these domains.5-7 Moreover, AI literacy has never been objectively measured in clinicians.
Thus, there is an urgent need for educational efforts aimed at closing this AI literacy gap in clinicians. As Dr. Elsey alludes, there are risks associated with AI use in high-stakes settings. To mitigate these risks, surgeons must know, understand, evaluate, and contribute to the development of AI tools.
The ACS is uniquely suited to support the development of AI literacy initiatives among surgeons. The Journal of the American College of Surgeons has already published a great deal of scientific research on applications of AI in surgical settings, and the ACS has released an online course for surgeons to learn more about AI and data science.8 In addition, the ACS has spent considerable effort gathering surgeon-scientists with expertise in AI to lead initiatives like the HIT Committee, which includes an AI subcommittee.
As AI begins to integrate itself into clinical workflows, the College can help develop AI literacy among surgeons in several ways. First, the ACS can function as an educational body and house materials related to foundational principles in AI, data science, statistics, and other domains. While AI itself is an expansive (and still rapidly expanding) field, applications in surgery are relatively nascent. As surgical AI methodology becomes more common in surgical research, it will become necessary for surgeons to understand the methods used in these papers to offer substantial critique.2,9,10 Moreover, the current limitations of AI models are still widely misunderstood by the general public, perpetuating “enchanted determinism” and perceptions of AI’s applications as limitless. Mitigating this bias will be crucial in the appraisal of AI research and technologies directed toward surgeons.
The ACS also can serve as a forum to engage in a meaningful dialogue about the future development of surgical AI, including concerns surrounding trust, privacy, and equity. As research and commercial interest grow, AI will become more of a part of surgeons’ daily lives. Likewise, there should be ample opportunities for surgeons to share their experiences with AI technologies, whether positive or negative.
In the scientific literature, AI models are often evaluated by certain performance metrics that may or may not reflect the stated goals of their creators.2 However, in clinical settings, models also will be evaluated by way of user experience. Whether surgeons find AI technologies acceptable for their stated uses will be a crucial component of their translational success. As such, user experience considerations should be integrated into the process of model development using insights gained from surgeons.
Finally, the ACS can establish professional standards for the appropriate use of AI applications in surgical settings and communicate this to the public. The landscape surrounding AI is in a state of flux, but the College can continue its role in advocating for our patients by promoting laws and regulations that protect surgeons and patients while enabling the research and development necessary to drive surgical innovation.
