The digital transformation of surgery

Nov 13, 2023

npj Digital Medicine volume 6, Article number: 103 (2023) Cite this article

402 Accesses

6 Altmetric

Metrics details

Rapid advances in digital technology and artificial intelligence in recent years have already begun to transform many industries, and are beginning to make headway into healthcare. There is tremendous potential for new digital technologies to improve the care of surgical patients. In this piece, we highlight work being done to advance surgical care using machine learning, computer vision, wearable devices, remote patient monitoring, and virtual and augmented reality. We describe ways these technologies can be used to improve the practice of surgery, and discuss opportunities and challenges to their widespread adoption and use in operating rooms and at the bedside.

Innovations in digital technology have begun to transform the practice of medicine. From FDA approved artificial intelligence systems for endoscopy1 to the growing use of wearable biosensors in clinical trials2, digital technologies are already being applied within various fields of medicine. Historically the field of surgery has justifiably been relatively cautious with deployment of new and potentially disruptive technologies that have not been extensively studied, given the possibility of direct and immediate patient harm3; surgery's experience with digital technologies so far is no different. However, with the tremendous potential for new digital technologies to improve surgical care, we believe its incorporation into daily surgical practice is inevitable. Indeed fields such as gastroenterology and radiology are already incorporating digital technologies in their practice4,5. In this npj Digital Medicine special collection, we highlight work being done in several areas of digital technology—machine learning, computer vision, wearable devices, remote patient monitoring, and virtual and augmented reality—all of which we believe are poised to transform surgery in the coming years. In this piece, we describe opportunities and challenges to these technologies’ adoption.

Machine learning models have the potential to learn complex, nuanced relationships between enormous numbers of clinical variables, including multimodal data, in ways that traditional statistical risk calculators cannot. This advantage may soon be leveraged to better predict the clinical trajectory of patients and help surgeons make more personalized patient care decisions.

Specifically, AI enabled decision support systems have the potential to predict surgical outcomes. Surgical outcome prediction is important for patient care both preoperatively, in deciding which patients would be candidates for benefit with surgical intervention, as well as postoperatively, in predicting risk of complications. Machine learning algorithms can be applied to both aspects to assist outcome prediction. Mobile applications with these kinds of algorithms have been developed and piloted at multiple large academic institutions6,7,8,9.

Highlighted in this collection, a review of machine learning models that learned from clinical data in vascular surgery found that several models perform better than existing clinical prediction tools, clinicians, and traditional regression models10. The study further finds that as machine learning tools are increasingly applied to vascular surgery, model performance continues to improve with newer studies. These findings emphasize that machine learning may one day serve as an important augmentation for analysis, diagnosis, and outcome prediction in surgery.

Computer vision—the application of machine learning algorithms for visual data analysis—holds tremendous potential to impact clinical care wherever images or video data are involved. In this special collection we focus mainly on applications of computer vision for analysis of intraoperative data. Rapid advancements in the field of computer vision, including the increasing use of deep neural networks, have led to the development of algorithms that can accurately identify clinically important aspects of intraoperative video data11. Among the most exciting potential applications of real-time intraoperative computer vision analysis to support safer surgery is providing an augmented reality experience for surgeons to aid in intraoperative decision-making12. Many existing published computer vision models have demonstrated the ability to assess operative complexity, assist with decision-making during minimally invasive procedures, assess the technical skill of surgeons in an automated and scalable fashion, provide intraoperative feedback, assess OR team dynamics, and even predict postoperative outcomes based on intraoperative events13,14,15. When combined with these technologies, virtual reality can be used as a training ground for operative skills assessment and education16. While the incorporation of these tools in regular surgical practice is not yet a reality, potential use cases will surely continue to grow as research in this field continues.

Wearable devices and other personal technologies that collect and transmit physiologic signals and patient reported variables in real time are another important technology with the potential to improve surgery. The use of wearable devices for remote patient monitoring, which enables evaluation of patients beyond traditional in-person encounters with the healthcare system, holds great opportunity for the improvement of postoperative care by enabling remote assessment of surgical wounds, functional status, pain control, and detecting clinical deterioration early. There is emerging evidence that this form of data capture can potentially reduce complication rates17. In our collection, Mori et al. use a digital health platform to better understand the nuances of post operative recovery in the home for cardiac surgery patients, far beyond just clinical events captured during postoperative ED visits or clinic visits18.

In addition to understanding the postoperative course overall, recent work shows the potential for remote patient monitoring changing the way postoperative care is delivered. A randomized control trial finds that postoperative surveillance of surgical site infection using a smartphone is just as effective as in person follow-up assessment19, suggesting that the use of wearable devices for remote patient wound monitoring may be safe, efficient, and reduce health service utilization. Another study shows that the use of a non-contact video camera to monitor vital signs can be used to track early signs of physiological deterioration, to function as a less disruptive, more efficient, early warning system20. These new capabilities may enable an eventual shift in the nucleus of post-surgical care from the hospital back to the home; some large academic medical centers have already established successful digital technology-enabled surgical home hospital programs that allow patients to be safely recover from surgery in their own home21.

While there are many emerging digital technologies that hold great promise for surgery, there are also important cautions and considerations for their use.

One important concern is equity. There are disparities in surgical outcomes for patients of racial minorities as well as lower socioeconomic status that persist until today. Just one example is in spine surgery, where the risk of postoperative complications has been estimated to be as much as 61% higher for Black patients22. Without a critical assessment of digital technologies from an equity lens, these disparities may continue to be perpetuated. Halamka et al. highlight the importance of using existing bias detection tools to detect irregularities in datasets and algorithms before deploying them in surgical practice23.

Another important concern is information security. Operating rooms are growing in terms of technological sophistication and the use of connected devices. Surgical robots for example, are now being widely used at growing rates around the world and increasingly rely on constant connectivity to deliver real-time insights and support. At the same time, the number of cyberattacks on hospitals has also been significantly increasing24. The rapid adoption of sophisticated connected technologies in operating rooms makes them uniquely susceptible to targeted attacks that can cause patient harm. An important contribution in this collection by Gordon et al. explains how health systems can mitigate the risks of cybersecurity attacks in operating rooms, including minimizing real-time connectivity during surgical procedures to only what is necessary, collaborating with vendors on developing safety measures, and establishing downtime procedures25.

A third important consideration is determining how these technologies will be regulated to ensure they are safe and effective. One unique element of AI-enabled technologies is their ability to adapt over time as they are introduced to new data. This may cause unanticipated shifts in performance and ultimately a very different tool from the one that was originally approved. The US Food and Drug Administration (FDA) is in the process of determining how to best regulate these tools. They developed a proposed regulatory framework in 201926 and have since been refining it; most recently in April 2023 they put forth a new draft guidance that recommends continuous product monitoring throughout the product lifecycle with real-world data and allows for small adjustments in performance over time27.

Among the most important barriers to widespread adoption of these technologies in surgery is limited evidence of their clinical impact28. For example, few studies have shown any improvement in care when AI-enabled decision support tools are used29. Some important barriers to clinical impact include translating ML model outputs into clinically actionable insights and incorporating them into existing workflows and clinical decision making processes15,21,30. Additionally, these new technologies must be built and rigorously evaluated in a way that is reproducible, transparent, and interpretable to users to earn their trust30,31. Access to larger and more diverse datasets may help address some of these issues12.

The number of new digital technologies with the potential to improve surgical care is rapidly growing. There are many use cases and technologies not covered in this special collection. Generative AI models, such as GPT-4 and DALL-E 2, are only just beginning to be explored in the healthcare literature, yet their tremendous abilities have sparked widespread interest in exploring potential use cases32,33. As another example, the use of ambient sensing monitors in operating rooms may help automate quality control processes, prevent medical errors, and improve a surgical team's overall perioperative situational awareness34.

Research in this field should focus on bridging the gap from in silico performance to real-world clinical benefit of new digital technologies. This will involve building out the implementation science of these tools, evaluating them with real-world evidence, and continuing to explore new applications35. Health systems will need to begin investing in the expertise and IT infrastructure necessary to deploy and maintain these tools36. The augmentation of surgery with these emerging digital technologies is necessary for progress toward more efficient and effective patient care.

Raza, M. M., Venkatesh, K. P., Diao, J. A. & Kvedar, J. C. Defining digital surgery for the future. NPJ Digit Med. 5, 155 (2022).

Article PubMed PubMed Central Google Scholar

Vijayan, V., Connolly, J. P., Condell, J., McKelvey, N. & Gardiner, P. Review of wearable devices and data collection considerations for connected health. Sensors 21, 1–31 (2021).

Article Google Scholar

Riskin, D. J., Longaker, M. T., Gertner, M. & Krummel, T. M. Innovation in surgery: a historical perspective. Ann. Surg. 244, 686–693 (2006).

Article PubMed PubMed Central Google Scholar

Biffi, C. et al. A novel AI device for real-time optical characterization of colorectal polyps. npj Digital Medicine vol. 5 Preprint at https://doi.org/10.1038/s41746-022-00633-6 (2022).

Kim, H.-E. et al. Changes in cancer detection and false-positive recall in mammography using artificial intelligence: a retrospective, multireader study. Lancet Digit Health 2, e138–e148 (2020).

Article PubMed Google Scholar

Maurer, L. R. et al. Trauma outcome predictor: an artificial intelligence interactive smartphone tool to predict outcomes in trauma patients. J. Trauma Acute Care Surg. 91, 93–99 (2021).

Article PubMed Google Scholar

Bertsimas, D., Dunn, J., Velmahos, G. C. & Kaafarani, H. M. A. Surgical risk is not linear: derivation and validation of a novel, user-friendly, and machine-learning-based predictive optimal trees in emergency surgery risk (POTTER) calculator. Ann. Surg. 268, 574–583 (2018).

Article PubMed Google Scholar

Corey, K. M. et al. Development and validation of machine learning models to identify high-risk surgical patients using automatically curated electronic health record data (Pythia): a retrospective, single-site study. PLoS Med. 15, e1002701 (2018).

Article PubMed PubMed Central Google Scholar

Ren, Y. et al. Performance of a machine learning algorithm using electronic health record data to predict postoperative complications and report on a mobile platform. JAMA Network Open vol. 5 e2211973 Preprint at https://doi.org/10.1001/jamanetworkopen.2022.11973 (2022).

Li, B. et al. Machine learning in vascular surgery: a systematic review and critical appraisal. NPJ Digit Med. 5, 7 (2022).

Article PubMed PubMed Central Google Scholar

Ward, T. M. et al. Computer vision in surgery. Surgery 169, 1253–1256 (2021).

Article PubMed Google Scholar

Colborn, K., Brat, G. & Callcut, R. Predictive Analytics and Artificial Intelligence in Surgery—Opportunities and Risks. JAMA Surg. https://doi.org/10.1001/jamasurg.2022.5444 (2022).

Mascagni, P. et al. Computer vision in surgery: from potential to clinical value. NPJ Digit Med. 5, 163 (2022).

Article PubMed PubMed Central Google Scholar

Lam, K. et al. Machine learning for technical skill assessment in surgery: a systematic review. NPJ Digit. Med. 5, 24 (2022).

Article PubMed PubMed Central Google Scholar

Ma, R. et al. Surgical gestures as a method to quantify surgical performance and predict patient outcomes. NPJ Digit Med. 5, 187 (2022).

Article PubMed PubMed Central Google Scholar

Yilmaz, R. et al. Continuous monitoring of surgical bimanual expertise using deep neural networks in virtual reality simulation. npj Digital Medicine vol. 5 Preprint at https://doi.org/10.1038/s41746-022-00596-8 (2022).

Knight, S. R. et al. Mobile devices and wearable technology for measuring patient outcomes after surgery: a systematic review. NPJ Digit Med. 4, 157 (2021).

Article PubMed PubMed Central Google Scholar

Mori, M., Dhruva, S. S., Geirsson, A. & Krumholz, H. M. Characterization of multi-domain postoperative recovery trajectories after cardiac surgery using a digital platform. npj Digital Med. 5, 1–8 (2022).

Article Google Scholar

McLean, K. A. et al. Remote diagnosis of surgical-site infection using a mobile digital intervention: a randomised controlled trial in emergency surgery patients. NPJ Digit Med. 4, 160 (2021).

Article PubMed PubMed Central Google Scholar

Jorge, J. et al. Non-contact physiological monitoring of post-operative patients in the intensive care unit. NPJ Digit Med. 5, 4 (2022).

Article PubMed PubMed Central Google Scholar

Pathak, K., Marwaha, J. S. & Tsai, T. C. The role of digital technology in surgical home hospital programs. npj Digital Med. 6, 1–6 (2023).

Article Google Scholar

Sanford, Z. et al. Racial disparities in surgical outcomes after spine surgery: an ACS-NSQIP analysis. Global Spine J. 9, 583–590 (2019).

Article PubMed Google Scholar

Halamka, J., Bydon, M., Cerrato, P. & Bhagra, A. Addressing racial disparities in surgical care with machine learning. NPJ Digit Med. 5, 152 (2022).

Article PubMed PubMed Central Google Scholar

Niki, O. ’brien, Saira, G., Arvind, S. & Mike, D. Cyber-attacks are a permanent and substantial threat to health systems: education must reflect that. Digit Health 8, 20552076221104665 (2022).

PubMed PubMed Central Google Scholar

Gordon, W. J., Ikoma, N., Lyu, H., Jackson, G. P. & Landman, A. Protecting procedural care—cybersecurity considerations for robotic surgery. npj Dig. Med. 5, 1–3 (2022).

Google Scholar

US FDA. Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) - Discussion Paper and Request for Feedback. https://www.fda.gov/files/medical%20devices/published/US-FDA-Artificial-Intelligence-and-Machine-Learning-Discussion-Paper.pdf (2019).

Center for Devices & Radiological Health. Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence/Machine Learning (AI/ML)-Enabled Device Software Functions. U.S. Food and Drug Administration https://www.fda.gov/regulatory-information/search-fda-guidance-documents/marketing-submission-recommendations-predetermined-change-control-plan-artificial.

Loftus, T. J. et al. Artificial Intelligence-enabled Decision Support in Surgery: State-of-the-art and Future Directions. Ann. Surg. https://doi.org/10.1097/SLA.0000000000005853.

Zhou, Q., Chen, Z.-H., Cao, Y.-H. & Peng, S. Clinical impact and quality of randomized controlled trials involving interventions evaluating artificial intelligence prediction tools: a systematic review. npj Digital Medicine 4, 1–12 (2021).

Article Google Scholar

Lee, C. K., Samad, M., Hofer, I., Cannesson, M. & Baldi, P. Development and validation of an interpretable neural network for prediction of postoperative in-hospital mortality. NPJ Digit Med. 4, 8 (2021).

Article PubMed PubMed Central Google Scholar

Marwaha, J. S. et al. Appraising the quality of development and reporting in surgical prediction models. JAMA Surg. https://doi.org/10.1001/jamasurg.2022.4488 (2022).

Article PubMed Google Scholar

Sarraju, A. et al. Appropriateness of Cardiovascular Disease Prevention Recommendations Obtained From a Popular Online Chat-Based Artificial Intelligence Model. JAMA https://doi.org/10.1001/jama.2023.1044 (2023).

ChatGPT: Will It Transform the World of Health Care? (2023).

Haque, A., Milstein, A. & Fei-Fei, L. Illuminating the dark spaces of healthcare with ambient intelligence. Nature 585, 193–202 (2020).

Article CAS PubMed Google Scholar

Marwaha, J. S. & Kvedar, J. C. Crossing the chasm from model performance to clinical impact: the need to improve implementation and evaluation of AI. npj Digital Med. 5, 1–2 (2022).

Article Google Scholar

Marwaha, J. S., Landman, A. B., Brat, G. A., Dunn, T. & Gordon, W. J. Deploying digital health tools within large, complex health systems: key considerations for adoption and implementation. NPJ Digital Med. 5, 13 (2022).

Article Google Scholar

Download references

J.S.M. is supported by a grant from the National Library of Medicine/National Institutes of Health (T15LM007092-30) and the Biomedical Informatics and Data Science Research Training (BIRT) Program of Harvard University.

Beth Israel Deaconess Medical Center, Boston, MA, USA

Jayson S. Marwaha

Harvard Medical School, Boston, MA, USA

Jayson S. Marwaha, Marium M. Raza & Joseph C. Kvedar

Mass General Brigham, Boston, MA, USA

Joseph C. Kvedar

You can also search for this author in PubMed Google Scholar

You can also search for this author in PubMed Google Scholar

You can also search for this author in PubMed Google Scholar

Initial draft was written by J.S.M. and M.M.R.; edited by J.C.K. All authors approved the final draft.

Correspondence to Jayson S. Marwaha.

J.C.K. is editor-in-chief of NPJ Digital Medicine. J.S.M. and M.M.R. have no competing interests to declare.

This submission is authored by the guest editors of the "emerging digital technologies in surgery" special collection and is meant to introduce the collection.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

Marwaha, J.S., Raza, M.M. & Kvedar, J.C. The digital transformation of surgery. npj Digit. Med. 6, 103 (2023). https://doi.org/10.1038/s41746-023-00846-3

Download citation

Received: 25 February 2023

Accepted: 15 May 2023

Published: 31 May 2023

DOI: https://doi.org/10.1038/s41746-023-00846-3

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative