Extended Reality Applications in Cardiac Surgery and Interventional Cardiology

DOI: https://doi.org/10.30702/ujcvs/23.31(02)/PP018-5057
Keywords: virtual reality, augmented reality, mixed reality, 3D printing, heart segmentation

Abstract

Extended reality combines the real and digital worlds. This technology has found applications in all fields of medicine, including cardiac surgery and interventional cardiology. The paper describes the application of three types of extended reality, namely virtual, augmented and mixed realities.

The aim. To explain the principles of operation of various types of extended reality using non-medical and medical applications as examples; to analyze the data from specialized publications in the field of cardiac interventions.

Materials. Articles from the Pubmed database.

Results. The article highlights important details of the heart and blood vessels image creation technique with which users operate. Primary data is obtained from imaging modalities like tomography or ultrasound, then it is segmented and processed for the virtual viewing. In virtual reality, three-dimensional (3D) images of the heart defects are analyzed in depth, and virtual manipulations can be performed that simulate the course of the operation. Virtual reality includes printing the heart on a 3D printer with subsequent executions on physical models, both diagnostic dissections and therapeutic surgical or endovascular simulations. In augmented reality, the created image of the internal anatomy of the defect is present near the surgeon, without interfering medical manipulations. In mixed reality, a virtual image is superimposed on the patient’s body, creating a detailed navigation map.

Conclusions. Extended reality application deepens the understanding of anatomy due to stereoscopic visualization of the structure of the heart and blood vessels. Creating a model of a patient’s heart defect and simulating an operation on it shortens the “learning curve”, improves the professional skills of surgeons and cardiologists, and also allows for surgical and endovascular interventions individualization. Planning interventions in cardiac surgery and interventional cardiology with extended reality technologies influences decision-making and reduces the duration of operations.

References

  1. Goo HW, Park SJ, Yoo SJ. Advanced Medical Use of Three-Dimensional Imaging in Congenital Heart Disease: Augmented Reality, Mixed Reality, Virtual Reality, and Three-Dimensional Printing. Korean J Radiol. 2020;21(2):133-145. https://doi.org/10.3348/kjr.2019.0625
  2. Narang A, Hitschrich N, Mor-Avi V, Schreckenberg M, Schummers G, Tiemann K, et al. Virtual Reality Analysis of Three-Dimensional Echocardiographic and Cardiac Computed Tomographic Data Sets. J Am Soc Echocardiogr. 2020;33(11):1306-1315. https://doi.org/10.1016/j.echo.2020.06.018
  3. Zablah JE, Morgan GJ. Innovations in Congenital Interventional Cardiology. Pediatr Clin North Am. 2020;67(5):973-993. https://doi.org/10.1016/j.pcl.2020.06.0124. Kelly JW, Cherep LA, Siegel ZD. Perceived Space in the HTC Vive. ACM Trans Appl Percept. 2017;15(1):2. https://doi.org/10.1145/3106155
  4. Laato S, Hyrynsalmi S, Rauti S, Islam AKMN, Laine TH. Location-based Games as Exergames - From Pokémon To The Wizarding World. International Journal of Serious Games. 2020;7(1):79-95. https://doi.org/10.17083/ijsg.v7i1.337
  5. Gitlin JMG. This tech replaces a car’s instrument panel with a holographic display [Internet]. Ars Technica. 2021 [cited 2023 May 13]. Available from: https://arstechnica.com/cars/2021/01/ti-shows-a-holographic-instrument-dis-play-for-car-windshields-at-ces/
  6. Statt N. Microsoft’s HoloLens explained: How it works and why it’s different [Internet]. CNET. 2015 [cited 23 May 13]. Available from: https://www.cnet.com/tech/computing/microsoft-hololens-explained-how-it-works-and-why-its-different/
  7. Valverde I, Gomez-Ciriza G, Hussain T, Suarez-Mejias C, Velasco-Forte MN, Byrne N, et al. Three-dimensional printed models for surgical planning of complex congenital heart defects: an international multicentre study. Eur J Cardiothorac Surg. 2017;52(6):1139-1148. https://doi.org/10.1093/ejcts/ezx208
  8. Gómez-Ciriza G, Gómez-Cía T, Rivas-González JA, Velasco Forte MN, Valverde I. Affordable Three-Dimensional Printed Heart Models. Front Cardiovasc Med. 2021;8:642011. https://doi.org/10.3389/fcvm.2021.642011
  9. Tiwari N, Ramamurthy HR, Kumar V, Kumar A, Dhanalakshmi B, Kumar G. The role of three-dimensional printed cardiac models in the management of complex congenital heart diseases. Med J Armed Forces India. 2021;77(3):322-330. https://doi.org/10.1016/j.mjafi.2021.01.019
  10. Mena KA, Urbain KP, Fahey KM, Bramlet MT. Exploration of time sequential, patient specific 3D heart unlocks clinical understanding. 3D Print Med. 2018;4(1):15. https://doi.org/10.1186/s41205-018-0034-7
  11. 3d.nih.gov [Internet]. U.S. Department of Health and Human Services - National Institutes of Health; 2021 [cited 23 May 23]. Available from: https://3dprint.nih.gov/discover/congenital-heart-disease
  12. Ayerbe VMC, Morales MLV, Rojas CJL, Cortés MLA. Visualization of 3D Models Through Virtual Reality in the Planning of Congenital Cardiothoracic Anomalies Correction: An Initial Experience. World J Pediatr Congenit Heart Surg. 2020;11(5):627-629. https://doi.org/10.1177/2150135120923618
  13. Mendez A, Hussain T, Hosseinpour AR, Valverde I. Virtual reality for preoperative planning in large ventricular septal defects. Eur Heart J. 2019;40(13):1092. https://doi.org/10.1093/eurheartj/ehy685
  14. Pisowodzka IK, Gründeman PF, Meijboom F, van Aarnhem G, Meijer R, Cramer MJ, et al. Added Value of Interactive 3-D Stereo Vision Echocardiography in the Heart Valve Team: A Post Hoc Analysis for Optimal Decision Making in Patients With Mitral Valve Regurgitation. Innovations (Phila). 2020;15(1):36-42. https://doi.org/10.1177/1556984519887973
  15. Nanchahal S, Arjomandi Rad A, Naruka V, Chacko J, Liu G, Afoke J, et al. Mitral valve surgery assisted by virtual and augmented reality: Cardiac surgery at the front of innovation. Perfusion. 2022 Oct 30:2676591221137480. Epub 2022 Oct 30. https://doi.org/10.1177/02676591221137480
  16. Szugye NA, Lorts A, Zafar F, Taylor M, Morales DLS, Moore RA. Can virtual heart transplantation via 3-dimensional imaging increase the maximum acceptable donor size? J Heart Lung Transplant. 2019;38(3):331-333. https://doi.org/10.1016/j.healun.2018.12.014
  17. Neugebauer M, Tautz L, Hüllebrand M, Sündermann S, Degener F, Goubergrits L, et al. Virtual downsizing for decision support in mitral valve repair. Int J Comput Assist Radiol Surg. 2019;14(2):357-371. https://doi.org/10.1007/s11548-018-1868-6
  18. Tandon A, Burkhardt BEU, Batsis M, Zellers TM, Velasco Forte MN, Valverde I, et al. Sinus Venosus Defects: Anatomic Variants and Transcatheter Closure Feasibility Using Virtual Reality Planning. JACC Cardiovasc Imaging. 2019;12(5):921-924. https://doi.org/10.1016/j.jcmg.2018.10.013
  19. Nam HH, Herz C, Lasso A, Drouin S, Posada A, Morray B, et al. Simulation of Transcatheter Atrial and Ventricular Septal Defect Device Closure Within Three-Dimensional Echocardiography-Derived Heart Models On Screen and in Virtual Reality. J Am Soc Echocardiogr. 2020;33(5):641-644.e2. https://doi.org/10.1016/j.echo.2020.01.011
  20. Jolley MA, Lasso A, Nam HH, Dinh PV, Scanlan AB, Nguyen AV, et al. Toward predictive modeling of catheter-based pulmonary valve replacement into native right ventricular outflow tracts. Catheter Cardiovasc Interv. 2019;93(3):E143-E152. https://doi.org/10.1002/ccd.27962
  21. Brun H, Bugge RAB, Suther LKR, Birkeland S, Kumar R, Pelanis E, et al. Mixed reality holograms for heart surgery planning: first user experience in congenital heart disease. Eur Heart J Cardiovasc Imaging. 2019;20(8):883-888. https://doi.org/10.1093/ehjci/jey184
  22. Gehrsitz P, Rompel O, Schöber M, Cesnjevar R, Purbojo A, Uder M, et al. Cinematic Rendering in Mixed-Reality Holograms: A New 3D Preoperative Planning Tool in Pediatric Heart Surgery. Front Cardiovasc Med. 2021;8:633611. https://doi.org/10.3389/fcvm.2021.633611
  23. Ye W, Zhang X, Li T, Luo C, Yang L. Mixed-reality hologram for diagnosis and surgical planning of double outlet of the right ventricle: a pilot study. Clin Radiol. 2021;76(3):237. e1-237.e7. https://doi.org/10.1016/j.crad.2020.10.017
  24. Cen J, Liufu R, Wen S, Qiu H, Liu X, Chen X, et al. Three-Dimensional Printing, Virtual Reality and Mixed Reality for Pulmonary Atresia: Early Surgical Outcomes Evaluation. Heart Lung Circ. 2021;30(2):296-302. https://doi.org/10.1016/j.hlc.2020.03.017
  25. Yoo S, Saprungruang A, Lam CZ, Anderson RH. Disharmonious Ventricular Relationship and Topology for the Given Atrioventricular Connections. Contemporary Diagnostic Approach Using 3D Modeling and Printing. Congenital Heart Disease. 2022;17(5):495-504. https://doi.org/10.32604/chd.2022.021155
  26. Biglino G, Capelli C, Leaver LK, Schievano S, Taylor AM, Wray J. Involving patients, families and medical staff in the evaluation of 3D printing models of congenital heart disease. Commun Med. 2015;12(2-3):157-169. https://doi.org/10.1558/cam.28455
  27. Jones TW, Seckeler MD. Use of 3D models of vascular rings and slings to improve resident education. Congenit Heart Dis. 2017;12(5):578-582. https://doi.org/10.1111/chd.12486
  28. Hussein N, Honjo O, Barron DJ, Haller C, Coles JG, Yoo SJ. The Incorporation of Hands-On Surgical Training in a Congenital Heart Surgery Training Curriculum. Ann Thorac Surg. 2021;112(5):1672-1680. https://doi.org/10.1016/j.athoracsur.2020.11.018
  29. Yamada T, Osaka M, Uchimuro T, Yoon R, Morikawa T, Sugimoto M, et al. Three-Dimensional Printing of Life-Like Models for Simulation and Training of Minimally Invasive Cardiac Surgery. Innovations (Phila). 2017;12(6):459-465. https://doi.org/10.1097/IMI.0000000000000423
  30. Valdis M, Chu MW, Schlachta CM, Kiaii B. Validation of a Novel Virtual Reality Training Curriculum for Robotic Cardiac Surgery: A Randomized Trial. Innovations (Phila). 2015;10(6):383-388. https://doi.org/10.1097/IMI.0000000000000222
  31. Rudarakanchana N, van Herzeele I, Desender L, Cheshire NJ. Virtual reality simulation for the optimization of endovascular procedures: current perspectives. Vasc Health Risk Manag. 2015;11:195-202. https://doi.org/10.2147/VHRM.S46194
  32. Jensen UJ, Jensen J, Ahlberg G, Tornvall P. Virtual reality training in coronary angiography and its transfer effect to real-life catheterisation lab. EuroIntervention. 2016;11(13):1503-1510. https://doi.org/10.4244/EIJY15M06_05
  33. Mafeld S, Nesbitt C, McCaslin J, Bagnall A, Davey P, Bose P, et al. Three-dimensional (3D) printed endovascular simulation models: a feasibility study. Ann Transl Med. 2017 Feb;5(3):42. https://doi.org/10.21037/atm.2017.01.16
  34. Levin D, Mackensen GB, Reisman M, McCabe JM, Dvir D, Ripley B. 3D Printing Applications for Transcatheter Aortic Valve Replacement. Curr Cardiol Rep. 2020 Feb;22(4):23. https://doi.org/10.1007/s11886-020-1276-8
  35. Valverde I, Gomez G, Coserria JF, Suarez-Mejias C, Uribe S, Sotelo J, et al. 3D printed models for planning endovascular stenting in transverse aortic arch hypoplasia. Catheter Cardiovasc Interv. 2015;85(6):1006-1012. https://doi.org/10.1002/ccd.25810
  36. Li H, Qingyao, Bingshen, Shu M, Lizhong, Wang X, et al. Application of 3D printing technology to left atrial appendage occlusion. Int J Cardiol. 2017;231:258-263. https://doi.org/10.1016/j.ijcard.2017.01.031
  37. Jalal Z, Seguela PE, Iriart X, Roubertie F, Quessard A, Kreitmann B, et al. Hybrid Melody Valve Implantation in Mitral Position in a Child: Usefulness of a 3-Dimensional Printed Model for Preprocedural Planning. Can J Cardiol. 2018;34(6):812.e5-812.e7. https://doi.org/10.1016/j.cjca.2018.02.011
  38. Witowski J, Darocha S, Kownacki Ł, Pietrasik A, Pietura R, Banaszkiewicz M, et al. Augmented reality and three-dimensional printing in percutaneous interventions on pulmonary arteries. Quant Imaging Med Surg. 2019;9(1):23-29. https://doi.org/10.21037/qims.2018.09.08
  39. Giugno L, Faccini A, Carminati M. Percutaneous Pulmonary Valve Implantation. Korean Circ J. 2020;50(4):302-316. https://doi.org/10.4070/kcj.2019.0291
  40. Opolski MP, Schumacher SP, Verouden NJW, van Diemen PA, Borucki BA, Sprengers R, et al. On-Site Computed Tomography Versus Angiography Alone to Guide Coronary Stent Implantation: A Prospective Randomized Study. J Invasive Cardiol. 2020;32(11):E268-E276.
  41. Kasprzak JD, Pawlowski J, Peruga JZ, Kaminski J, Lipiec P. First-in-man experience with real-time holographic mixed reality display of three-dimensional echocardiography during structural intervention: balloon mitral commissurotomy. Eur Heart J. 2020;41(6):801. https://doi.org/10.1093/eurheartj/ehz127
  42. Zhu H, Li Y, Wang C, Li QY, Xu ZY, Li X, et al. A first attempt of inferior vena cava filter successfully guided by a mixed-reality system: a case report. J Geriatr Cardiol. 2019;16(7):575-577. https://doi.org/10.11909/j.issn.1671-5411.2019.07.008
  43. Zhu H, Li Y, Gong G, Zhao MX, Liu L, Yao SY, et al. A world’s first attempt of mixed-reality system guided inferior vena cava filter implantation under remote guidance of 5G communication. J Geriatr Cardiol. 2021;18(3):233-237. https://doi.org/10.11909/j.issn.1671-5411.2021.03.008
  44. Belhaj Soulami R, Verhoye JP, Nguyen Duc H, Castro M, Auffret V, Anselmi A, et al. Computer-Assisted Transcatheter Heart Valve Implantation in Valve-in-Valve Procedures. Innovations (Phila). 2016;11(3):193-200. https://doi.org/10.1097/IMI.0000000000000259
  45. Minderhoud SCS, van der Stelt F, Molenschot MMC, Koster MS, Krings GJ, Breur JMPJ. Dramatic Dose Reduction in Three-Dimensional Rotational Angiography After Implementation of a Simple Dose Reduction Protocol. Pediatr Cardiol. 2018;39(8):1635-1641. https://doi.org/10.1007/s00246-018-1943-3
  46. Balzer J, Zeus T, Hellhammer K, Veulemans V, Eschenhagen S, Kehmeier E, et al. Initial clinical experience using the EchoNavigator®-system during structural heart disease interventions. World J Cardiol. 2015;7(9):562-570. https://doi.org/10.4330/wjc.v7.i9.562
Published
2023-06-26
How to Cite
1.
Petrov VF, Pankiv MV. Extended Reality Applications in Cardiac Surgery and Interventional Cardiology. ujcvs [Internet]. 2023Jun.26 [cited 2024Dec.21];31(2):50-7. Available from: https://cvs.org.ua/index.php/ujcvs/article/view/564
Issue
Ukrainian Journal of Cardiovascular Surgery Vol 31 No 2 2023
Section
GENERAL ISSUES OF TREATMENT OF PATIENTS WITH CARDIOVASCULAR PATHOLOGY