Use of extended realities in interventional cardiology: mixed reality for TAVI procedure

To the Editor,

A novel aspect of medical imaging visualization are the so-called extended realities, a term that includes a plethora of very different technologies such as virtual, augmented, and mixed reality. The latter being the most recent one of all. The ultimate feature of mixed reality headsets is their capacity to perceive the real world while mixing virtual models to complement the sources of information traditionally available. In principle, there are multiple possible applications to the medical field being particularly interesting their integration into surgical and interventional procedures. Currently, the main limitation preventing their clinical applicability is that no commercial applications remain available in the market for users. Also that, for every specific new case, specific solutions need to be developed.

In the context of the «3D Augmented Reality Cath Lab» research project (the HAMMOND project) only 1 preliminary clinical experience integrating the mixed reality HoloLens 2 headset (Microsoft, United States) has taken place (figure 1) in the percutaneous coronary intervention setting. Prior to the research ethics committee approval (CASVE-PI-GR-20-2001) a mixed reality application was developed for cardiac catheterization care that was tried in 9 patients treated with transcatheter aortic valve implantation (TAVI).

Figure 1. HoloLens extended mixed reality headset 2.

The descriptive results of this early experience with mixed reality in the different stages of TAVI procedures—shown on figure 1 and video 1 of the supplementary data—follow next:

• – Vascular puncture guidance support: holograms with echocardiogram imaging were generated in real time (figure 2A). Therefore, through the creation of large virtual «screens» with better ergonomics the operator can visualize both his hands and the imaging support simultaneously. The fine resolution and little latency of the system used allowed us to gain ultrasound-guided arterial access as seen on the HoloLens 2 headset in 5 consecutive cases. Room for improvement has been identified and is in the pipeline right now: the integration of the ultrasound image acquired with the ultrasound probe in such a way that the holograms would be visualized in the «real» position towards the inside of the patient (figure 2B). Currently, several early experiences have been reported with this type of integration1 like hologram-guided punctures mixed with computed tomography (CT) images. The use of holograms from CT generated organs overlapping the patient’s real anatomy has been reported allowing us to simplify complex vascular procedures.2
• – Interaction with CT generated images in conditions of asepsis: the capacity of augmented reality headsets to be used via voice commands or the user’s own hands (as seen on video 1 of the supplementary data) allows the operator to check different sources of information without losing sterility or interfering with the procedure. Figure 2C and video 1 of the supplementary data exemplify this application with the intraoperative examination of the patient’s CT scan of using the 3mensio Structural Heart software (Pie medical imaging, The Netherlands).
• – Remote procedural supervision: remote mentoring has proven an extremely useful imaging modality during the COVID-19 pandemic.3,4 In our own single experience with 9 cases including 4 transcatheter aortic valve implantation procedures, 3 bicaval valve implantation procedures for tricuspid regurgitation, and 2 chronic total coronary occlusions, the images and the sound captured by the HoloLens 2 headset can be remotely transmitted to the expert in real time. Therefore, he can not only see the x-ray and the ultrasound images as it is the case with conventional remote mentoring (teleconference), but also the operator’s point of view. Also, the mentor can control the information available through the virtual windows that the operator can see life, for instance, the CT working station. Alternatively, different solutions of telepresence and remote communication have become available like the Mesh software (Microsoft, United States) that allow the operator to visualize the mentor as a real-time hologram (figure 2D).

Figure 2. A: vascular ultrasound integration during puncture. B: schematic representation of simultaneous integration in vascular puncture.1 C: image from the software analysis of the manageable computed tomography scan in conditions of asepsis; D: remote mentoring simulation.

Mixed reality-based technology is giving its first early steps regarding surgical and interventional procedures. Our early experience shows—using scientific methodology—the current actual clinical experience applied to TAVI procedures beyond the future potential benefits of this technology.

The limitations identified when implementing this technology in the interventional cardiology field are basically the complexity associated with the process of integrating hologram-like real-time ultrasound images into the patient’s anatomical structures. Improving this aspect requires better software (by developing applications specifically designed for the use this technology in the interventional cardiology field) and hardware (to facilitate the spatial location of both holograms and patients).

In conclusion, mixed reality can improve the integration of different imaging modalities while performing cardiovascular procedures on our patients. Also, it allows the operator to focus on a single working site, which has the potential of improving the patient’s safety parameters. Thanks to these advantages and the fact that technology has finally reached its maturity at an affordable cost, the continuity of this line of work could be crucial to push interventional cardiology into the future.

FUNDING

The HAMMOND project received a grant from Castille and León Regional Health Authority (GRS 2275/A/2020), and Instituto de Salud Carlos III (DTS21/00158).

AUTHORS’ CONTRIBUTIONS

A. Redondo, and I.J. Amat-Santos contributed substantially to the study design, data curation, analysis or interpretation. C. Baldrón, and J.M. Aguiar contributed to the study design, data curation and interpretation. J.R. González Juanatey, and A. San Román gave their final approval. All the authors performed a critical review of the manuscript intellectual content, and take full responsibility for the accuracy and truthfulness of the study.

CONFLICTS OF INTEREST

None whatsoever.

REFERENCES