Mixed Reality and Augmented Reality in the health sector

What until recently seemed to be a science-fiction medical scenario, today is reality. Indeed, augmented reality and virtual reality! The first is the superimposition of three-dimensional digital information and content on the real environment through the use of mobile devices such as the latest generation wearables. The second one projects the user in an alternative environment.

First field of application

The first field of application of this technology was technical assistance. GE Healthcare, the medical division of General Electric has developed the Drive system (Digital Reality and Intelligent Virtual Experience). This is a 4.0 technical assistance service which, through the AR, creates a holistic communication system between the GE Healthcare technicians and the staff present on site, which is guided in repairs and maintenance of the equipment. This allows you to provide effective, timely remote support.

Surgical Theathre

The Surgical Theathre, a US giant in the Healthcare sector, offers patients an all-round immersive experience. Wearing VR visors, the patient walks within his anatomy, between the vessels and his own illness, acquiring further knowledge and a greater awareness of therapeutic interventions.

This tool is useful for neurosurgeons for the prior evaluation of the case, for the study of anatomy, for the determination of the best procedure to follow, for the reproduction of different surgical procedures and for the communication to patients of the condition under examination.


The RealView Imaging, an Israeli-based start-up has designed Holoscope, a holographic system that can be used during minimally invasive procedures, allowing physicians to manipulate 3D images in real time. Designed to be used in cardiology, with the touch of a hand, the doctor can rotate the holographic image 360 ​​degrees, can virtually open and display its internal structure, mark an area of ​​clinical significance or mark two points for accurately measure anatomical structures.


Another Israeli company, Augmedics has designed Vizor, a viewer that allows information to be superimposed on the patient’s medical record on the operating field.

In doing so it does not take your eyes off the patient to observe the monitors and cut down the time required for consultations.

Hololens to reconstruct limbs

At the Imperial College University Hospital in London , Hololens are used to help surgeons rebuild traumatized limbs.

Through the TAC and other diagnostic tests, the software elaborates the 3D model of the limb in question. We are not just talking about the outline, but the whole internal structure, and in particular the blood vessels. The surgeon, wearing the Hololens perfectly superimposes the 3D model on the real limb having 1: 1 scale elements to trace the points in which to intervene to intercept the ideal vessels for vascularization of the flap to be implanted.

VerimaCT 1.0

WitHECA, a special division of Witapp that develops products for the Healthcare sector, has created software to be applied to viewers for Mixed Reality. Let’s talk about VerimaCT 1.0 which automatically converts the TC and resonance data from the dicom format in 3D format files. The end result is a hologram of the region concerned that can be rotated or enlarged and approached. This allows obtaining a precise perception of depth of the anatomical region of the patient. This innovative diagnostic tool is helpful for radiologists and surgeons. In addition to reducing the margins of error in diagnosis and interventions, the time needed to refer back is also reduced. Greater timeliness of intervention is achieved, particularly in the event of emergencies.

The software is currently in use at the Careggi University Hospital in Florence.