First test of Virtual Surgery Instruction

First test of our Virtual Surgery Instructor

Two senior physicians Dr. Patrick House (Epilepsy and Neurology Department, Evangelisches Krankenhaus Alsterdorf Hamburg) und Dr. Hans-J. von Lücken (Head and Neck Surgery Department, Marienkrankenhaus Hamburg) came together on the 24th of July 2017. They met in apoQlar premises, where they tested the new functions of our Virtual Surgery Instructor.

Presentation

At first, the physicians wanted to test the quality of the 3D CTs and MRI images, which are shown in the HoloLens. Both took a closer look at two different orbit (head) MRIs and dove into the individual layers. The MRI images differed within the layer thickness, 1.1 mm and 0.9 mm. Consequently, an enormous difference in quality was visible. The MRI image with the thinner layers had a much better resolution quality and faster reaction time. This made an impression on the physicians.

ct mri image on head

Functionalities

The next focus lied on the functionalities of the VSI. These split up into two different categories: The ‘Free Mode’ and the ‘Fix Mode’.

Free mode

The free mode can be used for surgical preparation, for communication with the patient and also during surgery. The CT / MRI image is freely visible in 3D, it can be moved, rotated, enlarged and reduced. The most important functions, however, are the slide and the ‘onion’ function. These allow visual immersion into individual layers of the CT / MRI image. The onion function removes individual layers from the outside, making the insides visible. This allows you to approach the treated area layer by layer. For the slide function, a virtual box (cube) slides through the layers and zooms in. Another function of the slide tool allows the 3D image to be cut open from every direction and angle possible.

Fix mode

The fix mode is designed for the operation itself. This mode allows the surgeon to place the 3-dimensional CT / MRI on the patient. Using Vuforia, three virtual landmarks are being placed onto the patient’s head. The CT / MRI image is then projected onto the head using the landmarks. The slide tool also works in the Fix Mode, so the surgeon can enter the individual layers of the CT / MRI. This enables him/her to gain deep insight into the patient’s tissue structure, during the operation.

The two physicians Dr. Patrick House and Dr. Hans von Lücken tested the Fix Mode. To do so, they placed the CT / MRI images onto each other’s heads and made the images brighter, darker and transparent. Afterward, they went through the individual layers with the slider and experienced the advantages of the VSI.

 

Speech control

Approximately 30% of the VSI functionalities can be controlled with speech. After testing this function, the doctors came to the conclusion, that especially this function is very practical in an operating scenario. Due to this feedback, apoQlar decided to focus on this function in their development, by expanding the tool.

 

Summary

The test showed that we are on a very good way concerning the development of the VSI. The picture quality, the different functionalities and also the voice control impressed the physicians. However, the Vuforia function is too complicated and time-consuming. So we will focus on automatic landmark recognition using image recognition algorithms. In addition to the box (cube) function, we currently also develop a sphere to allow a particularly comfortable and fitted view into the CT / MRT image for the physicians and surgeons.