World leader: mixed reality in medicine
Operating in 3D
Since the beginning of 2018, the “Kath. Marienkrankenhaus” (catholic hospital) in Hamburg is using a highly innovative technology for surgeries that is more familiar from movies or video games than for use in medicine: mixed reality projects a virtual image into the real world environment.
Virtual images merge with the real world
The intelligent software of the assistance system – Virtual Surgery Intelligence (VSI) – can project MRI and CT images of the patient three-dimensionally and in impressive quality in a real environment. The images merge onto the part of the real patient to be operated on by using a special headset (Microsoft HoloLens). This supports doctors and junior doctors during their surgeries. The images are projected into real space, so that the real environment remains visible and hands free to act.
Senior physician Dr. med. Hans-J. von Lücken and his colleague Dr. med. Kathrin von Usslar, Department of Otolaryngology, Head, Neck and Plastic Facial Surgery at Marienkrankenhaus Hamburg, are developing the VSI Assistant together with the startup apoQlar. “After the first pre-tests of the different functionalities in the summer of 2017, we are using this innovative assistant half a year later at the Marienkrankenhaus Hamburg. With the VSI we can call up the CT / MRI images of our patients, view them virtually and fix them directly on them (mixed reality). The VSI is an intelligent application that recognizes the patient through anatomical landmarks and places the 3D images virtually on it.” says Dr. von Lücken. A manual comparison with the previously created 3D display on a screen, for example of a tumor tissue on the surgical
side, is not needed anymore and is significantly facilitated by the VSI and above all it is refined. “We have found that this gives us a much deeper and more detailed insight into the anatomical structures. But also for our interns the VSI is a big help. This way, they can orientate themselves more quickly and the more experienced doctor can better communicate their surgical approaches and procedures, “says the senior physician and initiator Dr. von Lücken. The VSI is controlled with language and gestures. In addition to MRI or CT images, other important patient data, such as surgical reports and laboratory data, can be retrieved and displayed, too.
Advantage for surgeons and patients
The new technology offers relief and additional support not only for the surgeons and doctors in training: Patients also benefit from the new assistant. The patient education of future surgeries is now more understandable and intuitive through the visual presentation. The assistant is at this time used supportively. The surgery remains in the hands of the attending physician and surgeon. He is still responsible.
Areas of application
Technological development has initially begun in the head and neck area and is steadily expanding to other areas. The VSI is suitable for all surgical specialties, such as in ENT medicine for sinus surgery, tumor and abdominal surgery, but also in radiology for diagnostics and interventional procedures. With the practical know-how in the hospital and the expertise for hardware and software of the company apoQlar, new requirements and functions as well as further application possibilities of the VSI assistant are developed and improved at all times.
The acquisition of the VSI Assistant was made possible by the support of the Friends and Sponsors Association at the Marienkrankenhaus Hamburg e.V.
The orginal press release of the catholic hospital Marienkrankenhaus in German can be found here.