Training the Next Generation of Surgeons with Augmented Reality

Two thirds of the world’s population don’t have access to safe surgery. That’s as many as 5 billion people.

And of the 313 million procedures undertaken globally every year, only 6% occur in the poorest areas where over one third of the world’s population lives.

To cope with this huge problem, it is estimated that by 2030 we will need to train over 1.27 million additional surgical providers, which of course presents some enormous logistical and financial challenges.

Proximie is a start-up looking to solve this problem with an Internet-enabled software tool that allows doctors to guide, help, support and train surgeons through the use of Augmented Reality. According to its Co-founder Dr. Nadine Hachach-Haram, the key advantage it offers is that it’s much more interactive than traditional simulation or live streaming tools, thus enabling higher levels of collaboration and better learning outcomes.

The Augmented environment is created by overlaying two live video feeds, one in the surgical theatre filming the operation, the other pointing to a blank surface connected to the surgeon’s tablet or laptop. This allows the surgeon to draw on their screen while looking at the live surgery feed, and for the remote team to see the marks made by the surgeon and follow them precisely at their end.

They are one of the companies being supported in their rapid growth by the UK Lebanon Tech Hub and have been attracting significant praise and attention from both the scientific and technology communities, winning the prestigious Royal Society of Medicine prize for Innovation in e-Health and being invited to join the MIT “Arab World Meets Silicon Valley” forum at TechCrunch Disrupt in September.

Dr. Hachach-Haram, a Lebanese surgeon based in the UK is passionate about the use of the software as a training aid for students:

When we had the old-fashioned viewing galleries, dozens of students were able to watch surgery without affecting or crowding the surgical area, and could engage with the surgeon and ask questions,” She recalls. “Fast-forward to 2016, where we have much smaller operating theatres, reduced facilities and overstretched operating lists- this environment is much less amenable to surgical education and engagement from medical students.

So Proximie is effectively updating that viewing gallery experience using 21st Century technology, so that a large number of students can log in and watch a procedure from wherever they are in the world.

Surgery is a very visual, precise and anatomical part of medicine and no matter how many times one reads it in a book, it’s always far more useful, interesting and engaging to be able to watch it live, ask questions, and visualise anatomy and pathology.

In September they will launch a pilot in partnership with UCL’s Royal Free Hospital to integrate Proximie into their surgical curriculum. The educational version of the platform also has a student dashboard function, which allows them to access elements relevant to their educational curriculum, giving the ability to take notes and screenshots for later review, for example.

A cohort of students will be given access to the platform and will be able to log in to a number of operating theatres to observe the surgeries in an interactive way with the added elements of annotation, AR and other great features that will make this a completely new way of learning.

This cutting -edge technology (no pun intended) includes features such as the ability to annotate live streams ad medical records integration, meaning doctors can easily access information about a patient during surgery and take snapshots during the procedure which are automatically added to that patient’s records.

Proximie is hardware agnostic and requires minimal infrastructure, working with any tablet, smartphone or laptop and providing an intuitive interface with no need for expensive tools. Surgeons can literally use AR to draw on their screen using something as simple and affordable as a capped bic pen.

This technology could have a significant impact in the quality of care provided to patients in conflict zones such as Gaza and Syria. The software allows an experienced surgeon to be virtually “parachuted in” to support and guide a procedure with their specialist expertise, without any of the danger and cost of physical travel.

There are, of course, some logistic challenges associated with live streaming technology, as anybody who’s ever been on a pixelated Skype call knows. Dr Hachach-Haram says that in situations where in remote areas or conflict zones they have modified the application so that it will still provide high quality video stream even at lower network speeds. “It’s an all-round adaptive solution, and while we’re a commercial platform, we’re also committed to social impact projects, as that’s a core component of our DNA. Proximie currently offers free licenses to charities to support them in tackling inequality in surgical knowledge and expertise.”

While the telesurgery feature currently needs to be scheduled in advance, future iterations are planned where the service will host a database of surgeons all over the world listed according to expertise, allowing colleagues to check their availability and seek assistance and advice in real time.

Even in developed countries such as the UK, the potential for this type of technology to maximize efficiency is enormous, as surgeons would be able to support procedures in different hospitals without the time and costs associated with commuting to various locations. It would also ensure better patient care, as the best specialist for a particular procedure could support it regardless of their geographical location at any given time.

Dr. Hachach-Haram says they will be watching the developments in the Virtual Reality market with interest, and it is not difficult to imagine how, as VR technology and hardware become more accessible and pervasive, it will offer even greater possibilities for immersive learning and expertise sharing.

Virtual and Augmented Reality certainly represent a booming sector, and there is a growing interest in the medical applications for these technologies. According to a recent report from Frost & Sullivan the next 10 years are set to redefine the future of both business and consumer processes and interactions and will be adopted widely across industries by 2025.

“AR and VR will be a giant step in transforming and democratising education and businesses, especially in medicine, where immersive technology can facilitate education and access-at-a-distance for medical care,” said Dr Rafael Grossmann, who was the first person to stream a live surgical procedure with Google Glass.His company Medical Realities, led by colleague Dr Shafi Ahmed, live-streamed the first VR surgery from The Royal London Hospital, with 54,000 viewers virtually present in the operating theatre.

From all around the world, they gained an exclusive point-of-view demonstration and interacted in real-time with the surgical event. Imagine how this level of interaction can deliver next-generation education for students worldwide, especially in developing countries, added Dr Grossman

There are also interesting developments in the hardware arena, with Georgian start-up InGlove developing a VR glove which could prove useful for applications such as medical simulations, and Purdue University researching a system called DeepHand, which uses a “convolutional neural network” that mimics the human brain and is capable of “deep learning” to understand the hand’s nearly endless complexity of joint angles and contortions. It uses a depth-sensing camera to capture the user’s hand and figure out where the hands and fingers are, applying algorithms to then replicate those motions in real time. Karthik Ramani, director of the University’s C Design Lab says that in Virtual Reality it is crucial for your hands to be able to accurately interact with the virtual world in order to provide full immersion.

It will be interesting to see how both new and existing technologies develop and find new applications across all sorts of areas such as medicine and education, and many agree that this will be the crucial factor in determining how quickly VR becomes pervasive. Watch this space