Life-saving knowledge: New technology sifts through surgical videos

Most endoscopic operations generate videos that are of interest to other surgeons. How were difficulties overcome? What worked well, what didn’t? At what point did preventable mistakes occur? Answers to these questions can be life-saving for the person on the operating table. Researchers at the University of Klagenfurt are working on making huge pools of surgical videos easier to sift through, so that both future surgeons and seasoned surgical staff can learn from them more effectively.

The videos created during endoscopic operations can be useful in many ways: they can be used in the training of future surgeons, but they can also be used to evaluate the quality of an operation retrospectively. In both cases, it is time-consuming to evaluate the videos manually and search for specific content, and this is not done – often to the detriment of patients.

“We need the most powerful and accurate search technologies possible. For instance, we can focus on similarity: If we find a segment in which an error occurred, this can serve as input for another search for errors,” says Klaus Schöffmann, who has been working on technologies of this kind at the Department of Information Technology for many years. The same applies to particularly challenging operations: “If a problem has been solved well, we can look for similar approaches in other videos. In sum, medicine and medical professionals learn with the support of technology. Patients benefit from this.”

Klaus Schöffmann and his team are currently working on an FWF-funded project dealing with quality assurance in gynaecological operations. “If surgeons want to evaluate the quality of their team’s operations, they currently have to go through the videos manually and evaluate and count individual errors,” Klaus Schöffmann outlines the problem. With the help of AI methods or Deep Learning, the error evaluation should be automatic in the future. The technology has already come quite far: “Today, we can already identify the instruments used, certain anatomical structures and surgical actions fairly accurately in the videos. This also allows us to compile different statistics across a large number of patients. In this way, we can answer questions such as: (1) How often and for how long was an instrument used? (2) How often was a certain surgical step performed? or (3) What is the frequency of a certain complication?”