D6.1 – Real-time surgeon action detection and recognition

This document aims at describing the progress made by the SARAS project under Task 6.1 (Online surgeon action recognition), part of WorkPackage 6, which concerns the detection and recognition in real time of what actions the main surgeon performs using standard laparoscopic tools, either manually or in tele-operation via a da Vinci master robot available at University of Verona.
The input to this component of SARAS is the video streaming in from the available laparoscopic camera. Its output is, for each video frame, a number of bounding boxes showing where the various actions of interest are taking place, with attached scores (produced by a neural network) for each action class.