Imaging the world from a surgeon's perspective with a gaze aligned head-mounted camera

Head-fixed camera systems are widely known, but since they are aligned by the head only and not by the eyes they are not able to always „look“ at what the user is looking at. The prototype of a new mobile and head-mounted camera system was developed that is continuously aligned with the orientation of gaze. With this camera a surgeon can record his manual activities without missing crucial events. The video recordings can be used for example for documentation and teaching purposes. Other applications like neurophysiological, psychological, and marketing research as well as documentary movies for sports and other activities are also conceivable. Eye movements are tracked in real-time by video-oculography (VOG) and used as signals to drive servo motors that rotate the camera. The camera therefore mimics the natural exploration of a visual scene by using the sensorimotor output of the human eye movement system – that evolved over millions of years – to move an artificial eye. All the capabilities of multi-sensory processing for eye, head, and surround motions are detected by the vestibular, visual, and somatosensory systems and used to drive a technical camera system. With this system not only the biological gaze stabilization is used for image stabilization, but the camera also „sees“ what the surgeon is seeing. For image stabilization to work similarly well as with the human vestibulo-ocular reflex a small latency between eye and camera movement is required. With a choice of digital VOG cameras working at 100Hz, low-latency image processing algorithms (1.5 ms) and fast servo drivers (1000 deg/s) a total latency of 36 ms was achieved. The fixation accuracy of the system is 0.5 deg, which is well within the foveal range of 2 deg, and the resolution in terms of RMS noise is below 0.1 deg. Mobility was achieved by using an off-the-shelf subnotebook (1.5kg) for VOG computations. The notebook battery also powers the electro-mechanical components of the head-mounted system, which has a total weight of 600g. The system was tested during a 2 1/2 hour abdominal surgery. In a post-operative interview the surgeon confirmed that he could comfortably wear the camera without restrictions of eye, head, legs or arm mobility and his field of view was also not restricted. Future work will focus on further improvements of size, weight, and latency.

Fig. 1

Fig. 2