Introduction of augmented reality in ultrasound training – The set up for the UppStudy (Ultrasound aPP Study)

Purpose: Virtual and augmented reality are upcoming new technical features used in the entertainment industry. Here the first theoretic preparations towards implementing an augmented reality based ultrasound training are presented.3D objects in a wavefront file format should be visualized like on an ultrasound screen. Implementing the physical laws of reflection, intensity and absorption (Law et al 2011).

Material & methods: We developed a raytracing/-casting-based rendering method in order to approximate ultra sound propagation in piecewise homogenous media. In contrast to existing methods, which are based on volume data (like CT images), our scene geometries consist only of surfaces delimiting the volumes inside which the actual ultra sound propagation takes place. This enables interactive volume rendering within the resource constraints of modern smart phones, but also implies that material properties can only be associated with those boundary surfaces. To determine which material properties apply at each position the ray is sampled, we keep track of the currently active volume using a stack of volume IDs, which we accordingly modify on entering/exiting a volume through its surface. After determining the reflected energy at each point along the ray, we also add noise and blur in a post-processing step. Our surface-based rendering method thus requires only a compact 3D scene description, which however must contain only closed, non-overlapping surfaces, similar to our simplified test scene, consisting of a larger sphere (the “body”), enclosing a smaller box, which in turn contains a much denser small sphere.

Results: The results enabled a sonographer to imagine the original 3D objects without a problem (graphic 1). Further optimizing of the algorithm is needed to enable a smooth calculation of more complicated objects like internal organs.

Conclusion: Augmented/virtual reality simulation may open up more training possibilities for students to learn the motorical skills faster and more efficiently and also enable students to train on various pathological findings.

Fig. 1: demonstration of the US screen of the 3D model