Haptic feedback is useful in remote manipulation of flexible endoscopes

Abstract

Background and study aims We developed the Endoscopic Operation Robot (EOR) version 3, offering built-in haptic feedback and manipulation of the entire scope with one hand. Manipulation of the flexible endoscope is done entirely remotely. However, inclusion of haptic feedback places a huge burden on the system. Our purpose in this study was to determine whether haptic feedback is needed in remote manipulation of a flexible endoscope.

Methods Five endoscopists performed total colonoscopy using a colonoscopy training model. A trial was conducted in which the endoscope was inserted up to the cecum five times with haptic feedback and five times without haptic feedback. Insertion time, maximum and mean haptic force, and incidence of sigmoid colon overstretching were compared between groups.

Results Insertion time was significantly shorter with haptic feedback than without, and overstretching of the sigmoid colon was less frequent. Insertion could thus be performed without using excessive force.

Conclusion Haptic feedback is useful for remote control manipulation of flexible endoscopes.

Introduction

In the platform of next-generation flexible endoscopes such as the Master and Slave Translumenal Endoscopic Robot (MASTER) [1] [2] [3] [4] and EndoSAMURAI [5] [6] [7] [8], attention has been mainly focused on achieving remote forceps manipulation. Meanwhile, manipulation of the flexible endoscope itself depends on traditional manipulation. This dependence limits developments in this field and accomplishing treatment with one endoscopist using intuitive manipulation is generally considered important. We therefore developed the Endoscopic Operation Robot (EOR) to first achieve the necessary conditions of remote manipulation and a platform for manipulation of the flexible endoscope itself [9] [10] [11] [12]. On the other hand, hepatic feedback reportedly is useful in robot-assisted medical procedures [13] [14] [15] [16]. Thus, the current EOR is the third generation, and includes haptic feedback (feelings of manipulation) felt through the master unit and a bilateral haptic feedback function that transmits the amount of force applied by the operator to the master unit to the tip of the scope with equal force. In addition, manipulation of the entire scope can be done with one hand [12].

Inclusion of haptic feedback makes the system larger and more complex and will be a source of increased costs. If the utility of haptic feedback were found to be low, its inclusion in the system, therefore, would not be warranted. In this study, we developed a new program in which haptic feedback in EOR version 3 does not function and investigated differences in manipulability with and without haptic feedback to clarify the system’s utility.

Methods

EOR version 3 system

The EOR ver.3 incorporates haptic feedback to provide complete remote control of flexible endoscope manipulation ([Fig. 1] and [Fig. 2]) [12]. The endoscope is an Olympus CF-240I (Tokyo, Japan), mounted on slave unit. Maximal length of an Olympus CF-240I is 130 cm. The colonoscopic insertion length is the same as a master device with the slave device to 60 cm. The colonoscopy training model was added to these to conduct this study.

Fig. 1 Components of master unit of EOR ver. 3. a Knob-like rotating part (a) (rotating knob). b Mini joystick. c Rotary motor. d Torque sensor. e Load cell. f Circuit switch.

Fig. 2 The slave unit of Endoscopic Operation Robot (EOR) ver. 3. to which the colonoscope (Olympus PCF-240; Tokyo, Japan) is attached.

The master unit of EOR ver. 3 ([Fig. 1]) consists of a knob-like rotating part (a) (rotating knob), a mini joystick (b), a rotary motor (c), torque sensor (d), a load cell (e) and the circuit switch (f). The master unit is an original device that enables four-axis movement of the flexible endoscope with one hand to provide intuitive manipulation. In brief, an operating knob equipped with a knob-like rotating part (a) (rotating knob) is installed on a linear motor with a long axis of 60 cm. By operating the linear motor in the long-axis direction with the rotating knob, the scope is inserted or retracted; by rotating the rotating knob, the scope can be rotated. To enable force feedback in these two axes, a load cell (e) is attached to the linear motor, and a rotary motor (c) and torque sensor (d) are installed on the rotation knob. Up-down and left-right angulation are performed with the thumb or index finger using a mini joystick (b) placed on top of the torque sensor. Because the master unit is short (60 cm) compared to the scope length of ≥ 1 meter, if the rotating knob reaches the edge of the master unit, target insertion and retraction cannot be performed. In this case, the master-slave circuit is first turned “off” using the circuit switch (f) on the left side of the joystick for angulation, then the rotating knob is moved to the opposite edge as operation only of the master unit. Later, the switch is turned “on” to connect the master-slave circuit, and the target insertion-retraction can be performed.

Because the slave unit ([Fig. 2]) has bidirectional haptic feedback, its construction is similar to that of the master unit. Instead of an operating knob, there is a rotating actuator that houses the endoscope handle. The rotating actuator includes a system to operate up-down angulation and left-right angulation and a system to operate the air supply/water supply buttons and air suction button (operated by 2-foot switches).

A new program in which haptic feedback does not function was added so that operability with and without haptic feedback could be compared. By checking the area of “Control” in a personal computer monitor, the program of haptic feedback was carried out ([Fig. 3]).

Fig. 3 The personal computer monitor. a The setting with haptic feedback checks “Master Free” and “MSON”. b Without haptic feedback, “Force Feed Off,” “Force Rot Off,” and “MSON” are checked in the area of “Control.”

Study design and protocol

A colonoscopy training model produced by KYOTO KAGAKU Co., LTD. (Kyoto, Japan) was used. This model has six training patterns (beginner’s grade 1 – 3, intermediate grade 1 – 2, and higher grade). For this study, beginner’s grade 2 was used.

In this study, a scope was inserted up to the cecum in a colonoscopy training model and the haptic sensations during insertion, time until insertion to the cecum, and incidence of overstretching of the sigmoid colon during insertion were compared between groups with and without haptic feedback. The robot operation monitor, including animation of the colonoscopy training model, was recorded using a video recorder. Overstretching of the sigmoid colon was confirmed by watching this recording. Overstretching of the sigmoid colon was considered present if, during push insertion, the sigmoid colon could be confirmed on the monitor to touch the right or transverse colon or abdominal wall even once.

Haptic sensations were recorded every 0.2 seconds with master unit insertion-retraction (push/pull; N), clockwise torque (N.m), and counterclockwise torque (N.m) as parameters. Maximum and mean values for the different parameters were obtained first with each insertion, after which the median values for these maximum and mean values were compared.

To achieve familiarity with manipulation of the EOR version 3 master unit, the endoscopists initially practiced scope insertion to the cecum twice. They then performed the same procedure five times with haptics, and the next five times without haptics. During this time, data were recorded for evaluation.

The endoscopists had more than 2 years’ experience performing colonoscopy or hadg performed more than 200 procedures.

Statistical analysis

Median and quartiles were calculated. P values were also computed by using Student’s t test. A probability less than 0.05 was considered to represent a significant difference between the samples studied.

Results

Total colonoscopy using the training model was performed a total of 50 times (with haptics (hap +) 25 times, without haptics (hap –) 25 times) by the five endoscopists. The cecum was reached in 100 % of cases. The following results are shown as median values. Insertion time was 70 seconds for hap + and 87 seconds for hap –. Maximum pull was 5.235 N for hap + and 7.335 N for hap –, and mean pull was 0.939 N for hap + and 1.158 N for hap –. Mean clockwise was 0.041 N.m for hap + and 0.072 N.m for hap –. Maximum counterclockwise was 0.064 N.m for hap + and 0.156 N.m for hap – and mean counterclockwise was 0.012 N.m for hap + and 0.029 N.m for hap –. Incidence of sigmoid colon overstretching was 8 % in hap + and 32 % in hap –. Significant differences (P < 0.05) were seen in all seven parameters ([Table 1]).

Table 1 Force parameters compared between groups with and without hepatic feedback.

	Hap (+)	Hap (–)
Parameter	Median (Q25, Q75)	Median (Q25, Q75)	P value
Max push (N)	17.90 (16.33, 24.08)	22.08 (19.49, 25.85)	0.12
Max pull (N)	5.27 (3.37, 7.99)	7.34 (5.51, 15.49)	< 0.05
Max clockwise (N.m)	0.163 (0.122, 0.305)	0.259 (0.202, 0.312)	0.38
Max counterclockwise (N.m)	0.064 (0.040, 0.134)	0.156 (0.051, 0.385)	< 0.05
Mean push (N)	4.07 (3.49, 4.52)	4.34 (3.79, 5.21)	0.21
Mean pull (N)	0.94 (0.68, 1.14)	1.16 (0.97, 1.63)	< 0.05
Mean clockwise (N.m)	0.041 (0.037, 0.049)	0.072 (0.047, 0.099)	< 0.05
Mean counterclockwise (N.m)	0.012 (0.009, 0.025)	0.029 (0.017, 0.128)	< 0.05
Examination time (min)	70 (63.5, 76.5)	87 (62.26, 119.5)	< 0.05
All insertion length (cm)	64.05 (63.75, 65.00)	64.10 (63.00, 65.25)	0.94
Overstretching (%)	8	32	< 0.05

Max, maximum; min, minutes.
Median (Q25, Q75), Q25, lower quartile (25 % quantile); Q75, upper quartile (75 % quantile)
Significant at < 0.05

Discussion

With the da Vinci surgical robot system, there have been calls for inclusion of tactile/haptic feedback. Even so, this system is currently in clinical use without such feedback and results have been good. In developing the EOR, we used an existing flexible endoscope with the aim of capturing an “intuitiveness” that reflects the feeling of scope operation that endoscopists have cultivated over many years. In EOR version 3, we included haptic feedback for the first time, but had not objectively evaluated whether this function is necessary. Inclusion of a haptic feedback function has disadvantages: among other things, it makes the system more complex with the addition of various sensors, heavier and larger to achieve sufficient rigidity that vibrations are not picked up. If this system did not offer clear benefits, we would clearly want to eliminate it from the system design.

This study showed that insertion time was shorter and incidence of sigmoid colon overstretching was lower with haptic feedback than without, and that insertion could be done without using excessive force. This demonstrates for the first time that haptic feedback is beneficial in remote manipulation of flexible endoscopes. When such feedback was not used, great force was needed in pulling, in particular, and a higher incidence of overstretching the sigmoid colon seemed to be a distinguishing characteristic. This was attributed to the fact that operators cannot feel resistance during insertion from the master unit with vision only and they exert too much force when pulling to shorten the sigmoid colon because they are overly conscious of the risk of overstretching the sigmoid colon. However, the actual frequency of overstretching the sigmoid colon is high. As a compromise for the purpose of inserting the scope to the cecum, endoscopists insert the scope with a push and cause overstretching. On the other hands, all insertion lengths were approximately the same in both groups. This is because the operator achieved intestinal shortening in the deep part beyond the sigmoid colon. Moreover, both maximum push and mean push did not show significant differences between hap (+) group and hap (–) groups. In our previous study, the colon was divided into two zones: Zone A was from the rectum to the sigmoid/descending colon transition and zone B was from the sigmoid/descending colon transition to the cecum [12]. Both maximum push and mean push were significantly higher for zone B than for zone A, demonstrating that a stronger force was required for deeper insertion. We opined that the strong force required for insertion of the colonoscope into the sigmoid colon did not significantly influence either the maximum or mean force required for insertion of the scope into the cecum. In achieving intestinal shortening, manipulation is needed without too much insertion while feeling the resistance as the scope is being inserted, and bilateral haptic feedback that can ascertain the haptic force in the master unit and transmit suitable force to the slave unit is considered necessary.

In the current, the overall time interval of the control cycle was 1 minute with a hardware interval trigger, while the data recording cycle was 0.2 second, as shown in the paper. The force and position data in the actual system were captured every 1 minute in the control interval cycle for the bilateral control method. Even with utilization of a preemptive multitasking thread program, the excessive usage of the data bus for accessing Solid state drive (SSD) might disturb the hardware interval period because today's computer systems tend to prefer access for data storage. Hence, the recording cycle was reduced to every 0.2 seconds. If we require more frequent data recording, Random access memory (RAM)-based data acquisition would be useful, given the advances in today's personal computer systems with the gigabyte (GB) order of dynamic random access memory (DRAM). Of course, we realize that the stability problem derives not only from the interval time length, but also from other software and mechanical elements.

While retaining its possibilities as a robot to assist in colonoscope insertion, the ultimate aim for the EOR is to provide a treatment robot. The current investigation was conducted as a preliminary step to demonstrate whether haptic feedback is useful in scope manipulation.

Creating a robot system for endoscopic treatment requires: 1) a flexible endoscope robot for remote forceps manipulation in the master; and 2) a robot that can perform scope manipulation to access the treatment target. This means, in addition to access in the broad sense of arriving in the vicinity of the treatment target, there is access that demands fine movements at treatment targets that cannot be handled with forceps manipulation only in the restricted workspace of the intestinal tract. For example, access manipulations include activities such as proximate operations that maintain a proper sense of distance with the lesion and approach angle manipulations. This second element is what EOR acts on. Endoscopic treatment was originally developed as a low-invasive method that can be performed in a relatively short time by a single endoscopist without the need for general anesthesia. By simply combining a flexible endoscope robot for remote manipulation of the master and other forceps together with EOR, the manipulation system may become more complex than useful, and doubts have been raised as to whether use of such robots offers any advantages. We think there is a need to take apart both operation systems and rebuild a platform that enables more intuitive manipulation, and this represents our next task [17].

Conclusion

In total colonoscopy with EOR version 3 using a training model, insertion time was significantly shortened and incidence of sigmoid colon overstretching was lower with haptic feedback than without it. Insertion could be achieved without using excessive force, and haptic feedback is considered beneficial.

Competing interests

None

Acknowledgements

This work was supported by Grant-in-Aid for Scientific Research (23500573, 263500554 and 17K01431), Grant of the Princess Takamatsu Cancer Research Fund (13-24505) and Terumo Foundation for Life and Arts (15-I101).

• References

• 1 Phee SJ, Ho KY, Lomanto D. et al. Natural orifice transgastric endoscopic wedge hepatic resection in an experimental model using an intuitively controlled master and slave translumenal endoscopic robot (MASTER). Surg Endoscopy 2010; 24: 2293-2298
• 2 Phee SJ, Low SC, Huynh VA. et al. Master and slave translumenal endoscopic robot (MASTER) for natural orifice translumenal endoscopic surgery (NOTES). Conf Proc IEEE Eng Med Biol Soc 2009; 4: 1192-1195
• 3 Ho KY, Phee SJ, Shabbir A. et al. Endoscopic submucosal dissection of gastric lesions by using a master and slave tranluminal endoscopic robot (MASTER). Gastrointest Endosc 2010; 72: 593-599
• 4 Phee SJ, Reddy N, Chiu PW. et al. Robot-assisted endoscopic submucosal dissection is effective in treating patients with early-stage gastric neoplasia. Clin Gastroenterol Hepatol 2012; 10: 1117-1121
• 5 Ikeda K, Sumiyama K, Tajiri H. et al. Evaluation of a new multitasking platform for endoscope full-thickness resection. Gastrointest Endosc 2011; 73: 117-122
• 6 Swanstrom LL. NOTES: platform development for a paradigm shift in flexible endoscopy. Gastroenterology 2011; 140: 1150-1154
• 7 Yeung BP, Gourlay T. A technical review of flexible endoscopic multitasking platforms. Int J Surg 2012; 10: 45-54
• 8 Klibansky D, Rothstein RI. Robotics in endoscopy. Curr Opin Gastroenterol 2012; 28: 477-482
• 9 Kume K, Kuroki T, Sugihara T. et al. Development of a novel endoscopic manipulation system: The endoscopic operation robot. World J Gastrointest Endosc 2011; 3: 145-150
• 10 Kume K, Kuroki T, Shingai M. et al. Endoscopic submucosal dissection using the endoscopic operation robot. Endoscopy 2012; 44: E399-E400
• 11 Kume K, Kuroki T, Shingai M. Development of a novel endoscopic manipulation system: The endoscopic operation robot ver. 2. Hepatogastroenterology 2015; 62: 43-45
• 12 Kume K, Sakai N, Goto T. Development of a novel endoscopic manipulation system: The endoscopic operation robot ver. 3. Endoscopy 2015; 47: 815-819
• 13 Koehn JK, Kuchenbecker KJ. Surgeons and non-surgeons prefer haptic feedback of instrument vibrations during robotic surgery. Surg Endosc 2015; 29: 2970-2983
• 14 Spinelli A, David G, Gidaro S. et al. First experience in colorectal surgery with a new robotic platform with haptic feedback. Colorectal Dis 2017; DOI: 10.1111/codi.13882. [Epub ahead of print]
• 15 Pacchierotti C, Prattichizzo D, Kuchenbecker KJ. Cutaneous feedback of fingertip deformation and vibration for palpation in robot surgey. IEEE Trans Biomed Eng 2016; 63: 278-287
• 16 Griffin JA, Zhu W, Nam CS. The role of haptic feedback in robotic-assisted retinal microsurgery systems: a systematic review. IEEE Trans Hepatics 2017; 10: 94-105
• 17 Kume K. Flexible robotic endoscopy: current and original devices. Comput Assist Surg 2016; 1: 150-159