Humanoid Assessing Rehabilitative Exercises

Summary

Introduction: This article is part of the Focus Theme of Methods of Information in Medicine on “New Methodologies for Patients Rehabilitation”.

Background: The article presents the approach in which the rehabilitative exercise prepared by healthcare professional is encoded as formal knowledge and used by humanoid robot to assist patients without involving other care actors.

Objectives: The main objective is the use of humanoids in rehabilitative care. An example is pulmonary rehabilitation in COPD patients. Another goal is the automated judgment functionality to determine how the rehabilitation exercise matches the pre-programmed correct sequence.

Methods: We use the Aldebaran Robotics’ NAO humanoid to set up artificial cognitive application. Pre-programmed NAO induces elderly patient to undertake humanoid-driven rehabilitation exercise, but needs to evaluate the human actions against the correct template. Patient is observed using NAO’s eyes. We use the Microsoft Kinect SDK to extract motion path from the humanoid’s recorded video. We compare human- and humanoid-operated process sequences by using the Dynamic Time Warping (DTW) and test the prototype.

Results: This artificial cognitive software showcases the use of DTW algorithm to enable humanoids to judge in near real-time about the correctness of rehabilitative exercises performed by patients following the robot’s indications.

Conclusion: One could enable better sustainable rehabilitative care services in remote residential settings by combining intelligent applications piloting humanoids with the DTW pattern matching algorithm applied at run time to compare humanoid- and human-operated process sequences. In turn, it will lower the need of human care.

Keywords

Rehabilitation - humanoid robots - robotics - COPD - DTW

• References

• 1 KSERA project. (Online) Available from. www.ksera-project.eu (accessed 1/1/2013).
• 2 Brooks D, Howard AM. A computational method for physical rehabilitation assessment. In: 3rd IEEE RAS and EMBS International Conference onBiomedical Robotics and Biomechatronics (BioRob), Sept.26-29.. 2010: 442-447.
• 3 Schwarz L, Mkhitaryan A, Mateus D, Navab N. Human skeleton tracking from depth data using geodesic distances and optical flow. Image and Vision Computing 2012; 30 (03) 217-226.
• 4 Achumba IE, Bersch S, Khusainov R, Azzi R, Kamalu U. On time series sensor data segmentation for fall and activity classification. In: IEEE 14th International Conference one-Health Networking, Applications and Services (Healthcom), Oct. 10-13. 2012: 427-430.
• 5 Gupta GS, Mukhopadhyay SC, Sutherland M, Demidenko S. Wireless Sensor Network for Selective Activity Monitoring in a Home for the Elderly. Instrumentation and Measurement Technology Conference Proceedings (IMTC), IEEE, May1-3. 2007: 1-6.
• 6 Johnson DO. et al. Socially Assistive Robots: A comprehensive approach to extending independent living. International Journal of Social Robotics. Springer: 2013: 1-19. DOI: 10.1007/s12369-013-0217-8 in press.
• 7 Sakoe H, Chiba S. Dynamic programming algorithm optimization for spoken word recognition. IEEE Transactions on Acoustics, Speech and Signal Processing 1978; 26 (01) 43-49.
• 8 Nakagawa S. et al. Speaker-Independent English consonant and Japanese word recognition by a Stochastic Dynamic Time Warping method. Journal of Institution of Electronics and Telecommunication Engineers 1988; 34 (01) 87-95.
• 9 Reyes M, Dominguez G, Escalera S. Feature weighting in dynamic time warping for gesture recognition in depth data. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops). IEEE: 2011: 1182-1188.
• 10 Su CJ. Personal Rehabilitation Exercise Assistant with Kinect and Dynamic Time Warping. International Journal of Information and Education Technology 2013; 3 (04) 448-454.
• 11 Su CJ, Huang JY, Huang SF. Ensuring Home-based Rehabilitation Exercise by Using Kinect and Fuzzified Dynamic Time Warping Algorithm. In. Kachitvichyanukul V, Luong HT, Pitakaso R. (eds). Proceedings of the Asia Pacific Industrial Engineering & Management Systems Conference APIEM-2012. Patong Beach; Phuket, Thailand: December 2-5 2012: 884-895.
• 12 Rakthanmanon T. et al. Searching and Mining Trillions of Time Series Subsequences under Dynamic Time Warping. In. The KDD ’12 Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining. Beijing, China: August 12-16 2012: 262-270.
• 13 Wobbrock JO, Wilson AD, Li Y. Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes. In: Proceedings of the 20th annual ACM symposium on User interface software and technology ACM UIST. Newport, USA: October 7-10 2007: 159-168.
• 14 Adistambha K, Ritz CH, Burnett IS. Motion Classification Using Dynamic Time Warping. IEEE 10th Workshop on Multimedia Signal Processing. Cairns, Qld: October 8-10 2008: 622-627.
• 15 Celebi S, Aydin AS, Temiz TT, Arici T. Gesture Recognition Using Skeleton Data with Weighted Dynamic Time Warping. In. Battiato S, Braz J. (eds)7. Proc of the Int Conf on Computer Vision Theory and Applications. Barcelona, Spain: SciTePress; Feb 21-24. 2013 2013: 620-625.
• 16 Qiu Y, Wu X, Guo L. The design of lower limbs rehabilitative training device control system. In: Proceedings of the 2010 International Conference onComputer Application and System Modeling (ICCASM). Taiyuan, China: Oct 22-24 2010. 11 619-622.
• 17 Rakhomov R, Primov E, Min D. Behavioral synchronization of human and humanoid robot. 8th Int Conf on Ubiquitous Robots and Ambient Intelligence (URAI). IEEE: Nov 23-26 2011: 655-660.
• 18 Muscillo R, Conforto S, Schmid M, Caselli P, D’Alessio T. Classification of Motor Activities through Derivative Dynamic Time Warping applied on Accelerometer Data, In: Proceedings of EMBS-2007. IEEE: Aug 22-26 2007: 4930-4933.
• 19 Stikic M, Huynh T, Laerhoven K, Schiele B. Assessing everyday life behavioral rhythms for the older generation. Journal Pervasive and Mobile Computing archive 2009; 5 (05) 606-622.
• 20 Kim E, Helal S, Cook D. Human activity recognition and pattern discovery. IEEE Pervasive Computing, NJ, USA, 2010; 9 (01) 48-53.
• 21 Bae J, Liu L, Caverlee J, Rouse WB. Process Mining, Discovery, and Integration using Distance Measures. Int Conf Web Serv. 2006: 479-488.
• 22 Schumann S. Distance measures for surgical process models. Methods Inf Med 2013; 52 (05) 422-431.
• 23 Chidambaram V, Chiang Y-H, Mutlu B. Designing persuasive robots: how robots might persuade people using vocal and nonverbal cues. In: Proc of HRI ’12 Conf on Human-Robot Interaction, ACM. NY, USA: 293-300.
• 24 Feys P, Helsen WF, Liu X, Lavrysen A, Nuttin B, Ketelaer P. Effects of vision and arm position on amplitude of arm postural tremor in patients with multiple sclerosis. Arch Phys Med Rehabil 2004; 85: 1031-1033.
• 25 Bartneck C, Kulić D. Croft E, Zoghbi S. Measurement instruments for the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots. Int J Soc Robot 2009; 1 (01) 71-81.
• 26 Torta E, Oberzaucher J, Werner F, Cuijpers RH, Juola JF. The attitude toward socially assistive robots in intelligent homes: results from laboratory studies and field trials. Int J Hum-Comput Interact 2012; 1 (02) 76-99.
• 27 Robot Operating SYstem. (Online.) Available from. wiki.ros.org wiki.ros.org/smach (accessed 1/1/2013).