Machine and Deep Learning Dominate Recent Innovations in Sensors, Signals and Imaging Informatics

Summary

Objectives: This review presents research papers highlighting notable developments and trends in sensors, signals, and imaging informatics (SSII) in 2022.

Method: We performed a bibliographic search in PubMed combining Medical Subject Heading (MeSH) terms and keywords to create particular queries for sensors, signals, and imaging informatics. Only papers published in journals containing greater than three articles in the search query were considered. Using a three-point Likert scale (1 = not include, 2 = perhaps include, 3 = include), we reviewed the titles and abstracts of all database results. Only articles that scored three times Likert scale 3, or two times Likert scale 3, and one time Likert scale 2 were considered for full paper review. On this pre-selection, only papers with a total of at least eight points of the three section co-editors were considered for external review. Based on the external reviewers, we selected the top two papers representing significant research in SSII.

Results: Among the 469 returned papers published in 2022 in the various areas of SSII, 90, 31, and 348 papers for sensors, signals, and imaging informatics, and then, the full review process selected the two best papers. From the 469 papers, the section co-editors identified 29 candidate papers with at least 8 Likert points in total, of which 9 were nominated as the best contributions after a full paper assessment. Five external reviewers evaluated the nominated papers, and the two highest-scoring papers were selected based on the overall scores of all external reviewers. A consensus of the International Medical Informatics Association (IMIA) Yearbook editorial board finally approved the nominated papers. Machine and deep learning-based techniques continue to be the dominant theme in this field.

Conclusions: Sensors, signals, and imaging informatics is a dynamic field of intensive research with increasing practical applications to support medical decision-making on a personalized basis.

Keywords

Biomedical informatics - machine learning - deep learning - personalized medicine

1 Introduction

Sensors, signals, and imaging informatics (SSII) are three separate, but not entirely independent, fields as defined by Hsu et al. as dealing with the acquisition, processing, analysis, and interpretation of the data to understand and treat a wide range of healthcare conditions [[1]]. The SSII field encompasses a vast research area with myriad potential clinical applications, and according to our annual reviews of the field, thousands of articles are published each year [[2] [3] [4]]. A growing number of research and review papers have been identified from year to year, with a large majority applying machine or deep learning-based approaches to well-defined SSII problems. The best papers selected this year also confirm this emerging trend in biomedical data analysis, which has an increasing number of clinically well-applicable approaches on an individual basis [[5], [6]]. However, it remains very difficult to identify and select truly novel and ground breaking approaches. In particular, the number of publications in the field of imaging informatics remains extremely high, and continues to be a key area of research in SSII.

Of the nine pre-selected candidate papers, one is from the field of sensor informatics, four are from signal informatics, and four are from imaging informatics. The only selected paper from the sensor informatics subfield reports on a very innovative multiple photonic band nearinfrared (mbNIR) sensor augmented with personalized medical features (PMF) in Shallow Dense Neural Networks (SDNN) for accurate, low-cost, and painless determination of the blood glucose level [[7]].

Interestingly, all contributions to signal informatics contribute this year to electrocardiography research and applications, including wearable electrocardiography (ECG) sensors, with advanced machine learning and deep learning-based algorithms to analyze, classify, and interpret ECG signals. Real-time patient-specific ECG classification by 1D Self-Operational Neural Networks (Self-ONNs) developed by Malik et al. is an excellent example that led to the selection of one of the two best papers [[5]]. A new method called BayeSlope, presented by De Giovanni et al. relies on unsupervised learning, Bayesian filtering, and nonlinear normalization to improve and correctly detect R-peaks in the ECG of a wearable sensor. Since BayeSlope is computationally intensive and can quickly drain the device's battery, the authors propose an online design that adapts its robustness to sudden physiological changes and its complexity to the heterogeneous resources of modern embedded platforms. It reports features that influence the classification decisions of deep models for multiclass classification of myocardial infarction and healthy ECGs [[8]]. In the study by Jahmunah et al. the authors developed convolutional neural network (CNN) models such as DensNet for classifying ECG signals from healthy subjects and patients with 10 classes of myocardial infarction (MI) based on the location of myocardial involvement. The models show some degree of visible explicability of the internal processes and may gain the necessary clinical acceptance. From a clinical perspective, they have the potential to be used for ECG triage of MI diagnosis in hospitals and outside the hospital [[9]]. Wen and Kang provide a deep learning package called Torch_ecg that compiles a large number of neural networks from existing and new literature for various ECG processing tasks and includes utilities for downloading, visualizing, preprocessing, and augmenting data. Torch_ecg also provides benchmark studies using the latest databases that illustrate the principles and pipelines for solving ECG processing tasks and reproduce results from the literature, providing the ECG research community with a powerful tool to meet the growing demand for the application of deep learning techniques [[10]].

For the imaging informatics subfield, the paper by Xia et al. was selected as one of the best papers. For accurate 3D modeling of cardiac chambers, the authors present an innovative approach for patient-specific generation of cardiac shapes and highlight the positive impact of incorporating patient data on the accuracy of predicted shapes while accelerating patient-specific acquisition of cardiovascular magnetic resonance (CMR) scans [[6]]. Because retinal images acquired with fundus cameras are often visually blurred due to imperfect imaging conditions, refractive medium turbidity, and motion blur, Qayyum et al. developed a single-shot deep image prior (DIP)-based approach for retinal image enhancement. Unlike typical deep learning-based approaches, this method does not require any training data. Instead, it learns the underlying image prior from a single degraded image. The proposed approach is time- and memory-efficient, which makes the solution viable for real-world resource-constrained environments [[11]]. Machine learning (ML) has also become an integral part of image-based diagnostics in pathology and radiology. Bridge et al. present the HighDicom library, which provides a high-level application programming interface (API) for the Python programming language that abstracts low-level details of the standard and allows encoding and decoding of image-derived information in Digital Imaging and Communications in Medicine (DICOM) format in a few lines of Python code. This work promotes the standardization of ML research and streamlines the ML model development and deployment process by making the library available as free and open-source [[12]]. Finally, Berntsen et al. investigate how a deep learning-based embryo selection model using only time-lapse image sequences performs in different patient ages and clinical conditions and how it correlates with traditional morpho-kinetic parameters. The authors show that fully automatic embryo scoring implies fewer manual evaluations and eliminates bias due to inter- and intra-observer variation [[13]].

Last but not least, we would like to draw your attention to this year's survey paper entitled “Security and Privacy in Machine Learning for Health Systems: Strategies and Challenges” by de Aguiar et al. also appearing in the SSII section of the IMIA Yearbook [[14]]. This paper investigates studies of security (attacks, defenses) and privacy-preserving strategies in ML for health systems and applications. We believe this topic is of great importance to SSII, as many ML/DL-based applications are already used in clinical settings and need regulatory approvement, where cybersecurity is an essential part of medical software regulation [[15]].

2 About the Paper Selection

Searching the literature for the best papers for the SSII section was challenging given the broad scope of this field. This year, we used the same search terms and acronyms for the queries as last year [[2]], which have been continuously expanded and harmonized in previous years [[1], [3], [4]]. Nevertheless, we focused on English-language research articles and excluded review articles. Subsequently, we performed the queries for sensors, signals, and imaging separately exclusively in PubMed to avoid duplicates from other repositories (see Appendix 2 in [[2]]).

In mid-January 2023, we performed the final query, which yielded a set of 90, 31, and 348 articles on sensors, signals, and imaging informatics, respectively. As last year [[2]], we set the threshold at a minimum of three articles for a relevant journal and excluded articles from irrelevant journals. We then reviewed all 469 titles and abstracts and scored them independently on a three-point Likert scale (1 = not included, 2 = maybe included, 3 = included). For 29 papers, all three section co-editors agreed on a total of at least 8 points. Then, the full papers were evaluated, again using the same 3-point Likert scale. Finally, we found 12 papers for which the section co-editors could agree again on at least 8 points: 2, 4, and 6 articles from sensors, signals, and imaging informatics, respectively, from which we selected the nine best candidate papers by consensus.

In consultation with the IMIA yearbook editors, we uploaded these nine papers for external review, in which six reviewers were invited, five of whom agreed to review the papers, while one did not respond. After the external review, the first and second ranked articles were selected as this year's best papers for the SSII section. Since the top two ranked papers represent a good cross-section of SSII, we suggested to the IMIA Yearbook Editor-in-Chief that the two papers be included, which was approved at the editorial meeting.

In addition, we would like to emphasize that while we have worked very hard to achieve a “perfect” formulation of the three queries, new terms, techniques, and technologies are emerging and the queries require continuous refinement of the formulation. Another open issue is the different spelling of journal names. We also believe that the “at least three” rule should be normalized to the total number of papers published by each journal to eliminate this inconsistency in the query protocol. These updates are planned for 2024.

The final selection of the top two papers this year was based on the originality of ML/DL-based methods taking into account patient individual information, outstanding results based on representative data repositories, the generalizability of the methods, and especially their excellent clinical applicability. [Table 1] presents the two selected articles. A content summary of these best papers can be found in the appendix of this synopsis.

Table 1 Best paper selection of articles for the IMIA Yearbook of Medical Informatics 2023 in the section 'Sensors, Signals and Imaging Informatics'. The articles are listed in alphabetical order of the first author's surname.

3 Conclusion and Outlook

The top nine papers for 2022 nicely illustrate recent efforts and trends in the use of ML/DL-based approaches in SSII to support medical decision-making, prognosis, and novel therapies. One observation is that all papers are based on ML/DL and an increasing number of papers address clinical questions on an individualized and personalized basis, highlighting this trend. As demonstrated in the survey paper on security and privacy aspects of machine learning, this issue also becomes very relevant when ML/DL-based tools are integrated into real-world applications for clinical use.

No conflict of interest has been declared by the author(s).

Acknowledgement

The section co-editors would like to thank Vivien Wegel and Christian Orsinger for running the database queries. We also thank Lina Soualmia, Kate Fultz Hollis and Adrien Ugon for supporting the external review process and the external reviewers for their input on the candidate best papers.

• References

• 1 Hsu W, Baumgartner C, Deserno TM. Notable papers and new directions in sensors, signals, and imaging informatics. IMIA Yearb Med Inform 2021Aug;30(1):150-8. doi: 10.1055/s-0041-1726526.
• 2 Baumgartner C, Deserno TM. Best Research Papers in the Field of Sensors, Signals, and Imaging Informatics 2021. Yearb Med Inform 2022 Aug;31(1):296-302. doi: 10.1055/s-0042-1742545.
• 3 Hsu W, Baumgartner C, Deserno TM. Notable papers and trends from 2019 in sensors, signals, and imaging informatics. IMIA Yearbook of Med Inform 2020 Aug;29(1):139-44. doi: 10.1055/s-0040-1702004.
• 4 Hsu W, Baumgartner C, Deserno TM. Advancing artificial intelligence in sensors, signals, and imaging informatics. IMIA Yearb Med Inform 2019 Aug;28(1):115-7. doi: 10.1055/s-0039-1677943.
• 5 Malik J, Devecioglu OC, Kiranyaz S, Ince T, Gabbouj M. Real-Time Patient-Specific ECG Classification by 1D Self-Operational Neural Networks. IEEE Trans Biomed Eng 2022 May;69(5):1788-801. doi: 10.1109/TBME.2021.3135622.
• 6 Xia Y, Chen X, Ravikumar N, Kelly C, Attar R, Aung N, et al. Automatic 3D+t four-chamber CMR quantification of the UK biobank: integrating imaging and non-imaging data priors at scale. Med Image Anal 2022 Aug;80:102498. doi: 10.1016/j.media.2022.102498.
• 7 Srichan C, Srichan W, Danvirutai P, Ritsongmuang C, Sharma A, Anutrakulchai S. Non-invasively accuracy enhanced blood glucose sensor using shallow dense neural networks with NIR monitoring and medical features. Sci Rep 2022 Feb 2;12(1):1769. doi: 10.1038/s41598-022-05570-8.
• 8 De Giovanni E, Teijeiro T, Millet GP, Atienza D. Adaptive R-Peak Detection on Wearable ECG Sensors for High-Intensity Exercise. IEEE Trans Biomed Eng 2023 Mar;70(3):941-53. doi: 10.1109/TBME.2022.3205304.
• 9 Jahmunah V, Ng EYK, Tan RS, Oh SL, Acharya UR. Explainable detection of myocardial infarction using deep learning models with Grad-CAM technique on ECG signals. Comput Biol Med 2022 Jul;146:105550. doi: 10.1016/j.compbiomed.2022.105550.
• 10 Wen H, Kang J. A novel deep learning package for electrocardiography research. Physiol Meas 2022 Nov 4;43(11). doi: 10.1088/1361-6579/ac9451.
• 11 Qayyum A, Sultani W, Shamshad F, Tufail R, Qadir J. Single-shot retinal image enhancement using untrained and pretrained neural networks priors integrated with analytical image priors. Comput Biol Med 2022 Sep;148:105879. doi: 10.1016/j.compbiomed.2022.105879.
• 12 Bridge CP, Gorman C, Pieper S, Doyle SW, Lennerz JK, Kalpathy-Cramer J, et al. Highdicom: a Python Library for Standardized Encoding of Image Annotations and Machine Learning Model Outputs in Pathology and Radiology. J Digit Imaging 2022 Dec;35(6):1719-37. doi: 10.1007/s10278-022-00683-y.
• 13 Berntsen J, Rimestad J, Lassen JT, Tran D, Kragh MF. Robust and generalizable embryo selection based on artificial intelligence and time-lapse image sequences. PLoS One 2022 Feb 2;17(2):e0262661. doi: 10.1371/journal.pone.0262661.
• 14 de Aguiar EJ, Traina C, Traina AJM. Security and Privacy in Machine Learning for Health Systems: Strategies and Challenges. IMIA Yearbook of Med Inform 2023:269-81.
• 15 Baumgartner C, Harer J, Schröttner J, editors. Medical Devices and In Vitro Diagnostics: Requirements in Europe. Cham: Springer Nature; 2022. ISBN: 2731-0493, eISBN: 2731-0507. doi: 0.1007/978-3-030-98743-5.

Publication History

Article published online:
26 December 2023