Evaluation of the Use of a Tablet Computer with a High-Resolution Display for Interpreting Emergency CT Scans

Abstract

Purpose: Evaluation of the potential usability of an iPad 3 with a high-resolution display in CT emergency diagnosis compared to a 3 D PACS workstation.

Materials and Methods: 3 readers used a 5-point Likert scale to evaluate 40 CCT scans and 40 CTPA scans to determine the detectability of early signs of infarction in CCT or segmental and subsegmental pulmonary embolisms in CT angiography of the pulmonary arteries (CTPA) on the iPad 3 (Apple Inc., USA) using an application for image viewing (Visage Ease, Visage Imaging GmbH, Berlin) and on a 3 D PACS workstation (Visage 7.1, Visage Imaging, Berlin) using a certified monitor for image viewing. The results were compared using the Wilcoxon rank sum test, Spearman’s correlation coefficient, and a kappa statistic.

Results: There was no significant difference in the median evaluations for the readings of both the CCT scans and the CTPA scans on the iPad 3 and on the workstation (p > 0.05) for all three readers. The mean Spearman’s correlation coefficient for CCT and CTPA was 0.46 (± 0.2) and 0.69 (± 0.16), respectively, for the comparison iPad/PACS, 0.41 (± 0.16) and 0.68 (± 0.06), respectively, for the interobserver agreement on the iPad, and 0.35 (± 0.05) and 0.68 (± 0.10), respectively, for the interobserver agreement on the PACS. Mean kappa values for CCT of 0.52 (± 0.17) for the comparison iPad/PACS and 0.33 (± 0.16) and 0.32 (± 0.16), respectively, for the interobserver agreement on the iPad and the PACS were achieved. For CTPA average kappa values of 0.67 (± 0.19) were calculated for the comparison iPad/PACS and 0.69 (± 0.08) and 0.60 (± 0.14), respectively, for the interobserver concordance on the iPad 3 and the PACS. All differences were not statistically significant (p > 0.05).

Conclusion: The variability of the interpretation of typical emergency scans on an iPad 3 with a high-resolution display and on a 3 D PACS workstation does not differ from the interobserver variability.

Key Points:

• Variability of interpretation of emergency scans on an iPad 3 and on a 3 D PACS workstation shows no difference in interobserver variabilitiy.

• New applications for the Ipad in radiology include quick preliminary evaluation on duty and real-time consultation of radiological background services or expert opinions.

• Limitations for the application of tablet-computers in radiology are screen size and inconsistent and non-standardized environmental conditions when using a mobile image display device.

Citation Format:

Tewes S, Rodt T, Marquardt S et al. Evaluation of the Use of a Tablet Computer with a High-Resolution Display for Interpreting Emergency CT Scans. Fortschr Röntgenstr 2013; 185: 1063 – 1069

Zusammenfassung

Ziel: Evaluation einer potentiellen Nutzbarkeit eines iPad 3 mit hochauflösendem Display im Rahmen der CT-Notfalldiagnostik im Vergleich zu einer 3D-PACS-Workstation.

Material und Methoden: Die Erkennbarkeit von Infarktfrühzeichen in der CCT bzw. von segmentalen und subsegmentalen Lungenembolien in der CT-Angiografie der Pulmonalarterien (CTPA) wurde durch 3 Leser anhand von jeweils 40 Untersuchungen sowohl am iPad 3 (Apple Inc., USA) unter Verwendung einer Applikation zur Bildbetrachtung (Visage Ease, Visage Imaging GmbH, Berlin) als auch an einer 3D-PACS-Workstation (Visage 7.1, Visage Imaging, Berlin) unter Verwendung eines zertifizierten Befundmonitors anhand einer 5-Stufen Likert-Skala evaluiert. Die Ergebnisse wurden mithilfe des Wilcoxon-Rangsummentests, des Korrelationskoeffizienten nach Spearman und einer Kappa-Statistik verglichen.

Ergebnisse: Es ergab sich bei allen 3 Lesern sowohl für die CCT als auch für die CTPA kein signifikanter Unterschied der medianen Bewertungen zwischen den Lesungen am iPad und am PACS (p > 0,05). Der mittlere Korrelationskoeffizient nach Spearman betrug für CCT und CTPA 0,46 (± 0,2) bzw. 0,69 (± 0,16) für den Vergleich iPad/PACS, 0,41 (± 0,16) bzw. 0,68 (± 0,06) für die Interobserverübereinstimmung am iPad und 0,35 (± 0,05) bzw. 0,68 (± 0,10) für die Interobserverübereinstimmung am PACS. Es ergaben sich für die CCT mittlere Kappa-Werte von 0,52 (± 0,17) für den Vergleich iPad/PACS sowie 0,33 (± 0,16) bzw. 0,32 (± 0,16) für die Interobserverübereinstimmung am iPad bzw. am PACS. Für die CTPA wurden mittlere Kappa-Werte von 0,67 (± 0,19) für den Vergleich iPad/PACS sowie 0,69 (± 0,08) bzw. 0,60 (± 0,14) für die Interobserverkonkordanz am iPad bzw. am PACS errechnet. Sämtliche Differenzen waren statistisch nicht signifikant (p > 0,05).

Schlussfolgerung: Die Variabilität der Interpretation von typischen Notfalluntersuchungen an einem iPad 3 mit hochauflösendem Display und an einer 3D-PACS-Workstation unterscheidet sich nicht von der Interobservervariabilität.

Key words

PACS - observer performance - technical aspects - CT angiography - brain - ischemia/infarction

Introduction

Since the introduction of the Apple iPad in April 2010, the prevalence of mobile tablet computers has increased rapidly and they already comprise a major portion of the PC market. Numerous different versions and competing products are now available. Based on initial studies with smartphones, both users and manufacturers showed great interest in applications for viewing radiological image data soon after the market introduction of the iPad [1] [2] [3] [4]. After significant initial regulatory concerns, the FDA finally approved the first mobile radiology application for the iPad for computed tomography (CT), magnetic resonance imaging (MRI), and positron emission tomography (PET) in February 2011 after a multi-year evaluation process [5]. The approval makes it possible to use a tablet for the primary interpretation of image data when a full diagnostic workstation is not available. Several additional applications have subsequently also received FDA approval. In Germany the iPad currently cannot be used for image interpretation due to the normative framework conditions. However, tablet computers with high-resolution displays, such as the iPad 3, largely meet the requirements of the current quality assurance guidelines according to the X-Ray Ordinance and DIN Prestandard 6868 – 57 [6] [7] [8].

The goal of this study was therefore to evaluate the potential usability of the iPad 3 for emergency diagnosis on the basis of two typical clinical pictures compared to a 3 D PACS workstation.

Materials and Methods

The CT scans were selected on the basis of two typical issues in radiological emergency diagnosis: Detection of early signs of cerebral infarction and diagnosis of pulmonary artery embolism. The retrospective data evaluation was approved by the local ethics commission.

CCT data acquisition

To evaluate early signs of infarction, 20 native cerebral CT scans (CCT) acquired between 10/2010 and 5/2012 during the night or weekend shift and exhibiting one or more early signs of infarction (hyperdense middle cerebral artery sign, loss of insular cortex, inability to differentiate between gray and white matter, sulcus effacement) were selected retrospectively from the PACS. The selection was performed primarily on the basis of the radiological findings. Prior to inclusion in the study, the data were verified by two of the authors in consensus. 20 additional native CCT scans with no abnormal acute ischemic findings were also selected. In total, 40 CCT scans from 40 patients (f = 23, m = 17, average age: 64, range: 22 – 91) were used. All CCT scans were acquired on 16-row multidetector spiral computed tomography units (MDCT) using a sequential scan technique and orbitomeatal angulation of the gantry. Two data sets with a slice thickness of 2.5 mm and 5.0 mm were reconstructed from the raw data using filtered back projection.

CTPA data acquisition

To evaluate pulmonary artery embolisms, 20 CT angiographies of the pulmonary arteries (CTPA) acquired between 5/2011 and 5/2012 during the night or weekend shift and exhibiting acute pulmonary artery embolisms on the segment or subsegment level were retrospectively selected from the PACS. The scans were selected as described for CCT. 20 additional CTPA scans without any evidence of embolism were also selected. In total, 40 CTPA scans from 40 patients (f = 19, m = 21, average age: 55, range: 23 – 86) were used. All CTPA scans were acquired with a 64-row MDCT unit (VCT, GE Healthcare, Chalfont St. Giles, Great Britain). The contrast agent (Imeron 350, Bracco, Constance, Germany), followed by 50 ml of a physiological saline solution, was injected via an antecubital indwelling venous cannula (18G) with a flow rate of 4 ml/s using a double-head injector. The examination region extended from the apex to the base of the lung. Data acquisition was started via automatic bolus detection 5 seconds after reaching a threshold value of 100 HU in the right ventricle. The image data were reconstructed with a slice thickness of 1.25 mm and a reconstruction interval of 1.0 mm via filtered back projection using the standard algorithm.

Reader study

The CT data sets were evaluated under controlled ambient conditions by three radiologists with 3, 4, and 6 years of experience evaluating emergency CT scans. The first reading was performed on a standard PACS workstation (HP Z800, Hewlett-Packard, Palo Alto, USA) with a calibrated monitor of category B (Eizo Radiforce RS210 (21.2 inches), Eizo, Hakusan, Japan) using the 3 D thin-client solution Visage 7.1 (Visage Imaging, Berlin). The readers had access to all software functions (including windowing, zooming, measuring, and multiplanar reformation). The second reading was performed at an interval of at least 3 weeks after renewed randomization of the sequence of the data sets on a 3^rd generation iPad (iPad 3, Apple Inc., Cupertino, USA) using an application for displaying radiological image data (Visage Ease, Visage Imaging, Berlin). A screenshot of the application with additional explanations is provided in [Fig. 1]. The iPad 3, which was released 5/2012, achieves a pixel density of 264 dpi at a resolution of 2048x1536 pixels and a screen size of 9.4 inches. The indicated application (app) allows the viewer to perform basic post-processing functions (windowing, zooming, measuring). The image data were loaded on a per case basis from the Visage server via a secure (WPA2) WLAN connection on the hospital-wide intranet.

Fig. 1 The figure shows a screenshot of the iPad app, Visage Ease, which was used to evaluate the image data. The scan is a CTPA scan with evidence of a pulmonary embolism. The application makes it possible to download and view image data on the Visage PACS server. In addition to being able to scroll through a series, the following basic post-processing functions can be operated via touchscreen: windowing, zooming, and panning.

The image data were evaluated with respect to the presence of early signs of infarction (see above) or a pulmonary artery embolism on the basis of a 5-point Likert confidence scale (definitely pathological – probably pathological – uncertain – probably not pathological – definitely not pathological). The readers did not know the clinical finding or the relative proportion of pathological scans. However, the readers were informed that the frequency distribution did not correspond to the clinical routine and that the entire range of the confidence scale was to be used for the evaluation. The readers were also asked to present their subjective impressions of the ease of use of the iPad and the application.

Statistics

The Likert scale evaluations were non-parametric on the ordinal scale level. The significance test of the median scores between the iPad and the workstation were performed for all three readers using the Wilcoxon rank sum test. The Spearman’s rank correlation coefficient (Spearman’s rho) was calculated for the concordance analysis between the different readers and the two interpretation situations. Moreover, the Cohen’s kappa statistic was calculated for the different observers as well as the two different interpretation situations according to the Likert scale (definitely pathological – probably pathological –> pathological/uncertain; probably not pathological – definitely not pathological –> not pathological). Differences in the mean correlation coefficients as well as the kappa values were checked for statistical significance via t-test. The null hypothesis was rejected for all tests with a p-value of less than 0.05. All statistical evaluations were performed using the SPSS software package (SPSS 20, IBM, Ehningen).

Test of the iPad 3 in accordance with the quality assurance guidelines

The image display properties of the iPad 3 were tested in accordance with the requirements of the quality assurance guidelines [6] and DIN V 6868 – 57 [7] using a calibrated device for measuring luminance (LX Plus, IBA Dosimetry GmbH, Schwarzenbruck) on the basis of the SMPTE test image.

Results

There was no significant difference in the median evaluations between the readings on the iPad 3 and on the workstation (p > 0.05) for all three readers both for the analysis of the CCT scans and the CTPA scans.

In the analysis of the ordinal scale evaluations of the CCT scans, the mean Spearman’s correlation coefficient was 0.46 (± 0.2) for the comparison of the two interpretation situations, 0.41 (± 0.16) for the interobserver agreement when using the iPad 3 and 0.35 (± 0.05) for the interobserver agreement when using the workstation. The differences were not statistically significant (p > 0.05). In the case of CTPA, the mean Spearman’s correlation coefficient was 0.69 (± 0.16) for the comparison of the two interpretation situations, 0.68 (± 0.06) for the interobserver agreement when using the iPad 3 and 0.68 (± 0.10) for the interobserver agreement when using the workstation ([Fig. 2]). The differences were also not statistically significant (p > 0.05).

Fig. 2 Graphic representation of the comparison of the iPad and workstation results for CCT and CTPA scans.

According to the Likert scale, the CCT scans resulted in mean kappa values of 0.52 (± 0.17) for the comparison of the two interpretation situations and 0.33 (± 0.16) and 0.32 (± 0.16), respectively, for the interobserver concordance on the iPad 3 and the workstation. For CTPA average kappa values of 0.67 (± 0.19) were calculated for the comparison of the two interpretation situations and 0.69 (± 0.08) and 0.60 (± 0.14), respectively, for the interobserver concordance on the iPad 3 and the workstation ([Fig. 2]). All differences were not statistically significant (p > 0.05). Based on Landis and Koch, the kappa values were evaluated as follows: k < 0: poor agreement, 0 <k< 0.20: minimal agreement, 0.21 < k < 0.40: sufficient agreement, 0.41 < k < 0.60: moderate agreement, 0.61 < k < 0.80: considerable agreement and k >  0.81: almost complete agreement.

As an example, typical findings of an early sign of infarction with (a) and without (b) a finding discrepancy between the iPad 3 and the workstation are illustrated in [Fig. 3a, b]. In the subjective evaluation of the usability of the iPad 3, all readers stated independently that windowing via touchscreen in particular was very unfamiliar and significantly less easy to use compared to the PACS workstation.

Fig. 3 Figure a shows an example of a CCT scan with evidence of early signs of infarction with discrete swelling of the brain and reduction of the ability to differentiate between the right-parietal medulla and cortex. The finding was identified as pathological by all readers on the PACS workstation but only by two of the three readers on the iPad. Figure b shows a hyperdense middle cerebral artery sign on the right side as a typical early sign of infarction. This finding was correctly identified by all readers in both interpretation situations.

The test of the display quality of the iPad 3 according to the quality assurance guidelines yielded a veiling luminance L_S of 0.3 cd/m², a minimum luminance L_min of 0.4 cd/m² and a maximum luminance L_max of 410 cd/m². The deviations in luminance within the image were less than ± 15 % compared to the center of the image and the visual image quality criteria were fulfilled. In table 1 the data are compared to the requirements of the quality assurance guidelines and the values of the monitor used for image viewing.

Discussion

In the present study, two different interpretation tools, the iPad 3 with a high-resolution display and a 3 D workstation as the gold standard, were examined in a reader study with respect to the evaluation of CT scans. The CT scans were selected on the basis of two typical issues in radiological emergency diagnosis: The detection of early signs of infarction in CCT and the diagnosis of pulmonary artery embolisms [9] [10]. The results show that the agreement of the evaluation of the two types of scan on the iPad 3 and the 3 D PACS workstation does not differ from the agreement between the different readers. The kappa values calculated by us are within the range of the data published in the radiology literature. Therefore, in a metaanalysis of 15 studies for detecting early signs of infarction using native CCT, Wardlaw et al. calculated kappa values of 0.14 to 0.78 with an average sensitivity of 66 % [11]. With respect to CT diagnosis of pulmonary artery embolisms, the significantly higher kappa values of 0.60 – 0.67 calculated by us also correspond to the data published in the literature. Consequently, in a prospective multicenter study, Courtney et al. calculated a kappa value of 0.75 for the interobserver agreement [12]. Kelly et al. report kappa values of 0.69 in the case of embolisms on the lobe level. A limitation of the analysis to the segment and subsegment level, as in our study, resulted in kappa values of 0.54 and 0.44 [13].

The previously published studies regarding the use of the iPad for radiological diagnosis use older versions of the device with a lower-resolution display (1024 × 768px, 132ppi). These studies yielded satisfactory to good agreement with the interpretation using PACS workstations. In a study regarding the detection of 50 pulmonary embolisms using the 1st generation iPad compared to the PACS, two readers interpreted 98 % of the cases correctly in both interpretation situations [14]. Mclaughlin et al. were able to show agreement in the interpretation of CCT scans in 93 of 100 cases [15]. The misdiagnoses on the iPad related to very discrete findings. The same authors used a low-contrast phantom to evaluate the display quality and only achieved results on the iPad comparable to those of a monitor (2 megapixels) by using the zoom function. It must be taken into consideration that the display of the 3^rd generation iPad used in our study has a significantly higher resolution than the iPad 1 used in the indicated study. In a study by John et al., 79 CT scans and 9 MRI scans were evaluated by three radiologists on the iPad and PACS with respect to typical emergency issues and major discrepancies were found in 3.4 % of cases and minor discrepancies in 5.6 % [16]. In the analysis of MRI scans in spinal emergency diagnosis, McNulty et al. did not find a difference in accuracy compared to interpretation using a “secondary class” monitor (comparable with category B of the quality assurance guidelines) [17].

In agreement with the mentioned studies, we see a potential role of the iPad 3 in radiology primarily as a supplement to established PACS workstations [14] [15] [16] [17]. Mobile access to image data allows quick preliminary initial assessment as well as teleconsultation (e. g. of the radiological background service) under emergency conditions. Remarkably the FDA has already approved multiple iPad applications for primary interpretation of scans if a full workstation is (temporarily) not available [5]. Approval of the iPad for the interpretation of scans currently cannot be expected in Germany. Although the iPad 3 fulfills significant parts of the currently valid quality assurance guidelines ([Table 1]), it does not reach the required minimum diagonal screen size of 15” (LCD). The inconstant environmental conditions associated with mobile operation are also problematic even though similar concerns on the part of the FDA were able to be resolved by software solutions for ensuring appropriate contrast detectability. In principle, the iPad 3 can determine the environmental conditions via the integrated camera and adapt the display accordingly. The display of the iPad 3 can be calibrated by the software, e. g. according to DICOM – Display Function (GSDF) [18]. The possibilities for the clinical use of the iPad 3 are currently further limited in Germany since the current quality assurance guidelines recommend monitors of category B also for the “informative viewing of already diagnosed images and for diagnostic orientation” [6].

Tab. 1 Technical data and measured values for the iPad 3 compared to the monitor and the current quality assurance guidelines [6].

	requirements according to quality assurance guidelines	iPad 3	monitor
diagonal screen size (according to manufacturer specifications)	≥ 15 inches (LCD)	9.7 inches	21.3 inches
matrix size	≥ 1000 × ≥ 1000px	2048 × 1536px	1200 × 1600px
max. luminance	> 120 cd/m2 (category B) > 200 cd/m2 (category A)	410 cd/m2	135 cd/m2 [1]
max. contrast	> 40 (category B) > 100 (category A)	≈ 1000	178 [1]
functions (recommended)	windowing, zoom, MPR, double monitor	depending on the software	depending on the software
display characteristics/calibration capability	DICOM/CIE, calibration capability	DICOM GSDF (in principle, able to be implemented and calibrated on the software side [18])	DICOM GSDF (able to be calibrated)
luminance deviation	≤ ± 20 % from the image center (cat. B) ≤ ± 15 % from the image center (cat. B)	≤ ± 15 % from the image center	≤ ± 15 % from the image center[1]

¹ According to current constancy test or acceptance protocol

With the upcoming amendment of the quality assurance guidelines and the pending coming into effect of new standard DIN 6868 – 157 as a replacement for prestandard DIN V 6868 – 57, the requirements for approval and constancy testing of image display devices will increase further [19]. The introduction of room classes for the different application scenarios represents a significant change compared to prestandard DIN V 6868 – 57. Depending on the activity primarily performed in the room from “evaluation” to “repetitive reproduction of a known and evaluated finding”, the maximum allowed room illumination increases in four stages from 50 lx, ≤ 100 lx and ≤ 500 lx to ≤ 1000 lx. The area of application of the standard now also clearly includes image viewing. Moreover, the entire image display system, i. e., the image display device including the software and the settings of the operating system, must be included in the test. Finally, limits for pixel errors were introduced and measurement methods and test images were adapted to DIN 62 563 – 1. A significant new aspect in this study is the explicit requirement of a constancy test after “major changes” to the image display system including a change of the room class and location changes as regularly occur in the case of mobile applications.

In addition to these stricter normative framework conditions, additional limitations must be taken into consideration when discussing the use of the iPad 3 in radiology. The device is only available with a reflective display which further emphasizes the significance of an interpretation situation with appropriate background lighting. Moreover, data protection and access rights administration are particularly important in the case of mobile devices. Software-based measures must be taken to prevent unauthorized access to patient data in the case of public networks and theft or loss of the device. In addition, the operation of the iPad 3 via touchscreen requires a novel operating concept for image viewing and also results in practical disadvantages such as increasing contamination of the screen with potential effects on image quality and in hygiene problems particularly when used in a hospital [20] [21].

Possible application areas are currently teaching, demonstration of diagnosed images to colleagues or patients, or use as a medical information system and expert system [22] [23] [24].

The present study has different limitations. First, the number of cases and the number of readers was limited as in every reader study. For reasons of practicability and to simplify the statistical analysis, the evaluation of the scans was limited to two areas evaluated on the basis of confidence scales. A full diagnosis was not established so that differences in the interpretation scenario with respect to the detection of secondary findings could not be evaluated.

The potential limitations of the iPad 3 in relation to the detectability of low-contrast lesions were only examined in this study on the basis of early signs of cerebral infarction and not on the basis of abdominal scans or phantom data which might show more subtle differences between the two interpretation scenarios. Moreover, the selected CT scans have a relatively high interobserver variability which is known from the literature and was also confirmed in this study [11] [12] [13]. However, this limitation is to be qualified by the fact that the focus of this study was typical emergency cases in which the use of the device is of clinical interest. There is no advantage to mobile image display for routine diagnostics and this was therefore not the subject of this study.

Moreover, a comparison of the time needed for diagnosis was dispensed with since a comparable routine for both tools would only have been able to be realized with substantial iPad 3 training. Therefore, it is unclear whether the handling criticism of all readers is a fundamental problem of the operation via touchscreen or is the result of a learning curve and would be resolved with increasing experience on the part of the user [15]. In addition, data transmission is extremely important when using a mobile diagnostic tool. While data transmission via a WLAN with a corresponding network connection of the WLAN router is non-critical, the use of mobile networks can result in significant limitations of the transmission rate depending on the quality of the network, thus preventing real-time diagnosis via the mobile device.

In summary, we were able to show in the present study that the variability of the interpretation of typical emergency scans on the iPad 3 and a PACS workstation do not differ from the interobserver variability. From a clinical standpoint, there are potentially attractive application areas for the tested device primarily with respect to quick preliminary evaluation on duty or for real-time consultation of a radiological background service or experts. The results only apply to the iPad 3 used in this study. The inconsistent and difficult to standardize environmental conditions when using a mobile image display device represent further limitations as is expressed in the stricter normative framework conditions.

• References

• 1 Robinson JD. The skeptical technophile: iPad review. J Digit Imaging 2012; 25: 365-368
• 2 Choudhri AF, Carr 3rd TM, Ho C et al. Handheld device review of abdominal CT for the evaluation of acute appendicitis. J Digit Imaging 2012; 25: 492-496
• 3 Modi J, Sharma P, Earl A et al. iPhone-based teleradiology for the diagnosis of acute cervico-dorsal spine trauma. Can J Neurol Sci 2010; 37: 849-854
• 4 Savel RH, Munro CL. Scalpel, stethoscope, iPad: the future is now in the intensive care unit. Am J Crit Care 2011; 20: 275-277
• 5 FDA. FDA clears first diagnostic radiology application for mobile devices. Silver Spring, Maryland, USA: 2011 Erreichbar unter: http://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/2011/ucm242295.htm
• 6 Bundesministerium für Umwelt, Naturschutz und Reaktorsicherheit: Richtlinie zur Durchführung der Qualitätssicherung bei Röntgeneinrichtungen zur Untersuchung oder Behandlung von Menschen nach den §§ 16 und 17 der Röntgenverordnung – Qualitätssicherungs-Richtlinie (QS-RL). – vom 20.11.2003, zuletzt geändert 12.7.2011. Gemeinsames Ministerialblatt; 2011: 718
• 7 Normenausschuss Radiologie (NAR). DIN V 6868-57: Sicherung der Bildqualität in röntgendiagnostischen Betrieben- Teil 57: Abnahmeprüfung an Bildwiedergabegeräten.
• 8 Kotter E, Weisser G, Walz M et al. Die Arbeitsgemeinschaft Informationstechnologie stellt sich vor. Fortschr Röntgenstr 2011; 183: 1111-1113
• 9 Gierhake D, Weber JE, Villringer K et al. CT im Notarztwagen: technische Aspekte der prähospitalen radiologischen Schlaganfalldiagnostik vor systemischer Thrombolyse. Fortschr Röntgenstr 2013; 185: 55-59
• 10 Rosenberg C, Langner S, Rosenberg B et al. Medizinische und rechtliche Aspekte der Teleradiologie in Deutschland. Fortschr Röntgenstr 2011; 183: 804-811
• 11 Wardlaw JM, Mielke O. Early signs of brain infarction at CT: observer reliability and outcome after thrombolytic treatment – systematic review. Radiology 2005; 235: 444-453
• 12 Courtney DM, Miller C, Smithline H et al. Prospective multicenter assessment of interobserver agreement for radiologist interpretation of multidetector computerized tomographic angiography for pulmonary embolism. J Thromb Haemost 2010; 8: 533-539
• 13 Kelly AM, Patel S, Carlos RC et al. Multidetector row CT pulmonary angiography and indirect venography for the diagnosis of venous thromboembolic disease in intensive care unit patients. Acad Radiol 2006; 13: 486-495
• 14 Johnson PT, Zimmerman SL, Heath D et al. The iPad as a mobile device for CT display and interpretation: diagnostic accuracy for identification of pulmonary embolism. Emerg Radiol 2012; 19: 323-327
• 15 Mc LaughlinP, Neill SO, Fanning N et al. Emergency CT brain: preliminary interpretation with a tablet device: image quality and diagnostic performance of the Apple iPad. Emerg Radiol 2012; 19: 127-133
• 16 John S, Poh AC, Lim TC et al. The iPad tablet computer for mobile on-call radiology diagnosis? Auditing discrepancy in CT and MRI reporting . J Digit Imaging 2012; 25: 628-634
• 17 McNulty JP, Ryan JT, Evanoff MG et al. Flexible image evaluation: iPad versus secondary-class monitors for review of MR spinal emergency cases, a comparative study. Acad Radiol 2012; 19: 1023-1028
• 18 De Paepe L, De Bock P, Vanovermeire O et al. Performance evaluation of a visual display calibration algorithm for iPad. . Proc. SPIE 8319, Medical Imaging 2012; Advanced PACS-based Imaging Informatics and Therapeutic Applications, 831909
• 19 Normenausschuss Radiologie (NAR). DIN 6868-157 „Sicherung der Bildqualität in röntgendiagnostischen Betrieben – Teil 157: Abnahme- und Konstanzprüfung nach RöV an Bildwiedergabesystemen in ihrer Umgebung“.
• 20 Pramann O, Graf K, Albrecht UV. Tablet-PC im Krankenhaus: Hygienische Aspekte beachten. Dtsch Arztebl 2012; 109: A 706-A707
• 21 Buerke B, Mellmann A, Kipp F et al. Hygiene in der Radiologie: Was der Radiologe wissen sollte. Fortschr Röntgenstr 2012; 184: 1099-1109
• 22 Korbage AC, Bedi HS. The iPad in radiology resident education. J Am Coll Radiol 2012; 9: 759-760
• 23 Tait AR, Voepel-Lewis T, Brennan-Martinez C et al. Using Animated Computer-generated Text and Graphics to Depict the Risks and Benefits of Medical Treatment. Am J Med 2012; 125: 1103-1110
• 24 Franko OI, Bhola S. iPad apps for orthopedic surgeons. Orthopedics 2011; 34: 978-981