Rofo 2020; 192(01): 65-73
DOI: 10.1055/a-0977-3453
Chest
© Georg Thieme Verlag KG Stuttgart · New York

Contribution of CAD to the Sensitivity for Detecting Lung Metastases on Thin-Section CT – A Prospective Study with Surgical and Histopathological Correlation

Beitrag der computerassistierten Detektion (CAD) zur Sensitivität der präoperativen Lokalisation von Lungenmetastasen im Dünnschicht-CT – prospektive Studie mit chirurgischer und histopathologischer Korrelation
Christian Meybaum
1   Department of Radiology, ELK Berlin Chest Hospital, Germany
,
Mareike Graff
2   Dept. of Thoracic Surgery, ELK Berlin Chest Hospital, Germany
,
Eva Maria Fallenberg
3   Department of Radiology, Charité University Medicine Berlin, Germany
,
Gunda Leschber
2   Dept. of Thoracic Surgery, ELK Berlin Chest Hospital, Germany
,
Dag Wormanns
1   Department of Radiology, ELK Berlin Chest Hospital, Germany
4   Department of Clinical Radiology, University Hospital Münster, Germany
› Author Affiliations
Further Information

Correspondence

Dr. Dag Wormanns
Department of Radiology, EL Berlin Chest Hospital
Lindenberger Weg 27
13125 Berlin
Germany   
Phone: ++ 49/30/94 80 21 60   
Fax: ++ 49/30/94 80 21 93   

Publication History

14 September 2018

08 July 2019

Publication Date:
01 August 2019 (online)

 

Abstract

Purpose To assess the sensitivity of radiologists and a CAD system for the detection of lung metastases on thin-section computed tomographic (CT) scans prior to pulmonary metastasectomy.

Materials and Methods All patients scheduled for resection of lung metastases were eligible for this prospective single-center trial. 95 patients with 115 surgical procedures (pulmonary metastasectomy using thoracotomy) were included. An experienced radiologist examined the CT scans for pulmonary metastases and documented his findings. A commercial CAD system was used as a second reader; additional CAD findings were recorded. A comparison of the sensitivity of the radiologist alone and with CAD was performed. Intraoperatively surgeons tried to identify the documented lesions and resected them as well as additionally palpable lesions. The standard of reference consisted of surgery and histopathology. Follow-up information for radiologically detected lesions missed during surgery was sought.

Results 693 lesions (262 metastases) were detected radiologically or surgically, 646 of them were resected. The sensitivity of radiologists without CAD was 67.5 % for all lesions (87.4 % for metastases). CAD highly significantly (p < 0.001) increased the sensitivity to 77.9 % (92.7 %). During surgery, 143 additional lesions (19 metastases) were detected. 49 radiologically detected lesions were not palpable during surgery: 4 metastases, 5 benign lesions, and 40 lesions of an unknown nature.

Conclusion CAD provides significant additional sensitivity for detecting lung metastases using MDCT compared to the performance of a radiologist alone. CT reveals a relevant number of non-palpable pulmonary lesions.

Key Points:

  • CAD significantly increased the sensitivity for the detection of lung metastases on CT.

  • Surgical palpation of the lungs missed 8.5 % of all radiologically detected lesions.

  • CT with CAD may increase the chance for complete metastasectomy.

Citation Format

  • Meybaum C, Graff M, Fallenberg EM et al. Contribution of CAD to the Sensitivity for Detecting Lung Metastases on Thin-Section CT – A Prospective Study with Surgical and Histopathological Correlation. Fortschr Röntgenstr 2020; 192: 65 – 73


#

Zusammenfassung

Ziel Bestimmung der Sensitivität von Radiologen und eines CAD-Systems für die Detektion von Lungenmetastasen im Dünnschicht-CT vor geplanter pulmonaler Metastasenresektion.

Material und Methoden Für diese monozentrische prospektive Studie waren alle für eine pulmonale Metastasenresektion vorgesehenen Patienten geeignet. 95 Patienten mit 115 via Thorakotomie durchgeführten Operationen wurden eingeschlossen. Ein erfahrener Radiologe dokumentierte alle von ihm im CT detektierten metastasensuspekten Herde. Ein kommerzielles CAD-System wurde als second reader eingesetzt, und zusätzliche Befunde wurden ebenfalls aufgezeichnet. Die Sensitivität des Radiologen ohne und mit CAD wurde verglichen. Während der Operation wurde versucht, die präoperativ dokumentierten Befunde zu lokalisieren, und sie wurden inklusive der zusätzlich palpierten Herde reseziert. Der Referenzstandard für die Auswertung bestand in den histopathologischen Befunden der resezierten Herde. Informationen über den weiteren Verlauf radiologisch detektierter, aber chirurgisch nicht gefundener Läsionen wurden beschafft, wo immer das möglich war.

Ergebnisse 693 Herde (262 Metastasen) wurden radiologisch oder chirurgisch detektiert, 646 von ihnen wurden reseziert. Die Sensitivität der Radiologen ohne CAD war 67,5 % für alle Herde (87,4 % für Metastasen). CAD verbesserte die Sensitivität hochsignifikant auf 77,9 % (92,7 %). Intraoperativ wurden 143 zusätzliche Herde detektiert (19 Metastasen). 49 radiologisch detektierte Läsionen wurden intraoperativ nicht gefunden: 4 Metastasen, 5 benigne Herde, das biologische Verhalten der übrigen 40 Herde ist unbekannt.

Schlussfolgerung CAD erhöht die Sensitivität der Detektion von Lungenmetastasen signifikant im Vergleich zum Radiologen ohne CAD. Die CT detektiert eine relevante Zahl nicht palpabler pulmonaler Herde.

Kernaussagen:

  • CAD verbessert die Sensitivität der Detektion von Lungenmetastasen im CT signifikant.

  • Intraoperativ konnten 8,5 % aller radiologisch detektierten Lungenherde nicht palpiert werden.

  • Die intraoperative Verwendung von CT + CAD erhöht die Chance auf eine komplette Metastasenresektion.


#

Introduction

Pulmonary metastasectomy is considered a useful treatment option in particular situations during the course of some metastatic malignant tumors of extrapulmonary origin. Surgical resection of lung metastases is aimed to completely remove all metastases. Long-term follow-up data after pulmonary metastasectomy demonstrated that overall survival was improved with complete resection, compared to incomplete resection [1] [2] [3]. These data emphasize the necessity to detect all pulmonary metastases during surgery. Manual palpation of the deflated lung during open thoracotomy is regarded as the most sensitive method. However, even a thorough examination cannot preclude occasionally missing of a lung nodule, especially in central lung areas or in coexisting chronic obstructive lung disease.

Prior to surgery, knowledge of the expected number and location of lung metastases may therefore be helpful to assure their complete resection. Computed tomography (CT) is widely used for this purpose. Unfortunately, CT findings were found to be in imperfect agreement with surgical findings in published studies [4] [5]. More metastases were surgically detected than previously found on CT images. Lung nodules as small as 2 mm are clearly visible on thin-slice CT scans. The underlying problem is probably caused by the non-detection of small nodules on hundreds of CT images by the radiologist rather than by the non-depiction of nodules by the CT scanner.

The moderate performance of radiologists with respect to the detection of small lung nodules on multi-detector row CT images is well known [6] [7] [8] [9]. To overcome these deficits, computer-aided detection (CAD) systems have been developed to automatically detect lung nodules on CT scans. Used as a second or concurrent reader to the radiologist, these systems are able to increase the radiologist’s sensitivity [6] [8] [10] [11].

Video-assisted thoracoscopy (VATS) is a less invasive surgical approach for the resection of small pulmonary lesions compared to open thoracotomy [12]. However, manual palpation is impossible during VATS, and therefore its suitability for the resection of a solitary pulmonary metastasis is still under discussion due to the current lack of other methods for the reliable exclusion of multiple lung metastases [13] [14].

This prospective study was designed to separately quantify the contribution of the radiologist and of CAD used as a second reader to the preoperative detection of lung metastases. The results are compared to the surgical and histopathological findings as the standard of reference.


#

Material and methods

The responsible IRB approved the study design (reference number EA1/310/10). Written informed consent was obtained from all patients.

This prospective single-center trial was designed to separately evaluate the radiologist’s and CAD performance with respect to the detection of pulmonary metastases in comparison to the surgical and histopathological findings as the standard of reference. The contribution of CAD as a second reader was assessed. For radiologically detected lesions missed during surgery, an attempt was made to obtain follow-up data to determine whether these lesions were metastatic or benign.

Patient population

Inclusion criteria are shown in [Table 1]. Patients were consecutively enrolled from January 2011 to July 2014. Especially for lesions missed during surgery, follow-up data was collected until June 2016.

Table 1

Surgical and radiological inclusion criteria.
Tab. 1 Chirurgische und radiologische Einschlusskriterien.

surgical inclusion criteria

radiological inclusion criteria

  • complete resection of malignant primary tumor

  • exclusion of tumor relapse

  • exclusion of synchronous metastases in other regions

  • limited number of lung metastases

  • technical and functional operability

  • current chest CT scan not older than 8 weeks at the time of surgery

  • thin-slice CT with slice thickness ≤ 2 mm and reconstruction increment ≤ 1 mm

  • CAD analysis performed by an experienced radiologist

During the study period, 95 patients (49 male, 46 female) met the inclusion criteria and were enrolled in the study. A total of 8 surgical procedures were excluded from evaluation because of deviations of the expected histopathology: 4 of these were proven to be primary lung cancer, 2 neuroendocrine tumors, 1 primary lung cancer and metastases simultaneously and 1 tuberculoma.

Patient age ranged from 17 to 83 years (mean: 62.8 ± 12.9 years, median: 64.6 years). A total of 115 surgical procedures were performed in these patients: all pulmonary nodules which were either palpable by the surgeon or detected on CT and marked on the CT image by the radiologist were resected via unilateral open thoracotomy. Primary tumors were colorectal cancer (34 cases, 29.6 %), renal cell cancer (33, 28.7 %), sarcoma (24, 20.9 %), head and neck cancer (6, 5.2 %), prostate cancer (3, 2.6 %), malignant melanoma (3, 2.6 %), malignant neurogenic tumor (2, 1.7 %), bladder cancer (2, 1.7 %) and one case of uterine cancer, cervical cancer, breast cancer, tracheal cancer, lung cancer, carcinoid, fibrohistiocytic skin tumor and pancreatic cancer, respectively.


#

Scan parameter and CAD performance

A CT scan with a slice thickness of ≤ 2 mm and a reconstruction increment of ≤ 1 mm not older than 8 weeks was required. Standard-dose CT scans from various institutions were accepted if the slice thickness requirements were fulfilled. Time between CT scan and surgery ranged from 0 to 56 days (mean: 17.9 ± 16.6 days, median: 12 days).

CT images were transferred to the CAD system (LMS 6.0, Median Technologies, Valbonne, France), the CAD analysis was sent to the PACS system.

CT scans were reported by one of four board-certified radiologists with specific experience in reporting chest CT scans (6 to 20 years) using CAD as a second reader:

  1. The radiologist read the CT using the LMS user interface which provides original images and sliding thin-slab maximum intensity projection (MIP) and documented all detected nodules without knowledge of the CAD results. Lesions were labeled (RE01, RE02 and so on for the right lung, LI01, LI02 and so on for the left lung), and a screenshot was obtained for each lesion.

  2. After completion of unassisted reading, the second reader CAD findings were given. The results were reviewed by the radiologist. True-positive findings were confirmed and labeled (RE51, RE52 and so on or LI51, LI52 and so on), false-positive CAD markers were rejected and the total number per CT scan was recorded in a comment field of the CAD analysis.

  3. Finally, the radiologist filled out a study documentation sheet containing information on lesion label, lung segment, lesion size, and a lung scheme in three views on which the radiologist marked the approximate position of all lesions ([Fig. 1]).

Zoom Image
Fig. 1 Study documentation sheet with notes by radiologist and surgeon.

Abb. 1 Studiendokumentationsbogen mit Eintragungen durch Radiologen und Chirurgen.

The CAD results were sent to the PACS and together with the CT images and the study documentation sheet, were available to the surgeon in the operating room.

The reporting radiologist was unaware of the surgical and histopathological findings that were obtained after completion of the CAD analysis. During the evaluation phase of the study, the CAD system was set to recalculate the CAD analysis on the same CT dataset, and an experienced chest radiologist (DW, 20 years professional experience) reviewed the CAD results to decide if CAD also detected the lesions that had been marked by the radiologist during the unassisted first part of the reading process. The CAD algorithm remained unchanged for the duration of the study. Moreover, in the same reading session the radiologist attempted to identify lesions additionally resected by the surgeon without having been detected preoperatively on the CT scans. If a lesion could retrospectively be identified on the CT images, the maximum lesion diameter was measured on the CT scan and was recorded. If a lesion could not be identified even retrospectively, the lesion size was recorded as missing.

The information as to whether each nodule was attached to the pleura was gathered at the time of the compilation of the study. The surgical free-text comments on the study documentation sheets were assessed, and if necessary, the CT images and the operation reports were reviewed.


#

Surgical and histopathological procedure

Pulmonary metastasectomy was conducted by one of four board-certified staff thoracic surgeons who regularly perform at least 300 thoracic surgical procedures per year.

Open thoracotomy was performed to explore the deflated lung manually. Consequently, the lung was palpated in a standardized fashion segment by segment, starting with the upper lobe down to the lower lobe segments before laser resection started with the first nodule in the craniocaudal direction.

During metastasectomy, the surgeon matched the preoperatively detected nodules to the intraoperatively palpated and resected lesions using the study documentation sheet. If a lesion was unidentifiable, the surgeon made a remark on the study documentation sheet. All additionally resected lesions were documented as well.

The necessity of this documentation sheet was a result of a prior retrospective study conducted by our department [15]. It bridged the gap between the radiologist, the surgeon and the pathologist and ensured that the radiologically detected lesion was equal to the one being resected.

All palpable intrapulmonary lesions were resected by laser. Each specimen was labeled and separately submitted to histopathological analysis. The label was recorded on the study documentation sheet. The pathologist reported the histopathological findings for each specimen separately with reference to the given label.


#

Follow-up

An attempt was made to obtain follow-up information for radiologically detected lesions missed during surgery. An attempt was also made to retrieve subsequent CT scans acquired either in-house or at external facilities. A lesion was considered benign if no lesion growth was documented for at least two years [16] [17]. Lesions demonstrating growth at follow-up of any duration were regarded as metastases. Otherwise, the lesion was counted as lost to follow-up.


#

Statistics

Statistical analysis was performed with Microsoft Excel 2010 (Microsoft Corp., Redmond, Washington, USA) and SPSS 15.0 (SPSS Inc., Chicago, Illinois, USA). One-way analysis of variance was used to test for sensitivity differences between radiologists. One-sided McNemar-Test was used to test for sensitivity differences between radiologists without and with CAD. P-values < 0.05 were considered to indicate statistical significance. The sensitivity and 95 % confidence intervals were calculated. In the given study design, the specificity could not be calculated on a per-nodule or per-patient basis. Both values were calculated on a per-lobe basis instead. The number of false-positive CAD findings per CT scan was given as documented in the comment section of the CAD report.

The main unit for all analyses dealing with metastases consisted of all resected nodules in which metastasis was confirmed at histopathology. For all analyses dealing with all lesions, the main unit consisted of all resected lesions independent of the histopathological result. In analyses dealing with radiologically detected lesions missed during surgery, the main unit consisted of all lesions detected on CT either by the radiologist or by CAD that were not resected during surgery. A metastatic nature of the lesion was assumed when growth consistent with malignancy was detected on follow-up CT scans.

From a retrospective pilot study [15], estimates of mean sensitivity and standard deviation were known. Assuming a sensitivity of the radiologist alone of 0.7 with a standard deviation of 0.2, and having the null hypothesis mean at 0.7 with a superiority margin of 0.05, the required study size calculates to 99 cases to test the superiority of radiologist with CAD over radiologist alone with a statistical power of 0.8 with a type I error rate of 5 %.


#
#

Results

Number and size of detected lesions

The number of lesions per case ranged from 1 to 18 (mean: 6.0 ± 3.8 lesions, median: 5 lesions). A total of 693 lesions were detected either with CT or surgically. 47 lesions were detected on CT but were not found during surgery. These lesions have not been included in the main unit and are described separately at the end of the results section. The remaining 646 lesions were resected, and consequently histopathology was available. Of these, 262 (37.8 %) were found to be metastases at histopathology. 550 (79.4 %) lesions were detected on CT.

The size distribution of the nodules is given in [Fig. 2]. The largest diameter was measured in millimeters. The maximum lesion diameter ranged from 1 to 70 mm (mean: 8.0 ± 9.4 mm, median: 4.0 mm). Only 20.0 % of all lesions were larger than 10 mm. Most lesions were very small, and the majority of lesions with a size smaller than 7 mm were non-metastatic as shown in [Fig. 3].

Zoom Image
Fig. 2 Size distribution of pulmonary nodules (longest axial diameter in mm).

Abb. 2 Größenverteilung der pulmonalen Herde (längster axialer Durchmesser in mm).
Zoom Image
Fig. 3 Nodule size and percentage of metastases.

Abb. 3 Herdgröße und Prozentsatz von Metastasen.

#

Radiological detection of lung lesions

Radiologists without CAD detected a total of 436 lesions out of 646 resected lesions, thus resulting in an overall sensitivity of 67.5 %. The sensitivity rose to 87.4 % when considering only metastases (229 of 262 metastases). All radiologists involved in the study performed equally; one-way analysis of variance found no difference in sensitivity (all lesions: p = 0.878; metastases: p = 0.970).

The sensitivity of CAD as a stand-alone system was 53.6 % for all lesions and 61.8 % for metastases. The number of false positive findings per CT study ranged from 0 to 99 with an average of 13.6 ± 30.9. CAD added 10.4 % sensitivity for all lesions (sensitivity of radiologist with CAD is 77.9 %) and 5.3 % sensitivity for metastases (sensitivity of radiologist with CAD is 92.7 %). These differences were highly significant with p < 0.001 for all nodules and for metastases.

More details regarding sensitivity are presented in [Table 2]. Moreover, it demonstrates that there are only negligible differences in the performance of radiologists and CAD when evaluating only non-pleural lesions compared to all lesions. Hence, the information as to whether a lesion was pleural was no longer taken into account in subsequent analyses.

Table 2

Sensitivities for the detection of all lesions, metastases and non-pleural metastases by radiologist and CAD combined, radiologist alone, CAD alone and palpation, (numbers in parentheses are 95 % confidence intervals).
Tab. 2 Sensitivität der Detektion aller Läsionen, aller Metastasen und aller nicht-pleuraler Metastasen durch Radiologen und CAD kombiniert, Radiologen allein, CAD allein, und chirurgische Palpation (Angaben in Klammern sind 95 %-Konfidenzintervalle).

all lesions (%)

all metastases (%)

non-pleural metastases (%)

radiologist + CAD

77.9 (74.7–81.1)

92.7 (89.6–95.9)

93.5 (90.4–96.6)

radiologist alone

67.5 (63.9–71.1)

87.4 (83.4–91.4)

88.2 (84.1–92.2)

CAD alone

53.6 (49.7–57.4)

61.8 (55.9–67.8)

63.3 (57.2–69.3)

palpation

99.5 (99.0–100.1)

99.2 (98.2–100.3)

99.2 (98.0–100.3)

The sensitivity was highly dependent on lesion size. Both radiologists and CAD performed poorly in the case of very small lesions and were nearly perfect in lesions with a diameter of at least 6 mm, as demonstrated in [Table 3].

Table 3

Sensitivity of radiologist, CAD, and both combined for the detection of all lesions and metastases (in parentheses: 95 % confidence interval).
Tab. 3 Sensitivität von Radiologen, CAD, und beidem kombiniert für die Detektion von allen Läsionen und Metastasen (Angaben in Klammern sind 95 %-Konfidenzintervalle).

lesion size (mm)

sensitivity

radiologist

CAD

radiologist with CAD

all lesions

1–2

3–4

5–6

7–8

9–10

> 10

0.30 (0.23–0.37)

0.65 (0.58–0.72)

0.78 (0.69–0.87)

0.93 (0.86–1.00)

1.00

0.97 (0.94–1.00)

0.20 (0.14–0.26)

0.65 (0.58–0.72)

0.79 (0.70–0.88)

0.77 (0.66–0.88)

0.82 (0.69–0.96)

0.50 (0.42–0.59)

0.35 (0.28–0.42)

0.88 (0.83–0.93)

0.94 (0.89–0.99)

1.00

1.00

0.98 (0.96–1.01)

metastases

1–2

3–4

5–6

7–8

9–10

> 10

0.47 (0.23–0.72)

0.68 (0.52–0.83)

0.81 (0.66–0.97)

0.94 (0.87–1.02)

1.00

0.97 (0.93–1.00)

0.37 (0.13–0.61)

0.51 (0.34–0.68)

0.85 (0.71–1.00)

0.83 (0.71–0.96)

0.82 (0.67–0.97)

0.52 (0.43–0.61)

0.53 (0.28–0.77)

0.81 (0.68–0.94)

0.96 (0.89–1.04)

1.00

1.00

0.98 (0.96–1.01)

The specificity on a per-lobe basis was 98.2 % for the radiologist alone, 88.7 % for CAD alone, and 99.5 % for the radiologist with CAD.

Of the 143 resected lesions missed by the radiologist and CAD, 75 could retrospectively be detected with the information provided by the surgeon. 68 lesions could not be identified on the CT scans even in retrospect, among them 3 (5 %) histologically proven metastases; 9 of these lesions were reported by the surgeon to be pleural. The average size of the missed lesions that could be retrospectively identified on CT was 2.6 mm (range: 1–12 mm, median: 2.0 mm).


#

Surgical detection of lung lesions

644 (99.7 %) of 646 surgically resected lesions were palpable during surgery. Two more lesions (0.3 %) were resected based on their location on CT although they were not palpable. Both were found to be metastases at histopathology. Of all lesions, 503 (77.9 %) were detected by the radiologist with CAD, including 243 metastases. An additional 143 lesions were palpated and resected during surgery. Of these additional lesions, 19 (13.3 %) were metastatic. Details regarding the sensitivity of palpation during surgery for detecting lesions and metastases are presented in [Table 2].

Compared to the radiological findings, all 12 (100 %) patients with a solitary lesion on CT were found to have only one nodule during surgery. 9 of these lesions were metastatic (75 %). If CT demonstrated more than one lesion, more lesions than expected on CT were found in 59 of the 103 surgical procedures (57.3 %), but only in two cases (1.9 %) were there more histologically proven metastases.

A total of 49 lesions could not be palpated. Two non-palpable lesions were resected based on their location on the CT scan. Both were found to be metastatic at histopathology. The other 47 lesions remained in situ.


#

Lesions missed during surgery

Radiologists with CAD detected 47 (6.8 %) lesions that were not found during surgery and were not resected. For 20 (42.6 %) lesions no follow-up information could be obtained. They were regarded as lost to follow-up. For the remaining lesions, the available follow-up ranged from 69 to 1843 days (mean: 509 ± 483 days, median: 258 days). Using the criteria described in the methods section, sufficient follow-up was available in 7 (14.9 %) lesions. The remaining 40 (85.1 %) lesions were also regarded as lost to follow-up. Of the 7 lesions with sufficient follow-up, 2 (29 %; 95 % confidence interval: –8 % to 65 %) demonstrated growth at subsequent CT scans and were therefore regarded as metastatic. The other 5 lesions were considered benign with a follow-up of 1032 to 1843 days. Of the two metastatic lesions, one was detected only by CAD and was missed by the radiologist ([Fig. 4]), while the other was detected by the radiologist but missed by CAD.

Zoom Image
Fig. 4 Metastasis in left upper lobe, overlooked by the radiologist but detected by CAD, missed during surgery and found to be metastatic on subsequent CT scans. Axial maximum-intensity projection with 8-mm slice thickness.

Abb. 4 Metastase im linken Oberlappen, vom Radiologen übersehen, von CAD detektiert, intraoperativ nicht gefunden, war progredient in späteren CT-Verlaufskontrollen. Axiale Maximum-Intensitäts-Projektion mit 8 mm Schichtdicke.

If the proportion of metastases found in the cases with sufficient follow-up (29 %) is projected to the population of all 47 lesions missed by the surgeon, it can be roughly estimated that 13 of the 47 nodules might be metastases missed by the surgeon.


#
#

Discussion

Previous studies on the detection of lung metastases yielded a moderate sensitivity of CT. In an older study using single-slice spiral CT with a 3 and 5-mm slice thickness, Diederich reported 64 % sensitivity for all pulmonary lesions and 77 % for metastases [4]. In 2008, Ludwig [5] compared the number of lung metastases detected on CT and during surgery. Details of the CT technique were not given, but it can be supposed that they were not consistently thin-slice as the data was from 1998 to 2003. This study found more metastases during surgery than on CT in 27.2 % of the cases, a situation that was rare in our study. In our study the sensitivity of CT was higher. This may be explained by the subtler CT technique with CAD. None of the previously published studies used CAD.

It could be expected that CAD increases the sensitivity of the detection of pulmonary metastases. Our study found a significant increase in the sensitivity for the detection of lung metastases when using CAD as a second reader. The detection of unexpected metastases during surgery was rare and occurred in less than 2 % of all cases. Moreover, a relevant proportion of lesions detected on CT but missed during surgery represents metastases. However, the absolute number of these lesions in which sufficient follow-up information was available was small. In surgical studies not using CAD, CT performance was markedly inferior compared to our data, with lung metastases missed on CT in up to 36 % of cases [5] [18] [19] [20] [21]. This is an indirect argument for the use of CAD in addition to the review of the CT scan by the radiologist.

The specificity of the CAD system with more than 13 false-positives per CT scan was only moderate compared to more recent systems with less than 5 false-positives and the tendency to higher sensitivities [22] [23] [24] [25] [26]. However, it proved to be useful since it significantly added sensitivity to the performance of the radiologist alone.

Not surprisingly, manual palpation was found to be the most sensitive method in our study which is consistent with other studies [18] [19] [20] [21]. However, a higher proportion of non-palpable lesions was metastatic in our study, thus emphasizing the potential benefit of CT with CAD as guidance for the surgeon.

Radiologists tend to detect typical metastases while ignoring atypical findings. 50 % of all lesions detected by radiologists were metastatic. Lesions not detected by the radiologist but detected by CAD were less likely to be metastatic (15.2 %). Lesions additionally detected by CAD tend to have CT-morphological features less typical for metastases. Nevertheless, CAD was useful even in these atypical lesions as illustrated in [Fig. 5].

Zoom Image
Fig. 5 Largest metastasis missed by the radiologist but detected by CAD. Interpreted as scar at first read, detected by CAD and considered a positive second reader CAD finding by the radiologist: 31-mm subpleural lesion in the middle lobe. Adjacent scars due to prior surgery. Primary tumor was colorectal cancer. Screenshot of the CAD user interface is shown.

Abb. 5 Größte vom Radiologen übersehene Metastase, die vom CAD detektiert wurde. Initial als Narbe interpretiert, wurde es als vom Radiologen als positiver Befund von CAD als second reader gewertet: 31 mm großer subpleuraler Herd im Mittellappen. Benachbart narbige Veränderungen nach früherer Metastasenchirurgie. Primärtumor war ein kolorektales Karzinom. Dargestellt ist die CAD-Nutzeroberfläche.

The detection of pleural metastases is more difficult than the detection of intrapulmonary metastases for both radiologists and CAD. This limitation is compensated by the easy detection of pleural metastases during surgery. CT can be regarded as complementary to intraoperative inspection and palpation since CT is able to easily detect small metastases deep inside the lung parenchyma which may be more challenging to detect during surgery.

Use of CAD increases the number of preoperatively known lesions and therefore has the potential to harm the patient by increasing the number of unnecessarily resected benign lesions. On the other hand, palpation during surgery results in even more additional lesions ([Fig. 6]). The percentage of additional metastases found by CAD was similar to the percentage of additional metastases found by manual palpation during surgery. Therefore, resection of lesions additionally detected by CAD is as beneficial to the patient as resection of additionally palpated lesions.

Zoom Image
Fig. 6 Number of findings and number of metastases detected by radiologist alone (upper box), additionally by CAD (middle box) and by manual palpation (lower box). First number: number of detected lesions. Number in parentheses: number of detected metastases. Percentage: percentage of metastases in all detected lesions. § estimated values (see text for details).

Abb. 6 Anzahl der Befunde und Anzahl der Metastasen, die durch den Radiologen allein (oberer Kasten), zusätzlich durch CAD (mittlerer Kasten) und intraoperativ durch Palpation (unterer Kasten) detektiert wurden. Erste Zahl: Anzahl der detektierten Läsionen. Zahl in Klammern: Anzahl der detektierten Metastasen. Prozentzahlen: Anteil der Metastasen an allen detektierten Läsionen. § Schätzwert (zu Details siehe Text).

Our study has certain limitations:

  • Since the presented study is a single-center study, the results may not be applicable to other centers with different levels of expertise resulting in higher or lower sensitivities for radiologists and surgeons. The study was performed in a “real world” setting, thus involving different radiologists and surgeons which may result in an inhomogeneous performance which is not quantifiable in our study. Moreover, the radiologist’s results with CAD are susceptible to the performance of the CAD system. In our study a single CAD system with its specific strengths and weaknesses regarding the detection of pulmonary nodules was used.

  • Even if a resected lesion was found to be non-metastatic at histopathology, it may well be possible that the lesion represented a metastasis that was sterilized by prior systemic treatment. Therefore, the number of metastases in our dataset may be underestimated. However, the study results taking into account all lesions did not substantially differ from the results for metastases.

  • No histopathological proof was available for lesions missed by surgery, and the percentage of lesions in which sufficient follow-up could be obtained was small.

  • Furthermore, a methodical limitation is a certain heterogeneity of imaging data in terms of CT vendor, slice thickness, reconstruction increment and reconstruction kernel because external CT scans from different institutions were accepted as long as they fulfilled the defined minimum requirements.

Clinical relevance
  • We found that CAD used as the second reader increases the sensitivity for the detection of lung metastases on CT scans.

  • The combination of CT with CAD and intraoperative palpation of the deflated lung may increase the chance for complete resection of lung metastases.

  • CT can therefore be used as guidance during surgery to achieve complete resection of lung metastases.


#
#

Conflict of Interest

The authors declare that they have no conflict of interest.

  • References

  • 1 Pastorino U, Buyse M, Friedel G. et al. Long-term results of lung metastasectomy: prognostic analyses based on 5206 cases. The Journal of Thoracic and Cardiovascular Surgery 1997; 113: 37-49
  • 2 Casiraghi M, de Pas T, Maisonneuve P. et al. A 10-year single-center experience on 708 lung metastasectomies: the evidence of the “international registry of lung metastases”. Journal of Thoracic Oncology 2011; 6: 1373-1378
  • 3 Pfannschmidt J, Egerer G, Bischof M. et al. Surgical intervention for pulmonary metastases. Deutsches Arzteblatt international 2012; 109: 645-651
  • 4 Diederich S, Semik M, Lentschig MG. et al. Helical CT of pulmonary nodules in patients with extrathoracic malignancy: CT-surgical correlation. American Journal of Roentgenology 1999; 172: 353-360
  • 5 Ludwig C, Cerinza J, Passlick B. et al. Comparison of the number of pre-, intra- and postoperative lung metastases. European Journal of Cardio-Thoracic Surgery 2008; 33: 470-472
  • 6 Das M, Muhlenbruch G, Mahnken AH. et al. Small pulmonary nodules: effect of two computer-aided detection systems on radiologist performance. Radiology 2006; 241: 564-571
  • 7 Wormanns D, Ludwig K, Beyer F. et al. Detection of pulmonary nodules at multirow-detector CT: effectiveness of double reading to improve sensitivity at standard-dose and low-dose chest CT. European radiology 2005; 15: 14-22
  • 8 Rubin GD, Lyo JK, Paik DS. et al. Pulmonary nodules on multi-detector row CT scans: performance comparison of radiologists and computer-aided detection. Radiology 2005; 234: 274-283
  • 9 Lee JW, Goo JM, Lee HJ. et al. The potential contribution of a computer-aided detection system for lung nodule detection in multidetector row computed tomography. Investigative Radiology 2004; 39: 649-655
  • 10 Saba L, Caddeo G, Mallarini G. Computer-aided detection of pulmonary nodules in computed tomography: analysis and review of the literature. Journal of Computer Assisted Tomography 2007; 31: 611-619
  • 11 Beyer F, Zierott L, Fallenberg EM. et al. Comparison of sensitivity and reading time for the use of computer-aided detection (CAD) of pulmonary nodules at MDCT as concurrent or second reader. Eur Radiol 2007; 17: 2941-2947
  • 12 Servais E, Swanson SJ. Thoracoscopic Management of Pulmonary Metastases. Thoracic Surgery Clinics 2016; 26: 91-97
  • 13 Dong S, Zhang L, Li W. et al. Evaluation of video-assisted thoracoscopic surgery for pulmonary metastases: a meta-analysis. PLoS one 2014; 9: e85329
  • 14 National Collaborating Centre for Cancer. Diagnosis and management of metastatic malignant disease of unknown primary origin. NICE National Guidelines No 104. Cardiff: 2010
  • 15 Schramm A, Wormanns D, Leschber G. et al. Reliability of a computer-aided detection system in detecting lung metastases compared to manual palpation during surgery. Interact Cardiovasc Thorac Surg 2011; 12: 20-23
  • 16 Yankelevitz DF, Henschke CI. Does 2-year stability imply that pulmonary nodules are benign?. Am J Roentgenol 1997; 168: 325-328
  • 17 MacMahon H, Austin JHM, Gamsu G. et al. Guidelines for management of small pulmonary nodules detected on CT scans: a statement from the Fleischner Society. Radiology 2005; 237: 395-400
  • 18 Christe A, Leidolt L, Huber A. et al. Lung cancer screening with CT: evaluation of radiologists and different computer assisted detection software (CAD) as first and second readers for lung nodule detection at different dose levels. Eur J Radiol 2013; 82: e873-e878
  • 19 Cerfolio RJ, Bryant AS, McCarty TP. et al. A prospective study to determine the incidence of non-imaged malignant pulmonary nodules in patients who undergo metastasectomy by thoracotomy with lung palpation. Ann Thorac Surg 2011; 91: 1696-1700
  • 20 Althagafi KT, Alashgar OA, Almaghrabi HS. et al. Missed pulmonary metastasis. Asian Cardiovasc Thorac Ann 2014; 22: 183-186
  • 21 Heaton TE, Hammond WJ, Farber BA. et al. A 20-year retrospective analysis of CT-based pre-operative identification of pulmonary metastases in patients with osteosarcoma: A single-center review. J Pediatr Surg 2017; 52: 115-119
  • 22 Lu L, Tan Y, Schwartz LH. et al. Hybrid detection of lung nodules on CT scan images. Med Phys 2015; 42: 5042-5054
  • 23 Setio AA, Jacobs C, Gelderblom J. et al. Automatic detection of large pulmonary solid nodules in thoracic CT images. Med Phys 2015; 42: 5642-5653
  • 24 Yoshida Y, Sakane T, Isogai J. et al. Computer-assisted detection of metastatic lung tumors on computed tomography. Asian Cardiovasc Thorac Ann 2019; 27: 199-207
  • 25 Vassallo L, Traverso A, Agnello M. et al. A cloud-based computer-aided detection system improves identification of lung nodules on computed tomography scans of patients with extra-thoracic malignancies. Eur Radiol 2019; 29: 144-152
  • 26 Krüger M, Zinne N, Shin H. et al. Minimally invasive chest surgery. Is palpation control still necessary with modern computed tomography?. Chirurg 2016; 87: 136-143

Correspondence

Dr. Dag Wormanns
Department of Radiology, EL Berlin Chest Hospital
Lindenberger Weg 27
13125 Berlin
Germany   
Phone: ++ 49/30/94 80 21 60   
Fax: ++ 49/30/94 80 21 93   

  • References

  • 1 Pastorino U, Buyse M, Friedel G. et al. Long-term results of lung metastasectomy: prognostic analyses based on 5206 cases. The Journal of Thoracic and Cardiovascular Surgery 1997; 113: 37-49
  • 2 Casiraghi M, de Pas T, Maisonneuve P. et al. A 10-year single-center experience on 708 lung metastasectomies: the evidence of the “international registry of lung metastases”. Journal of Thoracic Oncology 2011; 6: 1373-1378
  • 3 Pfannschmidt J, Egerer G, Bischof M. et al. Surgical intervention for pulmonary metastases. Deutsches Arzteblatt international 2012; 109: 645-651
  • 4 Diederich S, Semik M, Lentschig MG. et al. Helical CT of pulmonary nodules in patients with extrathoracic malignancy: CT-surgical correlation. American Journal of Roentgenology 1999; 172: 353-360
  • 5 Ludwig C, Cerinza J, Passlick B. et al. Comparison of the number of pre-, intra- and postoperative lung metastases. European Journal of Cardio-Thoracic Surgery 2008; 33: 470-472
  • 6 Das M, Muhlenbruch G, Mahnken AH. et al. Small pulmonary nodules: effect of two computer-aided detection systems on radiologist performance. Radiology 2006; 241: 564-571
  • 7 Wormanns D, Ludwig K, Beyer F. et al. Detection of pulmonary nodules at multirow-detector CT: effectiveness of double reading to improve sensitivity at standard-dose and low-dose chest CT. European radiology 2005; 15: 14-22
  • 8 Rubin GD, Lyo JK, Paik DS. et al. Pulmonary nodules on multi-detector row CT scans: performance comparison of radiologists and computer-aided detection. Radiology 2005; 234: 274-283
  • 9 Lee JW, Goo JM, Lee HJ. et al. The potential contribution of a computer-aided detection system for lung nodule detection in multidetector row computed tomography. Investigative Radiology 2004; 39: 649-655
  • 10 Saba L, Caddeo G, Mallarini G. Computer-aided detection of pulmonary nodules in computed tomography: analysis and review of the literature. Journal of Computer Assisted Tomography 2007; 31: 611-619
  • 11 Beyer F, Zierott L, Fallenberg EM. et al. Comparison of sensitivity and reading time for the use of computer-aided detection (CAD) of pulmonary nodules at MDCT as concurrent or second reader. Eur Radiol 2007; 17: 2941-2947
  • 12 Servais E, Swanson SJ. Thoracoscopic Management of Pulmonary Metastases. Thoracic Surgery Clinics 2016; 26: 91-97
  • 13 Dong S, Zhang L, Li W. et al. Evaluation of video-assisted thoracoscopic surgery for pulmonary metastases: a meta-analysis. PLoS one 2014; 9: e85329
  • 14 National Collaborating Centre for Cancer. Diagnosis and management of metastatic malignant disease of unknown primary origin. NICE National Guidelines No 104. Cardiff: 2010
  • 15 Schramm A, Wormanns D, Leschber G. et al. Reliability of a computer-aided detection system in detecting lung metastases compared to manual palpation during surgery. Interact Cardiovasc Thorac Surg 2011; 12: 20-23
  • 16 Yankelevitz DF, Henschke CI. Does 2-year stability imply that pulmonary nodules are benign?. Am J Roentgenol 1997; 168: 325-328
  • 17 MacMahon H, Austin JHM, Gamsu G. et al. Guidelines for management of small pulmonary nodules detected on CT scans: a statement from the Fleischner Society. Radiology 2005; 237: 395-400
  • 18 Christe A, Leidolt L, Huber A. et al. Lung cancer screening with CT: evaluation of radiologists and different computer assisted detection software (CAD) as first and second readers for lung nodule detection at different dose levels. Eur J Radiol 2013; 82: e873-e878
  • 19 Cerfolio RJ, Bryant AS, McCarty TP. et al. A prospective study to determine the incidence of non-imaged malignant pulmonary nodules in patients who undergo metastasectomy by thoracotomy with lung palpation. Ann Thorac Surg 2011; 91: 1696-1700
  • 20 Althagafi KT, Alashgar OA, Almaghrabi HS. et al. Missed pulmonary metastasis. Asian Cardiovasc Thorac Ann 2014; 22: 183-186
  • 21 Heaton TE, Hammond WJ, Farber BA. et al. A 20-year retrospective analysis of CT-based pre-operative identification of pulmonary metastases in patients with osteosarcoma: A single-center review. J Pediatr Surg 2017; 52: 115-119
  • 22 Lu L, Tan Y, Schwartz LH. et al. Hybrid detection of lung nodules on CT scan images. Med Phys 2015; 42: 5042-5054
  • 23 Setio AA, Jacobs C, Gelderblom J. et al. Automatic detection of large pulmonary solid nodules in thoracic CT images. Med Phys 2015; 42: 5642-5653
  • 24 Yoshida Y, Sakane T, Isogai J. et al. Computer-assisted detection of metastatic lung tumors on computed tomography. Asian Cardiovasc Thorac Ann 2019; 27: 199-207
  • 25 Vassallo L, Traverso A, Agnello M. et al. A cloud-based computer-aided detection system improves identification of lung nodules on computed tomography scans of patients with extra-thoracic malignancies. Eur Radiol 2019; 29: 144-152
  • 26 Krüger M, Zinne N, Shin H. et al. Minimally invasive chest surgery. Is palpation control still necessary with modern computed tomography?. Chirurg 2016; 87: 136-143

Zoom Image
Fig. 1 Study documentation sheet with notes by radiologist and surgeon.

Abb. 1 Studiendokumentationsbogen mit Eintragungen durch Radiologen und Chirurgen.
Zoom Image
Fig. 2 Size distribution of pulmonary nodules (longest axial diameter in mm).

Abb. 2 Größenverteilung der pulmonalen Herde (längster axialer Durchmesser in mm).
Zoom Image
Fig. 3 Nodule size and percentage of metastases.

Abb. 3 Herdgröße und Prozentsatz von Metastasen.
Zoom Image
Fig. 4 Metastasis in left upper lobe, overlooked by the radiologist but detected by CAD, missed during surgery and found to be metastatic on subsequent CT scans. Axial maximum-intensity projection with 8-mm slice thickness.

Abb. 4 Metastase im linken Oberlappen, vom Radiologen übersehen, von CAD detektiert, intraoperativ nicht gefunden, war progredient in späteren CT-Verlaufskontrollen. Axiale Maximum-Intensitäts-Projektion mit 8 mm Schichtdicke.
Zoom Image
Fig. 5 Largest metastasis missed by the radiologist but detected by CAD. Interpreted as scar at first read, detected by CAD and considered a positive second reader CAD finding by the radiologist: 31-mm subpleural lesion in the middle lobe. Adjacent scars due to prior surgery. Primary tumor was colorectal cancer. Screenshot of the CAD user interface is shown.

Abb. 5 Größte vom Radiologen übersehene Metastase, die vom CAD detektiert wurde. Initial als Narbe interpretiert, wurde es als vom Radiologen als positiver Befund von CAD als second reader gewertet: 31 mm großer subpleuraler Herd im Mittellappen. Benachbart narbige Veränderungen nach früherer Metastasenchirurgie. Primärtumor war ein kolorektales Karzinom. Dargestellt ist die CAD-Nutzeroberfläche.
Zoom Image
Fig. 6 Number of findings and number of metastases detected by radiologist alone (upper box), additionally by CAD (middle box) and by manual palpation (lower box). First number: number of detected lesions. Number in parentheses: number of detected metastases. Percentage: percentage of metastases in all detected lesions. § estimated values (see text for details).

Abb. 6 Anzahl der Befunde und Anzahl der Metastasen, die durch den Radiologen allein (oberer Kasten), zusätzlich durch CAD (mittlerer Kasten) und intraoperativ durch Palpation (unterer Kasten) detektiert wurden. Erste Zahl: Anzahl der detektierten Läsionen. Zahl in Klammern: Anzahl der detektierten Metastasen. Prozentzahlen: Anteil der Metastasen an allen detektierten Läsionen. § Schätzwert (zu Details siehe Text).