Comparison of Scientific Publications from Three Different Clinical Disciplines of German Universities

Abstract

Background Scientific publications are important for the advancement in medicine. Surgical disciplines including cardiac surgery are frequently considered not scientifically leading. However, a specific comparison between surgical and nonsurgical disciplines has not yet been performed. We thus compared scientific output of German departments of cardiac surgery with nonsurgical cardiology departments and surgical departments not addressing the heart (general surgery) of 34 universities in Germany.

Methods For each university, the personnel working at the different departments were identified on the internet homepage in 2014. We searched for publications of these persons in 2011 to 2013 on PubMed, identified author position, coauthors, and type of article, as well as journal impact factor (JIF).

Results There were 931 academic persons in cardiac surgery, 1,486 in general surgery, and 1,814 in cardiology with 12,096 publications related to these persons on PubMed. Cardiology published most manuscripts, including manuscripts from research conducted (first author), initiated (senior author), or both. Cardiac surgery had the least publications and had fewer authors from other departments or institutions. The average JIF was higher in cardiology compared with the two surgical disciplines. However, relating the number of publications to the number of employees in the departments, the differences were no longer apparent.

Conclusion We conclude that the number of publications in German universities appears to be a function of the number of academic personnel and not of the discipline. The lower JIFs in surgery may be due to the smaller surgical fields and/or due to less high impact interdisciplinary/interinstitutional publications in surgery.

Keywords

scientific output - university - research

Introduction

Scientific investigation and publication play an important role to advance human knowledge and are the key feature of our understanding of universities. Every medical field depends on scientific research to develop and advance patient care and treatment. While nonuniversity hospitals focus primarily on patient care, medical departments at universities need to be committed to research. Scientific research necessitates publication to share and discuss knowledge. Thus, published manuscripts may be considered the most visible signs for research activity.[1] [2]

Cardiac surgery is a relatively young medical specialty and has mainly developed in the past 60 years.[3] As a consequence, research has focused on the development of new as well as improved procedures.[4] [5] Basic research has led, for instance, to the development of cardiac protection strategies prolonging time spans of ischemia tolerance thereby increasing the surgeons' ability to perform even complex technical operations properly.[6] Both activities should lead to the publication of ample scientific manuscripts. However, compared with cardiology, cardiac surgery is frequently not considered scientifically leading.[3] [7] It is possible that the rather young age of the discipline or the often smaller size of the department contributes to this perception. Thus far, these considerations have not yet been systematically investigated.

We, therefore, compared in this analysis whether differences in scientific output (measured as manuscripts published) at German universities exist among departments of cardiac surgery (young and small), general surgery (established and larger), and cardiology (nonsurgical and large).

Methods

Data Collection

Medical and scientific personnel, within this manuscript referred to as academic personnel, of 34 German university departments of general surgery, cardiology, and cardiac surgery were identified on the individual departments' Web sites. For cities with more than one university, all disciplines were combined for this analysis to avoid distortions due to potential unclear associations. Subsequently, all publications authored by these persons in the years 2011 to 2013 were located in the National Center for Biotechnology Information database PubMed and included independent of positions of the authors. When several dates were given for a publication of the manuscript, the earliest was used. Furthermore, the position of the authors on the manuscript and the total number of authors as well as the journal the manuscript was published were identified. As in publication databases, manuscripts were categorized as “original research,” “review,” “case report,” or “letter or editorial” following the categories given in the database. The journal impact factor (JIF) for each journal was taken from the Web of Science database and the values for 2014 were used. Manuscripts were considered conducted in the department when the first author was located there. Senior authorship out of a department was interpreted as initiation of the research and manuscripts with first and senior authors from the department were considered as initiated and conducted there.

Calculations

The categories “original research,” “reviews,” “case reports,” and “letter or editorial” are mostly not considered equal. This is reflected in the rules of German universities used for performance-oriented distribution of funds. As an example, case reports are neglected at some universities, but at others, they are counted as full articles. Original research is normally accepted with high value, while letters and editorials are mostly not included. We thus aimed to reflect these rules in evaluating the research. Therefore, as a factor for original scientific content of a publication, publications were weighted with factor 1 for original research, 0.7 for reviews, 0.5 for case reports, and 0.1 for letters or editorials. These factors were additionally used for adaptation of the JIF to type of publication.

Calculation of the JIF for the position of the authors was performed according to the suggestions of the German Research Foundation suggested in 2004 for a transition phase.[8] [9] Single author publications were counted completely for this author, and dual author contributions were split evenly between both authors. With three or more authors, one-third of the JIF was assigned to first and senior authors each and one-third was split among all authors between these. To adapt for both author position and type of publication both weighting factors and author distribution were performed. Mean numbers (number of authors, mean JIF) as well as cumulated factors (manuscripts, cumulated JIF, both in relation to employees) were calculated for each discipline and university.

Statistical Analysis

Calculation of median as well as percentiles for graphical use and graphs was created using Sigmaplot 10.0. Statistical analysis was performed with Sigmastat 3.5 integrated into Sigmaplot using analysis of variance on ranks and Dunn's pairwise comparison. Differences were considered significant with p > 0.05.

Results

Our investigation identified 931 academic persons working in cardiac surgery, 1,486 in general surgery, and 1,814 in cardiology at the 34 German universities. We counted a total of 12,096 manuscripts published in 2011 to 2013.

[Fig. 1A] indicates a higher number of all publications in cardiology compared with general and cardiac surgery. Analyzing the types of manuscripts revealed a higher number of original manuscripts and reviews in general surgery and cardiology in comparison to cardiac surgery ([Fig. 1B], [C]). The number of letters or editorials was higher in cardiology compared with both surgical disciplines ([Fig. 1D]). There were less case reports in general surgery compared with cardiology. Cardiac surgery was in between the other two disciplines ([Fig. 1E]).

Fig. 1 Total number of publications (A) in cardiology (green, ascending lines), general (orange, open bars) and cardiac surgery (blue, descending lines) departments of German universities in 2011 to and 2013. Total number (B–E) as well as relative number (G–K) of manuscripts published as original contribution (B, G), review (C, H), letter or editorial (D, I), or case report (E, K). Weighted amount of publications with factors 1 (original), 0.7 (review), 0.1 (letter or editorial), or 0.5 (case reports) (F). Data are median with 10, 25, 75, and 90 percentiles of 32 (cardiac surgery) or 34 (cardiology or general surgery) departments. ^* p < 0.05. CAR, cardiology, CS, cardiac surgery; GS, general surgery.

We further calculated the number of manuscripts in the four categories in relation to the total number. Here, no difference among the three disciplines was found for original contributions ([Fig. 1G]). In cardiac surgery, a relatively little number of reviews was published compared with cardiology ([Fig. 1H]). In cardiology, a higher fraction of letters and editorials compared with general as well as cardiac surgery was found ([Fig. 1I]). In cardiac surgery, more case reports were published compared with cardiology or general surgery ([Fig. 1K]). If the number of publications were weighted by the type of manuscript as described in Methods section, there was again a lower scientific output in cardiac surgery compared with the cardiology as well as general surgery ([Fig. 1F]).

To assess whether differences between disciplines may be due to differences in conduction of research projects, we assessed the number of manuscripts with a first author from the department. Similarly, the initiation of projects was assessed locating manuscripts with a senior author in the department. A manuscript with both the first and senior authors from a department indicated conductance of the research in the respective department. In [Fig. 2A] to [C], it becomes clearly visible that in cardiology, more manuscripts were conducted, initiated, or both indicating higher amounts of scientific research activities in cardiology.

Fig. 2 Manuscripts published with a first (A), senior (B), or both (C) authors from the department of cardiology (green, ascending lines), general (orange, open bars), and cardiac surgery (blue, descending lines) at German universities in 2011 to 2013. Mean author number on manuscripts in the three disciplines (D). Mean author number from outside department (E). Data are median with 10, 25, 75 and 90 percentiles of 32 (cardiac surgery) or 34 (cardiology or general surgery) departments. ^* p < 0.05. CAR, cardiology; CS, cardiac surgery; GS, general surgery.

In [Fig. 2D], the number of authors on the manuscripts of the three disciplines was compared. In cardiology, the number of authors was higher than in both surgical disciplines ([Fig. 2D]). Furthermore, on manuscripts from general surgery, more authors were found compared with cardiac surgery. There were less differences in author numbers with conducted, initiated, or both in the three disciplines. However, cardiology manuscripts contained more authors than cardiac surgery (for first or senior authored manuscripts) and more authors than general surgery for senior as well as first and senior authored publications. These results may indicate bigger sized research teams in cardiology.

We further assessed the number of authors from outside the department and found higher numbers of “external” authors for cardiology as well as general surgery compared with cardiac surgery. These data suggest a higher degree of collaboration and/or multicenter studies in cardiology and general surgery compared with cardiac surgery ([Fig. 2E]).

[Fig. 3] shows the correlation between the number of academic employees of cardiology, cardiac surgery, and general surgery and the number of manuscripts published in the department. Here, it can be clearly deduced that cardiac surgery departments were smaller in size than general surgery or cardiology departments. Linear regression was comparable for all three disciplines, indicating that differences in manuscript numbers depended on employee number. There was a significant correlation between employees and published manuscripts (Spearman's correlation, p = 0.000). Linear regression indicates that an additional employee led to three additional manuscripts in 2011 to 2013.

Fig. 3 Correlation of academic employee number and number of manuscripts published between 2011 and 2013 in general surgery (orange circles, short dashed orange line), cardiology (green squares, long dashed line), as well as cardiac surgery (blue triangles, dotted blue line) and regression line for all (black line).

Albeit scientifically highly debatable, the JIF is nevertheless frequently used to compare research between departments. [Fig. 4] shows analyses of JIFs for the published manuscripts. In cardiology, the mean impact factor of journals used to publish research was higher than the two surgical disciplines ([Fig. 4A]). This resulted in a higher cumulated impact factor of the journals where the manuscripts appeared in as well ([Fig. 4B]). Both parameters presented with similar differences between cardiology and surgical disciplines independent whether projects were conducted ([Supplementary Fig. 1A], [B]), initiated ([Supplementary Fig. 1C], [D]), or conducted and initiated ([Supplementary Fig. 1E], [F]) in the department. These results indicated that cardiologists tend to publish their research in more frequently cited journals.

Fig. 4 JIF of journals' manuscripts was published in median (A) as well as cumulated (B) JIF of all manuscripts. Data are median with 10, 25, 75, and 90 percentiles of 32 (cardiac surgery) or 34 (cardiology or general surgery) departments. ^* p < 0.05. CAR, cardiology (green, ascending lines); CS, cardiac surgery (blue, descending lines); GS, general surgery (orange, open bars); JIF, journal impact factor.

We next tested whether the higher employee number in cardiology may account for the higher cumulated JIF compared with cardiac and general surgeries. [Supplementary Fig. 2] shows cumulated impact factors in relation to employee number for all manuscripts ([Supplementary Fig. 2A]), manuscripts conducted in the department ([Supplementary Fig. 2B]), initiated in the department ([Supplementary Fig. 2C]), or initiated and conducted in the department ([Supplementary Fig. 2D]). Again, the results show an increased cumulated JIF in cardiology compared with both surgical disciplines, indicating that the higher cumulated JIF seems not due to higher number of employees. These analyses did not take into account differences in types of publication and author positions on manuscript. Adjustment was done for the cumulated JIF of all publications to the type of publication ([Supplementary Fig. 2E]), to author position ([Supplementary Fig. 2F]), or both ([Supplementary Fig. 2G]) in relation to employee number. Again, the results show a significantly higher JIF in cardiology compared with cardiac as well as general surgery with these adaptations.

Discussion

We demonstrate in this investigation that the number of publications in German universities appears to be a function of the number of academic personnel working in the department and not of the discipline. The lower JIFs in surgery may be due to the smaller surgical fields and/or due to substantially less high impact interdisciplinary and interinstitutional publications in surgery.

Consistent with the stereotype addressed in the introduction, cardiology published more manuscripts compared with both surgical disciplines. Unexpectedly, however, our analysis revealed a similar number of manuscripts in all three disciplines when the number of publications was related to personnel working in the departments. This comparison remained equal even when we applied a weighting system of publication types in an attempt to account for differences between, for instance, a simple case reports and an extensive original contribution. Thus, the above-mentioned stereotype does not appear to be correct. The often quoted excuse that operative obligations prevent the surgeon from publishing is also not substantiated by facts and seems to be unnecessary. In addition, in today's more and more interventional cardiology departments, this argument might as well be used by cardiologists, spending significant amounts of time in the cath laboratory or the hybrid operating room.

Despite the equality in publication per person among the disciplines, cardiology publishes in journals with higher JIF. It is a common notion that the JIF reflects scientific value and many German universities use the JIF for evaluation of academic achievement and for distribution of parts of research-directed funds (performance-related allocation).[10] Interestingly, there is a complete paucity of data supporting this perception, but there is ample evidence suggesting that the scientific value of manuscripts cannot be judged by the JIF.[11] [12] [13] For instance, the following reasons may explain the different impact factors in the three disciplines without reflecting the quality of the scientific work: The frequency of a journal published has been found a cause for differences in JIF.[14] [15] [16] Similarly, the number of references cited in a manuscript,[15] English compared with other languages,[17] and length of articles[18] are all positively correlated with the JIF.

Another main reason for differences in JIF is the size of the field (potential recipients citing a manuscript).[17] [19] The cardiologic community is larger and the interest is broader. In contrast, many questions addressed in cardiac surgery manuscripts may be too specific for a large community. As a result, the lower number of surgeons citing surgical manuscripts may result in lower JIF. These aspects may render the JIF of cardiac surgery, cardiology, and general surgery not comparable. Such differences of JIF in different disciplines have been described for biomedical fields[20] as for other disciplines. Our results may thus support the statement that the JIF seems hardly suitable for comparisons of different disciplines.[21] [22] [23] However, in our investigation, we did not assess these factors.

Some other potential causes may be additionally relevant in this context: A higher number of authors lead to a higher citation rate independent of self-citations.[16] [17] The number of authors allowed on a manuscript depends on journal policies and may be different between disciplines.[24] However, if a higher number of authors on cardiology manuscripts is generalizable, this could be one reason causing a higher JIF for cardiology journals in general.[15] [16]

Despite the many reasons against the JIF as a tool for assessing scientific value of manuscripts, we believe ourselves that it is more likely for a manuscript to receive broad attention in the community, if it is published in a high impact journal. For instance, although we hope to receive maximal attention with this analysis, the actual attention would likely be better in the New England Journal of Medicine or the European Heart Journal. Thus, the higher impact factor in cardiology is in our view an advantage for visibility in the field despite the many arguments against it.

In this investigation, we found a difference in the distribution of manuscript types between categories in cardiac surgery compared with general surgery and cardiology. We discovered a high rate of case reports in cardiac surgery and a low number of letters and editorials as well as reviews compared with cardiology. In cardiac surgery, even ground-breaking innovations have sometimes be published as case reports.[5] In the field of cardiac surgery, interesting observations in single patients seem still more frequent. Medical therapies instead seem to be analyzed in controlled trials and discussed by a wider audience and reviewed frequently. These differences thus may also reflect a cultural difference.

It is an accepted fact that the primary outcomes of randomized controlled trials (RCTs) are published in journals with high JIF. From that perspective, it is interesting to note that manuscripts from cardiology display more foreign authors. This observation suggests that a greater fraction of cardiology manuscripts may be multicenter, international trials.[16] In contrast, the number of such multicenter publications (including RCT) in both cardiac and general surgeries appears rather limited. However, as an example, the success of the CTSNet trials investigators in the United States continuously publishing primary outcomes of prospective RCTs (PRCTs) in the New England Journal of Medicine demonstrates that the differences are not necessarily discipline dependent.[25]

Limitations

This analysis is limited by the fact that it only uses average values and does not account for superb clinical performance in one department. However, the average suggests that this heterogeneity may be similar in the three disciplines. In addition, the investigation depended on correct information on the Web site of the institution. Thus, the number of academic personnel may not be accurate, potentially causing a systematic bias in this investigation. Furthermore, the assumption of initiation of an investigation/manuscript by the last author and conductance by the first author does not necessarily hold true for all manuscripts. Our results were not influenced by nonuniversity-associated authors on the manuscript, as manuscripts were included independently such authors' role or position on the manuscript.

In summary, we demonstrate in this investigation that the number of publications in German universities appears to be a function of the number of academic personnel working in the department and not of the discipline. The lower JIFs in surgery seem to be due to the smaller surgical fields and/or due to substantially less high impact interdisciplinary and interinstitutional publications in surgery.

No conflict of interest has been declared by the author(s).

• References

• 1 May RM. The scientific wealth of nations. Science 1997; 275: 793-796
• 2 King DA. The scientific impact of nations. Nature 2004; 430 (6997): 311-316
• 3 Doenst T, Schlensak C, Schibilsky D, Faerber G. Do we need basic research in cardiac surgery?. Thorac Cardiovasc Surg 2018; 66 (01) 2-6
• 4 Weber S, Haverich A. [Pioneering surgical innovations in Germany: part 2: public funding and origins of surgical innovations]. Chirurg 2016; 87 (05) 433-437
• 5 Weber S, Haverich A. [Pioneering surgical innovations in Germany: part 1: generation of medical evidence]. Chirurg 2016; 87 (05) 423-432
• 6 Bretschneider HJ, Hübner G, Knoll D, Lohr B, Nordbeck H, Spieckermann PG. Myocardial resistance and tolerance to ischemia: physiological and biochemical basis. J Cardiovasc Surg (Torino) 1975; 16 (03) 241-260
• 7 Hagl C, Schilling T, Khaladj N, Haverich A. Zwischen zwei Welten? Der Chirurg in Klinik und Forschung. Zeitschrift für Herz-, Thorax- und Gefäßchirurgie 2011; 25: 45-50
• 8 AWMF-Kommission “Bibliometrie”. Bibliometrie - eine Methode der Leistungsevaluation von Forschung; 2000
• 9 DFG Senatskommission. Empfehlungen zu einer »Leistungsorientierten Mittelvergabe« (LOM) an den Medizinischen Fakultäten. 2004
• 10 Kaltenborn KF, Kuhn K. [The journal impact factor as a parameter for the evaluation of researchers and research]. Med Klin 2003; 98: 153-169
• 11 The PLoS Medicine Editors. The impact factor game. It is time to find a better way to assess the scientific literature. PLoS 2006; 3: e291
• 12 Seglen PO. Why the impact factor of journals should not be used for evaluating research. BMJ 1997; 314 (7079): 498-502
• 13 Seglen PO. Citation rates and journal impact factors are not suitable for evaluation of research. Acta Orthop Scand 1998; 69 (03) 224-229
• 14 Tsay M-y, Chen Y-l. Journals of general & internal medicine and surgery: an analysis and comparison of citation. Scientometrics 2005; 64: 17-30
• 15 Fox CW, Paine CET, Sauterey B. Citations increase with manuscript length, author number, and references cited in ecology journals. Ecol Evol 2016; 6: 7717-7726
• 16 Glänzel W, Thijs B. Does co-authorship inflate the share of self-citations?. Scientometrics 2004; 61: 395-404
• 17 Slyder JB, Stein BR, Sams BS. , et al. Citation pattern and lifespan: a comparison of discipline, institution, and individual. Scientometrics 2011; 89: 955-966
• 18 Falagas ME, Zarkali A, Karageorgopoulos DE, Bardakas V, Mavros MN. The impact of article length on the number of future citations: a bibliometric analysis of general medicine journals. PLoS One 2013; 8 (02) e49476
• 19 Castellano C, Radicchi F. On the fairness of using relative indicators for comparing citation performance in different disciplines. Arch Immunol Ther Exp (Warsz) 2009; 57 (02) 85-90
• 20 Cleaton-Jones P, Myers G. A method for comparison of biomedical publication quality across ISI discipline categories. J Dent Educ 2002; 66 (06) 690-696
• 21 Batista PD, Campiteli MG, Kinouchi O, Martinez AS. Is it possible to compare researchers with different scientific interests?. Scientometrics 2006; 68: 179-189
• 22 Bordons M, Zulueta Ma. Comparison of research team activity in two biomedical fields. Scientometrics 1997; 40: 423-436
• 23 Vanclay JK. Impact factor: outdated artefact or stepping-stone to journal certification?. Scientometrics 2012; 92: 211-238
• 24 Dorta-González P, Dorta-González MI. Comparing journals from different fields of science and social science through a JCR subject categories normalized impact factor. Scientometrics 2013; 95: 645-672
• 25 Mehta RH, Leimberger JD, van Diepen S. , et al; LEVO-CTS Investigators. Levosimendan in patients with left ventricular dysfunction undergoing cardiac surgery. N Engl J Med 2017; 376 (21) 2032-2042