The integrated interventional radiology match: a bibliometric analysis of matched first-year residents

INTRODUCTION

The emergence of the new integrated interventional radiology (IR) residency has changed the educational landscape of how medical students enter the field of IR. For the past decade, the Society of IR (SIR) has put in tremendous efforts to collaborate with the American Board of Medical Specialties to establish IR as a distinct medical specialty.^[1] In 2017, the first large- scale American College of Graduate Medical Education (ACGME) match for the integrated-IR residency matched 119 students, in a pool of over 600 applicants, into 61 institutions across the United States.^[3-5] For the 2018 residency match, 53.8% (136 of the 253) of the US senior applicants matched into an IR/diagnostic radiology residency program.^[2] As the competitiveness of the specialty continues to increase, there has been a large push from both parties, the applicants and program directors, to determine what factors make each institution and the overall specialty competitive. Anecdotally, program directors have often stated that research is important in the IR residency application. In 2017, DePietro et al. surveyed program directors and matched Integrated IR applicants on what each party perceived to be important elements in evaluating applicants.^[5] Research experience and number of publications were ranked in the top 10 and top 15, respectively, by both groups.^[5]

Each year, the National Resident Matching Program (NRMP) releases a comprehensive report detailing the characteristics of the average US allopathic medical school and osteopathic medical school seniors who matched into their preferred specialty. Among these factors, the mean number of abstracts, presentations, and publications is also reported for the matched applicant. For the 2018 integrated-IR match, a mean of 8.4 abstracts, presentations, and publications was recorded for the matched US allopathic medical school senior applicant and a mean of 7.7 abstracts, presentations, and publications for the US osteopathic medical school senior applicant. However, these stand-alone statistics do not provide a detailed insight and analysis of the importance of research in successfully matching into an integrated-IR program.

Although preliminary data based on survey-collected information^[5] and the 2018 NRMP report are available, a comprehensive bibliometric analysis documenting the importance of research in applying to an integrated IR residency program currently does not exist. The purpose of this study is to perform a bibliometric analysis of the residents having matched into an integrated IR residency in the 2018 NRMP match. This involves analyzing the quantity and quality of publicly available indexed research published by these incoming residents. Similar bibliometric analyses exist for the neurosurgery, ophthalmology, and otolaryngology residencies.^[6-8] As the first bibliometric analysis of matched IR residents, the data obtained from this study fills a gap in the literature.

MATERIALS AND METHODS

After approval by the Institutional Review Board at the authors’ home institution, a list of all ACGME-approved integrated-IR residency programs was obtained from the SIR website. Integrated IR program directors were contacted through email asking for the name(s) of the matched 2018 resident(s) and the medical school(s) from which they matriculated. One hundred and seventeen out of the 136 residents who matched into a 2018 integrated IR residency position were identified and a cohort was selected. Authors V.C., E.W. and N.J. then retrospectively collected the following information for each matched resident using the Scopus database and author search feature: Total publications produced, number of original publications, number of case reports/series, number of review articles, number of radiology specific publications, number of first-author publications, average number of citations, average impact factor of published journals, highest impact factor among published journals, the Hirsch index (h-index), and doctoral degrees (MD, DO, PhD).

The h-index is an objective measure that represents the quality and quantity of research produced by an author. h represents the number of publications that have been cited at least h times 9. This measure has been frequently attributable to the level of research success and academic appointment.^[9] Applicants were assigned a h-index of 0 if they did not have any publications or if their publication had been present in literature for <1 year. Sixty-one residents had a h-index of 0, 20 of whom had one publication, and 41 of whom had no publications. These 20 matriculants were excluded from univariate analysis, as their true h-index cannot be calculated until 1 year after their publication.

The primary outcome determined for this study was the tier of the residency matriculation. Five tiers were established, tier one having the highest faculty average h-index, and tier five having the lowest faculty average h-index. To determine which tier each residency program belonged in, the h-index for each IR faculty member at every residency program was identified and an average institutional h-index was calculated. Each residency program was then placed into one of five tiers. Previous studies on the bibliometric analyses for the neurosurgery and ophthalmology matches were used to determine the number of tiers in our study.^[6,7]

Statistical analysis was performed using SPSS version 25 (SPSS Inc., Chicago, IL, USA, 2017). Chi-square analysis was used for all normally distributed categorical variables. Normally distributed continuous variables were compared using Student’s t-test and non-parametric data were compared using the Kruskal–Wallis test. Statistically significant for all analyses was determined to be P ≤ 0.05.

RESULTS

In total, 73 institutions participated in the ACGME integrated-IR match in 2018. Among these 73 programs, three programs only offered PGY-2 physician reserved physicians and were thus excluded from the analysis. Of the remaining 70 programs, five programs were excluded from the analysis since residents at these institutions could not be identified after reaching out to program directors and program coordinators at these institutions. An additional two programs were removed from the analysis since IR faculty could not be appropriately identified to calculate the average institutional h-index. The remaining 63 institutions and their 117 matched residents were included and reviewed on the Scopus database [Figure 1].

Close

Figure 1:

Flow diagram documenting the selection process for students included in the final cohort for analysis. ACGME: American College of Graduate Medical Education.

Export to PPT

Among the cohort for the 2018 residency cycle, 274 total publications were produced by the matched applicants, with a mean of 2.34 ± 0.41 publications per matched applicant. Of the total 274 total publications, 210 (76.6%) were original research articles, 36 (13.1%) were case reports, and 27 (9.9%) were review articles. One hundred and eighteen (43.1%) of the total publications were radiology publications and 91 (33.2%) of the total were first author publications. The average journal impact factor was 1.82 ± 0.18. The average highest journal impact factor was 2.76 ± 0.46. The average h-index for the matched applicant was 0.96 ± 0.13. These research characteristics are shown in Table 1.

All bibliometric data displayed a right-skewed distribution. This was confirmed by a one-sample Kolmogorov–Smirnov test, which indicated that all of the measured bibliometric variables do not follow a normal distribution [Figure 2]. Institutional h-indices for all 63 residency programs were calculated by taking an average of individual h-indices of every attending physician at the institution at the time of the 2018 match. The mean institutional h-index was 9.20 ± 5.71, with the distribution and tiering of these 63 programs are shown in Figure 3. A box plot representation of matriculant h-indices who matched within each tier is shown in Figure 4.

Close

Figure 2:

One-sample Komogorow–Smirnov test for normality and histograms of publication variable. (a) In this one-sample Komogorow– Smirnov test, all variable tested deviated from a normal distribution. (b) All three histograms generated for total publications, total radiology publications and h-indices of matriculants display a right-sided tail and skewness.

Export to PPT

Close

Figure 3:

Histogram of average institutional H-index of 63 institutions included in analysis. Note the tiering of this institutions.

Export to PPT

Close

Figure 4:

Box plots of publication variable for each matriculants matching into various program tiers.

Export to PPT

A univariate analysis was conducted between the tier of matriculating program and each research variable [Table 2]. The number of radiology-related publications was associated with an increased likelihood of matching into a higher tier program (P = 0.047). There was also a statistically significant relationship between higher tier institutions and highest journal impact factor (P = 0.042). The relationship between the tier of program and h-indices of matched residents was also found to be statistically significant (P = 0.037). The relationship between tier of residency institutions and other research variables such as total number of publications (P = 0.338), number of first-author publications (P = 0.083), number of original research articles (P = 0.343), number of research articles (P = 0.198), number of case reports (0.716), average journal impact factor (0.232), and years from first publication to application submission (P = 0.503) displayed no statistical significance.

DISCUSSION

The purpose of our analysis was to determine the importance of research in successfully matching into an integrated-IR program. By quantifying and qualifying publicly available indexed research published by incoming residents, this study served to be the first bibliometric analysis for the integrated IR match. In a methodology similar to Kashkoush et al.’s analysis of neurosurgery applicants for the 2016 match cycle, all programs were arranged into one of five tiers. These tiers were created based on the average h-index for all IR faculty in that institution.^[6] Kashkoush et al. found that within the neurosurgery match, there was an association between tier of program and an applicant’s h-index.^[6] Furthermore, h-index is extensively used in academic programs to assess the level of research performance.

In this study, a greater h-index was observed to have a strong correlation with matching into a higher tier residency programs (P = 0.037), suggesting the importance of the quality of research in matching into a top tier program. There was a positive trend between publishing more manuscripts and matching into tier one and two programs. Further analysis between tier of program and the number of radiology publications showed statistical significance (P = 0.047). Matched residents in tiers one, two, and three produced an average of two radiology publications, while residents in tier four and five programs had zero radiology publications. Interestingly, all four applicants with PhD degrees matched into tier one programs. In addition, statistical significance was shown between tier of residency and highest journal impact factor (P = 0.042), suggesting that applicants that published in more competitive and prestigious journals matched into higher tier programs.

Correlation between tier of the program and total number of applicant publications on univariate analysis was not statistically significant. Analysis of the relationship between tier of program and other research variables such as case reports, number of first-author publications, and average journal impact factor was also not statistically significant. These results suggest that non-radiology related research may not play as important of a role in the integrated-IR match as compared to other competitive specialties.

Similar to what was discussed in the Kashkoush et al.’s and Thangamathesvaran et al.’s analyses of the impact of research on the neurosurgery and otolaryngology matches, respectively, our study also suggests that the quality of research based on h-index, radiology-related research, and highest journal impact factors play a role in successfully matching in the integrated-IR residency program.^[6,8] However, research may not be a most important part of a candidate’s application, as other elements such as early interest in IR and involvement in the SIR may hold greater value in matching into IR. DePietro et al.’s findings showed that program directors value a litany of factors when considering an applicant, the top is a deep-seeded interest in IR.^[5] This study adds further confirmation that while research is seen favorably among program directors, it may not be a significant predictor in the IR match.

The integrated-IR match is relatively new. Before the 2018 match, only one large scale integrated IR match has existed. Due to its recent emergence, the integrated IR match has no standard guidelines or consensus on what characteristics of a medical student’s application should be prioritized. With the constant addition of integrated-IR residency programs each year, there is much variability of what factors to consider.

There were several limitations to the study. First and foremost, our analyses are based on compilations of 1-year worth of match data. This constraint provides a small sample size, giving the study less power. In addition, only h-indices were utilized to analyze the quality and quantity of research by applicants. Abstracts and poster presentations are also considered as research publications on Electronic Residency Application Service. However, data on these publications were not as publicly available as manuscripts and case reports. In addition, institutional tiers were created based on faculty h-indices. There was great variability in the number of IR faculty and work experience among the studied institutions. This may have caused institutions to be falsely represented. Furthermore, this study only included publications published before the March 2018 match cycle.

The integrated-IR match has emerged into existence for only the past 2 years, and this study serves to be the first bibliometric analysis for the integrated-IR match. In many competitive specialties such as otolaryngology and neurosurgery, an applicant’s level of research has often been positively correlated to matching into a higher tier residency program. Our findings demonstrate a similar correlation between tier of program and h-index, number of radiology publications, and highest journal impact factor. Other research elements such as case reports, and first author publications were not predictors of match success into a higher tier program. While having a deep-seeded interest in IR has been described to be the most important factor that program directors consider in candidates, strong quality research output is associated with matching into higher tier programs.

CONCLUSIONS

The integrated IR residency has only been established relatively recently as compared to other specialties. Determining the exact factors that influence the match process is important. Research serves as an important element in successfully matching into an integrated-IR residency. More importantly, h-index, number of radiology- related manuscripts, and highest journal impact factors are all positively associated with matching into a higher tier program. Specifically publishing manuscripts related to radiology was a significant determinant of matching into top tier residency programs. Characteristics of medical students who matched into top tier programs included a significant number of publications in journals with high impact factors. The quality and quantity of research have been strong determinants into matching into IR.