The use of the Hirsch index in benchmarking hepatic surgery research

8
The use of the Hirsch index in benchmarking hepatic surgery research Alessandro Cucchetti, M.D.*, Federico Mazzotti, M.D., Sara Pellegrini, M.D., Matteo Cescon, M.D., Lorenzo Maroni, M.D., Giorgio Ercolani, M.D., Antonio Daniele Pinna, M.D. Department of Medical and Surgical Sciences - DIMEC, Alma Mater Studiorum, University of Bologna, S.Orsola - Malpighi Hospital, Via Massarenti 9, Bologna 40138, Italy KEYWORDS: Hepatic surgery; Surgical therapy; H-index; Academic research; Bibliometric measures Abstract BACKGROUND: The Hirsch index (h-index) is recognized as an effective way to summarize an in- dividual’s scientific research output. However, a benchmark for evaluating surgeon scientists in the field of hepatic surgery is still not available. METHODS: A total of 3,251 authors who published between 1949 and 2011 were identified using the Scopus identification number. The h-index, the total number of cited document, the total number of citations, and the scientific age were calculated for each author using both Scopus and Google Scholar. RESULTS: The median h-index was 6 and the median scientific age, assessed with Google Scholar, was 19 years. The numbers of cited documents, numbers of citations, and h-indexes obtained from Sco- pus and Google Scholar showed good correlation with one another; however, the results from the 2 da- tabases were modified in different ways by scientific age. By plotting scientific age against h-index percentiles an h-index growth chart for both Scopus database and Google Scholar was provided. CONCLUSIONS: This analysis provides a first benchmark to assess surgeon scientists’ productivity in the field of liver surgery. Ó 2013 Elsevier Inc. All rights reserved. Rapid progress in hepatic surgery over the past decades has been accompanied by vast increases in research pub- lications. Determination of the quality and quantity of this published work is considered critical in the evaluation and comparison of scientists, both for employment purposes and for funding and resource allocation. The assessment of an individual’s research output remains problematic, and there was a lack of consensus as to how to optimally score individual research performance. 1,2 New statistics have re- cently been proposed to provide more balanced methods of quantifying publication records. These include the h-in- dex, 2,3 the g-index, 4 and the age-weighted citation rate (AWCR). 5 Among these, the citation metric that has been most actively debated, studied, and adopted is the h-index (Hirsch index). Hirsch’s h-index is defined as the number of h publications cited at least h times in the literature. 3 It can quantify and predict research output and incorporates productivity and relevance of a body of work into a single statistic. 3,6 Even if criticized, 7 the h-index is now recog- nized as a simple and effective way to summarize an indi- vidual’s scientific research output. 8 The popularity of this measure is demonstrated by the fact that Hirsch’s original article has already received over 400 citations and over The authors declare no conflict of interest. * Corresponding author. Tel.: 139-051-6363721; fax: 139-051- 304902. E-mail address: [email protected] Manuscript received October 17, 2012; revised manuscript January 11, 2013 0002-9610/$ - see front matter Ó 2013 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.amjsurg.2013.01.037 The American Journal of Surgery (2013) -, --

Transcript of The use of the Hirsch index in benchmarking hepatic surgery research

The American Journal of Surgery (2013) - -ndash-

The use of the Hirsch index in benchmarking hepaticsurgery research

Alessandro Cucchetti MD Federico Mazzotti MD Sara Pellegrini MDMatteo Cescon MD Lorenzo Maroni MD Giorgio Ercolani MDAntonio Daniele Pinna MD

Department of Medical and Surgical Sciences - DIMEC Al

ma Mater Studiorum University of Bologna SOrsola -Malpighi Hospital Via Massarenti 9 Bologna 40138 Italy

KEYWORDSHepatic surgerySurgical therapyH-indexAcademic researchBibliometric measures

The authors declare no conflict of in

Corresponding author Tel 13

304902

E-mail address aleqkoliberoit

Manuscript received October 17 201

2013

0002-9610$ - see front matter 2013

httpdxdoiorg101016jamjsurg20

AbstractBACKGROUND The Hirsch index (h-index) is recognized as an effective way to summarize an in-

dividualrsquos scientific research output However a benchmark for evaluating surgeon scientists in thefield of hepatic surgery is still not available

METHODS A total of 3251 authors who published between 1949 and 2011 were identified using theScopus identification number The h-index the total number of cited document the total number ofcitations and the scientific age were calculated for each author using both Scopus and Google Scholar

RESULTS The median h-index was 6 and the median scientific age assessed with Google Scholarwas 19 years The numbers of cited documents numbers of citations and h-indexes obtained from Sco-pus and Google Scholar showed good correlation with one another however the results from the 2 da-tabases were modified in different ways by scientific age By plotting scientific age against h-indexpercentiles an h-index growth chart for both Scopus database and Google Scholar was provided

CONCLUSIONS This analysis provides a first benchmark to assess surgeon scientistsrsquo productivityin the field of liver surgery 2013 Elsevier Inc All rights reserved

Rapid progress in hepatic surgery over the past decadeshas been accompanied by vast increases in research pub-lications Determination of the quality and quantity of thispublished work is considered critical in the evaluation andcomparison of scientists both for employment purposesand for funding and resource allocation The assessment ofan individualrsquos research output remains problematic andthere was a lack of consensus as to how to optimally score

terest

9-051-6363721 fax 139-051-

2 revised manuscript January 11

Elsevier Inc All rights reserved

1301037

individual research performance12 New statistics have re-cently been proposed to provide more balanced methods ofquantifying publication records These include the h-in-dex23 the g-index4 and the age-weighted citation rate(AWCR)5 Among these the citation metric that has beenmost actively debated studied and adopted is the h-index(Hirsch index) Hirschrsquos h-index is defined as the numberof h publications cited at least h times in the literature3

It can quantify and predict research output and incorporatesproductivity and relevance of a body of work into a singlestatistic36 Even if criticized7 the h-index is now recog-nized as a simple and effective way to summarize an indi-vidualrsquos scientific research output8 The popularity of thismeasure is demonstrated by the fact that Hirschrsquos originalarticle has already received over 400 citations and over

2 The American Journal of Surgery Vol - No - - 2013

110000 downloads9 The method is increasingly used byacademic research and federal institutions worldwide forresearch policymaking monitoring of scientific develop-ments and comparisons between institutions as well as in-dividual scientists1011

As with any bibliometric indicator however a scientistrsquosh-index must be viewed strictly in the context of thespecialty area within which he or she is working and shouldnot be extrapolated for comparison with the output of thoseoperating in other scientific areas This study aimed toestablish the spread of h-indexes for those specificallyworking in the hepatic surgery field to set standards andprovide a benchmark for evaluating individuals in this area

Methods

Author identification

The first part of the study concerned the extraction ofauthorship data from hepatic surgeryndashrelated studies in-dexed in the MEDLINE and Embase databases from 1949to 2011 inclusive In order to focus on the surgeonsinvolved rather than other physician figures the literatureresearch was limited to the 4 top citation index generalsurgery journals based on the highest impact factorsaccording to the Journal Citation Report of 201012 MED-LINE and Embase were searched using the keywordslsquolsquohepatic resectionrsquorsquo or lsquolsquohepatectomyrsquorsquo in the title or inthe abstract text The literature search was also limited tothe English language and human species resulting in a totalof 1015 unique articles checked for author selectionAuthor identification was performed up to the 14th positionin the collaboratorsrsquo list resulting in a total of 3377authors An iterative approach was adopted to identifyhomonyms and duplicate authors Each single author waschecked for Scopus identification number by comparingname middle name surname affiliation and article titleobtained with the literature research This approach leadto the exclusion of 126 duplicates and the final study sam-ple consisted of 3251 unique authors

Measure of bibliometric indexes

In the second part of the study the number of citeddocuments the total number of citations and the h-indexwere calculated for each of the selected authors using bothGoogle Scholar and Scopus The h-index is based on the setof the scientistrsquos most cited papers and the number ofcitations the scientist has received in other publicationsThe h-index is the greatest number of articles an individualis the author or coauthor of that have each been cited h ormore number of times An h-index of 10 therefore indi-cates that a scientific author has 10 publications eachwith at least 10 citations Google Scholar was searchedusing Harzingrsquos software program Publish or Perish13

The program analyzes raw citations from Google Scholar

and calculates the h-index and other statistics from thedata The AWCR index and publication age were also cal-culated using the Publish or Perish program the AWCR isthe average number of citations received by articles pub-lished by a single author weighted by the age of the arti-cles In particular it is the square root of the sum of allage-weighted citation counts over all articles that contributeto the h-index and theoretically this allows younger and yetless cited papers to contribute to the AWCR although theymay not yet contribute to the h-index5 Publication age isthe time between the first and the last publication (alsocalled the scientific age)

The surname and first initial of each author was used forthe search from both Scopus and Google Scholar Scopusresearch was limited to the Health Sciences subject area andincluded the terms lsquolsquohepatic resectionrsquorsquo OR lsquolsquohepatectomyrsquorsquoOR lsquolsquoliver transplantationrsquorsquo in the title or abstract GoogleScholar research was limited to the Medicine Pharmacol-ogy Veterinary Sciences fields and included the termslsquolsquohepatic resectionrsquorsquo OR lsquolsquohepatectomyrsquorsquo OR lsquolsquoliver trans-plantationrsquorsquo in the search field lsquolsquoAll of the wordsrsquorsquo After thequery results were reported the affiliation and each referencewere further reviewed to ensure that the results correctlyincluded publications by the intended author by noting thejournal in which it was published and if necessary linking tothe article to review it Any incorrect references wereremoved and the h-index recalculated

Statistical analysis

All data were transferred to Microsoft Excel and statis-tical analysis was performed using the Statistical Packagefor Social Sciences version 130 (SPSS Chicago IL)Continuous data were explored for normality of theirdistribution using the KolmogorovndashSmirnov test becauseno normal distribution was observed for any of the dataanalyzed all the subsequent analyses were nonparametricData were thus reported in median and ranges andrelationships between each bibliographic measure wereconsidered and outcome measures were explored with theSpearman correlation (rho) The Wilcoxon signed-rank testand the Kruskal-Wallis H test were also applied as appro-priate A P value of less than 05 was considered statisti-cally significant in all the analyses

Results

The distribution of the various bibliographic measuresconsidered among the 3251 authors identified is reportedin Table 1 Briefly the median number of cited documentsper surgeon was 13 with both Scopus database and GoogleScholar the median number of citations per surgeon was229 with the Scopus database and 313 with GoogleScholar The median h-index was 6 with both Scopusand Google Scholar and the median scientific age assessedwith Google Scholar was 19 years (range 0 to 57) Fig 1

Table 1 Distributions of the bibliographic measures

Bibliographic measureAuthors considered(N 5 3251)

Scopus Cited DocumentsMedian (range) 13 (1ndash938)25th75th95th percentile 437130

Scopus Number of CitationsMedian (range) 229 (0ndash18004)25th75th95th percentile 627422754

Scopus H-IndexMedian (range) 6 (0ndash81)25th75th95th percentile 31326

Google Scholar Cited DocumentsMedian (range) 12 (1ndash1000)25th75th95th percentile 538172

Google Scholar Number of CitationsMedian (range) 313 (0ndash26377)25th75th95th percentile 889684006

Google Scholar H-IndexMedian (range) 6 (0ndash117)25th75th95th percentile 21432

Google Scholar G-IndexMedian (range) 11 (0ndash182)25th75th95th percentile 42658

Google Scholar AWCRMedian (range) 353 (0ndash3016)25th75th95th percentile 969984151

Scientific Age (years)Median (range) 19 (0ndash57)25th75th95th percentile 1228460ndash10 years 677 (208)11ndash20 years 1137 (350)21ndash30 years 729 (224)30 years 708 (218)

AWCR5 age-weighted citation rate g-index5--- h-index5Hirsch index

Figure 1 Relationships observed between Scopus and GoogleScholar bibliometric measures numbers of cited documents (A)numbers of citations (B) and Hirsch index (h-index) (C)

A Cucchetti et al Hirsch index in hepatic surgery research 3

reports the relationships observed between numbers ofcited documents (Fig 1A) numbers of citations(Fig 1B) and h-indexes (Fig 1C) obtained from Scopusand Google Scholar All 3 bibliographic measures showedgood correlations when results from Scopus and GoogleScholar were compared in particular the number of citeddocuments had an rho of 821 (P 5 001) the number ofcitations an rho of 815 (P 5 001) and the h-index anrho of 838 (P 5 001) Table 2 reports the correlations be-tween each bibliographic measure considered and h-in-dexes calculated with both Scopus and Google ScholarOf note results from the same database demonstratedvery good correlations whereas scientific age showed aweak monotonic relationship with h-index (an rho of345 vs Scopus h-index and 500 vs Google h-index)

The role of scientific age

Results were found to be modified in different ways byscientific age as can be observed from Table 3 Regarding

Table 2 Relationships observed between each bibliographicmeasure considered and Google Scholar and Scopus Hirschindexes

Bibliographicmeasure

Scopush-index

Google Scholarh-index

Scopus total cited documents 945 827Scopus total citation 917 762Google Scholar totalcited documents

780 958

Google Scholar total citation 770 914Google Scholar ACWR 768 912Scientific age 345 500

Values represent Spearman correlation (rho)

AWCR 5 age-weighted citation rate h-index 5 Hirsch index

4 The American Journal of Surgery Vol - No - - 2013

Scopus results the median number of cited documentsranged from 5 for authors with a scientific age less than10 years to 23 for authors with a scientific age greaterthan 30 years (P 5 001) with the median number of cita-tions from 59 to 602 (P 5 001) and the median h-indexfrom 3 to 9 (P 5 001) A similar relationship was observed

Table 3 Relationships observed between each bibliographic measurscientific age

Bibliographic measure Scopus

Scientific age 10 years (n 5 677)Median cited documents (range) 5 (1ndash164)25th75th95th percentile 21238

Median number of citations 59 (0ndash2437)25th75th95th percentile 20175547

Median h-index 3 (0ndash29)25th75th95th percentile 2511

Scientific age 11ndash20 years (n 5 1137)Median cited documents (range) 12 (1ndash423)25th75th95th percentile 33093

Median number of citations 273 (0ndash8875)25th75th95th percentile 846571833

Median h-index 6 (0ndash47)25th75th95th percentile 31221

Scientific age 21ndash30 years (n 5 729)Median cited documents (range) 22 (1ndash535)25th75th95th percentile 860169

Median number of citations 437 (0ndash14039)25th75th95th percentile 14411814045

Median h-index 9 (0ndash64)25th75th95th percentile 51734

Scientific age 30 years (n 5 708)Median cited documents (range) 23 (1ndash938)25th75th95th percentile 969194

Median number of citations 602 (0ndash18004)25th75th95th percentile 10911764348

Median h-index 9 (0ndash81)25th75th95th percentile 41733

h-index 5 Hirsch index

P values are referred to the Wilcoxon signed-rank testdaggerRho is the results of Spearman correlation between Scopus and Google Sc

for Google Scholar results the median number of citeddocuments ranged from 4 to 465 (P 5 001) the mediannumber of citations from 52 to 1017 (P 5 001) and themedian h-index from 2 to 15 (P 5 001) respectivelyThese 3 bibliographic measures change according to the da-tabase used for calculation and the scientific age In partic-ular it should be noted that for authors with a scientificage less than 10 years the number of cited documents(P 5 001) number of citations (P 5 019) and h-index(P 5 001) are higher even if only slightly with Scopuswhen compared with Google Scholar results These differ-ences become less pronounced for authors with a scientificage between 10 and 20 years (P 5 049 001 and 020 re-spectively) and from the 21st year onward Google Scholargives higher values than Scopus database results (P 5001in all cases)

The Hirsch index growth curve

In Fig 2 scientific age was plotted against h-indexes andvalues interpolated to obtain an h-index growth chart forboth the Scopus database (Fig 2A) and Google Scholar(Fig 2B) These growth charts consist of a series of

e considered and Google Scholar and Scopus databases by

Google Scholar P Rhodagger

4 (1ndash90) 001 817292652 (0ndash2065) 019 810141585352 (0ndash20) 001 7851510

11 (1ndash296) 049 80552369240 (0ndash14802) 001 609127717204255 (0ndash57) 020 82921121

23 (1ndash906) 001 831957185617 (0ndash19846) 001 6412141409449310 (0ndash75) 001 83851834

465 (1ndash1000) 001 883121204041017 (0ndash26377) 001 6592902464814915 (0ndash117) 001 78662444

holar results

Figure 2 Hirsch-index (h-index) growth chart for Scopus data-base (A) and Google Scholar (B) (C) Reports the difference inthe median curve (50th percentile) between the 2 databases

A Cucchetti et al Hirsch index in hepatic surgery research 5

percentile curves that illustrate the distribution of the h-in-dex obtained with Scopus or Google Scholar in relationshipwith scientific age in order to track how the h-index canchange over time Fig 2C reports the difference in the me-dian curve (50th percentile) between the 2 databases Con-sidering an author with a scientific age of 5 years and anh-index of 5 his productivity can be placed above the 75thpercentile of the hepatic surgery scientific community withboth Scopus and Google Scholar methods meaning thatthis author had a very good scientific production Consider-ing an author with a scientific age of 10 years and an h-indexof 18 his productivity can be placed above the 95th percen-tile of the hepatic surgery community meaning that this au-thor had an excellent scientific production Of particular noteis that percentile curves were different between Scopusrsquos andGoogle Scholarrsquos h-indexes In fact as can be observed fromFig 2C the difference between the 50th percentile h-indexesof Scopus and Google Scholar was minimal within the first10 to 15 years after which Google Scholar provided higherh-indexes than the Scopus database

The age-weighted citation rate

The possibility of younger and yet less cited articlesbeing included in a different bibliographic measure wasexplored by analyzing the age-weighted citation rate Thescatter-plot of Fig 3A effectively shows that the AWCRcan handle scientific age in a single measure (rho 358)However as can be observed from Fig 3BC the AWCRwas found to be strictly related to both the Google Scholarnumber of citations (rho 960) and the h-index (rho 912)

Comments

The assessment of academic research represents acritical issue of investigation because the determinationof the quality and quantity of published work is ofparamount importance both for employment purposes andfor funding allocation1ndash3 In the field of hepatic surgery ametric of comparison is still not available nonethelesssuch a benchmark seems to be important in the presenceof the remarkable development of liver surgery observedin recent decades14 Results from this study could suggeststandards for evaluating individuals in this specific surgicalarea The literature research and the author selection for theh-index calculation of the present study identified surgeonscientists on the basis of their h-index and their scientificage in a growth chart useful for the assessment of scientificresearch in the field of hepatic surgery worldwide

There are some significant results from this study thatdeserve particular attention Attention must be first focusedon the difference observed in h-indexes obtained fromScopus and Google Scholar databases in relationship withscientific age A weak monotonic relationship was foundbetween scientific age and h-index in fact after a scientificage of about 15 to 20 years Google Scholar gives higher

h-indexes than the Scopus database This is a result of theliterature availability being from only 1996 for the h-indexcalculation of Scopus This means that comparisons can bereliably performed between the 2 databases only for younger

Figure 3 Relationship observed between the age-weighted ci-tation rate (AWCR) and scientific age (A rho 358) numberof citations (B rho 960) and the Hirsch index (h-index)(C rho 912)

6 The American Journal of Surgery Vol - No - - 2013

surgeon scientists on the contrary this approach will give anincorrect comparison in older authors These observationssupport the suggestion of using the same database foracademic research comparison especially for older authorsLooking at the growth h-index curves provided the findingthat there are some younger authors (around the 95thpercentile) who have an h-index that grossly correspondsto the 25th percentile of older ones is of interest This findingindicates that there are relatively young surgeons who havealready produced a relevant amount of scientific workscomparable or even higher to that of older surgeons Thusthe chronological age of a scientist should not represent anobstacle for employment in the academic field fundingallocation or career progression This observation supportsthe need to assess academic activity on the number of high-quality research reports rather than on academic ageApplication of a global surgical benchmark would allowthe constructive appraisal of academic surgery to achieve itsfuture goals especially among young scientists14

The second result that should attract attention is therelationship observed between the various bibliometricmeasures considered and the databases used In fact goodcorrelations were found between the number of citeddocuments the number of citations and the h-index inboth Scopus and Google Scholar databases It should alsobe noted that the age-weighted citation was found to bestrictly related to both h-index and the number of citationsThese findings mean that all bibliometric measures that arecommonly used to assess relevance of a body of work havesimilar accuracy however in a short period of time the h-index has become a widely recognized measure of quan-tifying an individualrsquos research output and to compareacademic and public institutions with one another4ndash11

Given its widespread use the proposed h-index benchmarkprovides a useful metric in the comparison of research inthe present surgical field with important practical aspectsin the evaluation of the academic career of surgeons Incomparing the h-index of candidates by means of the pre-sent growth curves institutions would be able to identifythe candidate who is best qualified for career advancementfor example in Italy the National Agency for the Evalua-tion of Universities and Research Institutes established in2012 a national scientific evaluation for career advance-ment toward associate professor or full professor basedon the h-index15 Similarly centers of academic surgicalexcellence can be recognized and targeted as sites of signif-icant academic recruitment and research output14

Some comments should be reserved for the limits of theh-index16 First this metric may not fully capture the qual-ity of a scientific research a published article could be citednot because of its quality but rather because of its poorvalue that raised harsh criticism Unfortunately this repre-sents an unavoidable limit of this citation metric Secondthe typical interfield differences in the h-index valuescaused by differences among fields in productivity and ci-tation practices can also be found when the field was

A Cucchetti et al Hirsch index in hepatic surgery research 7

selected as was found in this analysis In fact in the area ofhepatic surgery intrafield differences could probably be ex-pected among surgeons who perform only general hepaticsurgery and surgeons involved in liver transplantationand this is an aspect that probably requires further dedi-cated investigations Third the h-index can be affected byjournal impact factor or citation index In fact it can be ex-pected that the higher the rank of a scientific journal thehigher the probability would be of a specific article pub-lished in such a journal being cited by other authors be-cause such a journal would be considered moreauthoritative in the field An indirect measure of such a re-lationship can probably be obtained from calculating the g-index which aims to improve the h-index by giving moreweight to highly cited articles which articles published inmore authoritative scientific journals can be This repre-sents another aspect that deserves further analysis4 Allthese observations highlight the need for caution in theuse of the h-index avoiding indiscriminate use because re-search performance has to be considered a complex multi-faceted endeavor that should not be underestimated17

The literature research and the author selection used inthe present study can have potential shortcomings It ispossible that some authors who have significantly contrib-uted to the growth of liver surgery were left out from thesearch criteria It is possible in fact that some surgeonswho have contributed to liver transplantation developmentmay not have been included because they published injournals other than those included for example transplan-tation targeted journals The exclusion of transplantationjournals in the inclusion criteria was determined a priori toavoid the inclusion of other physician figures such asanesthesiologists or hepatologists11 It should be notedhowever that is fairly unlikely that a surgeon who has re-ally contributed to the development of liver surgery hasnever published a scientific article in 1 of the 4 most impor-tant surgical journals This does not mean that these figureshave not played a decisive role in the growth of liver sur-gery but it contrasts with the aim of this study whichwas to identify surgeons who have shaped liver surgeryas we now know it Attention should also be given to theother databases for h-index calculation such as ISIrsquos Webof Knowledge database12 or Firefoxrsquos add-in for GoogleScholar18 Although it was not the aim of this study to com-pare h-index calculation to other databases this featureprobably represents an issue that deserves further investiga-tion Another item to note would be the potential bias rep-resented by homonyms which in this study were kept to aminimum because the Scopus identification number wasapplied for author identification and all references wereaccurately checked and any incorrect references wereremoved

Conclusion

This study provides an original first benchmark useful toassess surgeon scientistsrsquo productivity in the field of liversurgery The h-index can be considered an accurate tool inthe determination of the quality and quantity of publishedwork Because the h-index calculation is influenced by thedatabase used the report of a scientistrsquos h-index mustinclude the database used for computation

Acknowledgment

The authors thank Susan West for her writing assistanceThe h-index of the corresponding Author calculated withGoogle Scholar can be found here httpscholargoogleitcitationsuser=6V_7iUEAAAAJamphl=it

References

1 Ball P Index aims for fair ranking of scientists Nature 2005436900

2 Hirsch JE Does the H index have predictive power Proc Natl Acad

Sci USA 200710419193ndash8

3 Hirsch JE An index to quantify an individualrsquos scientific research out-

put Proc Natl Acad Sci USA 200510216569ndash72

4 Egghe L Theory and practice of the g-index Scientometrics 200669

131ndash52

5 Jin B The AR-index complementing the h-index ISSI Newsletter

199736

6 Lee J Kraus KL Couldwell WT Use of the h index in neurosurgery

J Neurosurg 2009111387ndash92

7 Horne R Petrie KH Wessely S H-index pathology implications for

medical researchers and practitioners BMJ 2009339b5356

8 Ball P Achievement index climbs the ranks Nature 2007448737

9 Fersht A The most influential journals impact factor and Eigenfactor

Proc Natl Acad Sci USA 20091066883ndash4

10 Podlubny I Kassayova K Law of the constant ratio Towards a better

list of citation superstars compiling a multidisciplinary list of highly

cited researchers Res Eval 200615154ndash62

11 Poynard T Thabut D Jabre P et al Ranking hepatologists which

Hirschrsquos h-index to prevent the lsquolsquoe-crise de foi-ersquorsquo Clin Res Hepatol

Gastroenterol 201135375ndash86

12 Thomson Reuters Community Forum Available at httpcommunity

thomsonreuterscom Accessed January 2 2012

13 Harzing AW Publish or Perish 2007 Available at httpwww

harzingcompophtm Accessed June 5 2013

14 Ashrafian H Rao C Darzi A Benchmarking in surgical research Lan-

cet 20093741045ndash7

15 Agenzia Nazionale di Valutazione del Sistema Universitario e della Ri-

cerca Available at httpwwwanvurorgq5en Accessed June 5 2013

16 Costas R Bordons M The h-index advantages limitations and its re-

lation with other bibliometric indicators at the micro level J Informetr

20071193ndash203

17 Martin BR The use of multiple indicators in the assessment of basic

research Scientometrics 199636343ndash62

18 Mozilla Available at httpswwwmatunicalitianniwiki

ScholarHIndexCalculator Accessed January 10 2011

000

The Use of the Hirsch Index in BenchmarkingHepatic Surgery Research

Alessandro Cucchetti Federico Mazzotti Sara PellegriniMatteo Cescon Lorenzo Maroni Giorgio Ercolani andAntonio Daniele Pinna

Rapid progress in hepatic surgery over the pastdecades has been accompanied by vast increases inresearch publications The citation metric that hasbeen most actively adopted is the h-index (Hirschindex) defined as the number of h publications citedat least h-times in the literature The scientific ageof surgeon scientists working in the field of hepaticsurgery was plotted against h-index percentiles inan h-index growth chart for both the Scopus data-base and Google Scholar providing a first bench-mark to assess surgeon scientistsrsquo productivity inthe field of liver surgery

2 The American Journal of Surgery Vol - No - - 2013

110000 downloads9 The method is increasingly used byacademic research and federal institutions worldwide forresearch policymaking monitoring of scientific develop-ments and comparisons between institutions as well as in-dividual scientists1011

As with any bibliometric indicator however a scientistrsquosh-index must be viewed strictly in the context of thespecialty area within which he or she is working and shouldnot be extrapolated for comparison with the output of thoseoperating in other scientific areas This study aimed toestablish the spread of h-indexes for those specificallyworking in the hepatic surgery field to set standards andprovide a benchmark for evaluating individuals in this area

Methods

Author identification

The first part of the study concerned the extraction ofauthorship data from hepatic surgeryndashrelated studies in-dexed in the MEDLINE and Embase databases from 1949to 2011 inclusive In order to focus on the surgeonsinvolved rather than other physician figures the literatureresearch was limited to the 4 top citation index generalsurgery journals based on the highest impact factorsaccording to the Journal Citation Report of 201012 MED-LINE and Embase were searched using the keywordslsquolsquohepatic resectionrsquorsquo or lsquolsquohepatectomyrsquorsquo in the title or inthe abstract text The literature search was also limited tothe English language and human species resulting in a totalof 1015 unique articles checked for author selectionAuthor identification was performed up to the 14th positionin the collaboratorsrsquo list resulting in a total of 3377authors An iterative approach was adopted to identifyhomonyms and duplicate authors Each single author waschecked for Scopus identification number by comparingname middle name surname affiliation and article titleobtained with the literature research This approach leadto the exclusion of 126 duplicates and the final study sam-ple consisted of 3251 unique authors

Measure of bibliometric indexes

In the second part of the study the number of citeddocuments the total number of citations and the h-indexwere calculated for each of the selected authors using bothGoogle Scholar and Scopus The h-index is based on the setof the scientistrsquos most cited papers and the number ofcitations the scientist has received in other publicationsThe h-index is the greatest number of articles an individualis the author or coauthor of that have each been cited h ormore number of times An h-index of 10 therefore indi-cates that a scientific author has 10 publications eachwith at least 10 citations Google Scholar was searchedusing Harzingrsquos software program Publish or Perish13

The program analyzes raw citations from Google Scholar

and calculates the h-index and other statistics from thedata The AWCR index and publication age were also cal-culated using the Publish or Perish program the AWCR isthe average number of citations received by articles pub-lished by a single author weighted by the age of the arti-cles In particular it is the square root of the sum of allage-weighted citation counts over all articles that contributeto the h-index and theoretically this allows younger and yetless cited papers to contribute to the AWCR although theymay not yet contribute to the h-index5 Publication age isthe time between the first and the last publication (alsocalled the scientific age)

The surname and first initial of each author was used forthe search from both Scopus and Google Scholar Scopusresearch was limited to the Health Sciences subject area andincluded the terms lsquolsquohepatic resectionrsquorsquo OR lsquolsquohepatectomyrsquorsquoOR lsquolsquoliver transplantationrsquorsquo in the title or abstract GoogleScholar research was limited to the Medicine Pharmacol-ogy Veterinary Sciences fields and included the termslsquolsquohepatic resectionrsquorsquo OR lsquolsquohepatectomyrsquorsquo OR lsquolsquoliver trans-plantationrsquorsquo in the search field lsquolsquoAll of the wordsrsquorsquo After thequery results were reported the affiliation and each referencewere further reviewed to ensure that the results correctlyincluded publications by the intended author by noting thejournal in which it was published and if necessary linking tothe article to review it Any incorrect references wereremoved and the h-index recalculated

Statistical analysis

All data were transferred to Microsoft Excel and statis-tical analysis was performed using the Statistical Packagefor Social Sciences version 130 (SPSS Chicago IL)Continuous data were explored for normality of theirdistribution using the KolmogorovndashSmirnov test becauseno normal distribution was observed for any of the dataanalyzed all the subsequent analyses were nonparametricData were thus reported in median and ranges andrelationships between each bibliographic measure wereconsidered and outcome measures were explored with theSpearman correlation (rho) The Wilcoxon signed-rank testand the Kruskal-Wallis H test were also applied as appro-priate A P value of less than 05 was considered statisti-cally significant in all the analyses

Results

The distribution of the various bibliographic measuresconsidered among the 3251 authors identified is reportedin Table 1 Briefly the median number of cited documentsper surgeon was 13 with both Scopus database and GoogleScholar the median number of citations per surgeon was229 with the Scopus database and 313 with GoogleScholar The median h-index was 6 with both Scopusand Google Scholar and the median scientific age assessedwith Google Scholar was 19 years (range 0 to 57) Fig 1

Table 1 Distributions of the bibliographic measures

Bibliographic measureAuthors considered(N 5 3251)

Scopus Cited DocumentsMedian (range) 13 (1ndash938)25th75th95th percentile 437130

Scopus Number of CitationsMedian (range) 229 (0ndash18004)25th75th95th percentile 627422754

Scopus H-IndexMedian (range) 6 (0ndash81)25th75th95th percentile 31326

Google Scholar Cited DocumentsMedian (range) 12 (1ndash1000)25th75th95th percentile 538172

Google Scholar Number of CitationsMedian (range) 313 (0ndash26377)25th75th95th percentile 889684006

Google Scholar H-IndexMedian (range) 6 (0ndash117)25th75th95th percentile 21432

Google Scholar G-IndexMedian (range) 11 (0ndash182)25th75th95th percentile 42658

Google Scholar AWCRMedian (range) 353 (0ndash3016)25th75th95th percentile 969984151

Scientific Age (years)Median (range) 19 (0ndash57)25th75th95th percentile 1228460ndash10 years 677 (208)11ndash20 years 1137 (350)21ndash30 years 729 (224)30 years 708 (218)

AWCR5 age-weighted citation rate g-index5--- h-index5Hirsch index

Figure 1 Relationships observed between Scopus and GoogleScholar bibliometric measures numbers of cited documents (A)numbers of citations (B) and Hirsch index (h-index) (C)

A Cucchetti et al Hirsch index in hepatic surgery research 3

reports the relationships observed between numbers ofcited documents (Fig 1A) numbers of citations(Fig 1B) and h-indexes (Fig 1C) obtained from Scopusand Google Scholar All 3 bibliographic measures showedgood correlations when results from Scopus and GoogleScholar were compared in particular the number of citeddocuments had an rho of 821 (P 5 001) the number ofcitations an rho of 815 (P 5 001) and the h-index anrho of 838 (P 5 001) Table 2 reports the correlations be-tween each bibliographic measure considered and h-in-dexes calculated with both Scopus and Google ScholarOf note results from the same database demonstratedvery good correlations whereas scientific age showed aweak monotonic relationship with h-index (an rho of345 vs Scopus h-index and 500 vs Google h-index)

The role of scientific age

Results were found to be modified in different ways byscientific age as can be observed from Table 3 Regarding

Table 2 Relationships observed between each bibliographicmeasure considered and Google Scholar and Scopus Hirschindexes

Bibliographicmeasure

Scopush-index

Google Scholarh-index

Scopus total cited documents 945 827Scopus total citation 917 762Google Scholar totalcited documents

780 958

Google Scholar total citation 770 914Google Scholar ACWR 768 912Scientific age 345 500

Values represent Spearman correlation (rho)

AWCR 5 age-weighted citation rate h-index 5 Hirsch index

4 The American Journal of Surgery Vol - No - - 2013

Scopus results the median number of cited documentsranged from 5 for authors with a scientific age less than10 years to 23 for authors with a scientific age greaterthan 30 years (P 5 001) with the median number of cita-tions from 59 to 602 (P 5 001) and the median h-indexfrom 3 to 9 (P 5 001) A similar relationship was observed

Table 3 Relationships observed between each bibliographic measurscientific age

Bibliographic measure Scopus

Scientific age 10 years (n 5 677)Median cited documents (range) 5 (1ndash164)25th75th95th percentile 21238

Median number of citations 59 (0ndash2437)25th75th95th percentile 20175547

Median h-index 3 (0ndash29)25th75th95th percentile 2511

Scientific age 11ndash20 years (n 5 1137)Median cited documents (range) 12 (1ndash423)25th75th95th percentile 33093

Median number of citations 273 (0ndash8875)25th75th95th percentile 846571833

Median h-index 6 (0ndash47)25th75th95th percentile 31221

Scientific age 21ndash30 years (n 5 729)Median cited documents (range) 22 (1ndash535)25th75th95th percentile 860169

Median number of citations 437 (0ndash14039)25th75th95th percentile 14411814045

Median h-index 9 (0ndash64)25th75th95th percentile 51734

Scientific age 30 years (n 5 708)Median cited documents (range) 23 (1ndash938)25th75th95th percentile 969194

Median number of citations 602 (0ndash18004)25th75th95th percentile 10911764348

Median h-index 9 (0ndash81)25th75th95th percentile 41733

h-index 5 Hirsch index

P values are referred to the Wilcoxon signed-rank testdaggerRho is the results of Spearman correlation between Scopus and Google Sc

for Google Scholar results the median number of citeddocuments ranged from 4 to 465 (P 5 001) the mediannumber of citations from 52 to 1017 (P 5 001) and themedian h-index from 2 to 15 (P 5 001) respectivelyThese 3 bibliographic measures change according to the da-tabase used for calculation and the scientific age In partic-ular it should be noted that for authors with a scientificage less than 10 years the number of cited documents(P 5 001) number of citations (P 5 019) and h-index(P 5 001) are higher even if only slightly with Scopuswhen compared with Google Scholar results These differ-ences become less pronounced for authors with a scientificage between 10 and 20 years (P 5 049 001 and 020 re-spectively) and from the 21st year onward Google Scholargives higher values than Scopus database results (P 5001in all cases)

The Hirsch index growth curve

In Fig 2 scientific age was plotted against h-indexes andvalues interpolated to obtain an h-index growth chart forboth the Scopus database (Fig 2A) and Google Scholar(Fig 2B) These growth charts consist of a series of

e considered and Google Scholar and Scopus databases by

Google Scholar P Rhodagger

4 (1ndash90) 001 817292652 (0ndash2065) 019 810141585352 (0ndash20) 001 7851510

11 (1ndash296) 049 80552369240 (0ndash14802) 001 609127717204255 (0ndash57) 020 82921121

23 (1ndash906) 001 831957185617 (0ndash19846) 001 6412141409449310 (0ndash75) 001 83851834

465 (1ndash1000) 001 883121204041017 (0ndash26377) 001 6592902464814915 (0ndash117) 001 78662444

holar results

Figure 2 Hirsch-index (h-index) growth chart for Scopus data-base (A) and Google Scholar (B) (C) Reports the difference inthe median curve (50th percentile) between the 2 databases

A Cucchetti et al Hirsch index in hepatic surgery research 5

percentile curves that illustrate the distribution of the h-in-dex obtained with Scopus or Google Scholar in relationshipwith scientific age in order to track how the h-index canchange over time Fig 2C reports the difference in the me-dian curve (50th percentile) between the 2 databases Con-sidering an author with a scientific age of 5 years and anh-index of 5 his productivity can be placed above the 75thpercentile of the hepatic surgery scientific community withboth Scopus and Google Scholar methods meaning thatthis author had a very good scientific production Consider-ing an author with a scientific age of 10 years and an h-indexof 18 his productivity can be placed above the 95th percen-tile of the hepatic surgery community meaning that this au-thor had an excellent scientific production Of particular noteis that percentile curves were different between Scopusrsquos andGoogle Scholarrsquos h-indexes In fact as can be observed fromFig 2C the difference between the 50th percentile h-indexesof Scopus and Google Scholar was minimal within the first10 to 15 years after which Google Scholar provided higherh-indexes than the Scopus database

The age-weighted citation rate

The possibility of younger and yet less cited articlesbeing included in a different bibliographic measure wasexplored by analyzing the age-weighted citation rate Thescatter-plot of Fig 3A effectively shows that the AWCRcan handle scientific age in a single measure (rho 358)However as can be observed from Fig 3BC the AWCRwas found to be strictly related to both the Google Scholarnumber of citations (rho 960) and the h-index (rho 912)

Comments

The assessment of academic research represents acritical issue of investigation because the determinationof the quality and quantity of published work is ofparamount importance both for employment purposes andfor funding allocation1ndash3 In the field of hepatic surgery ametric of comparison is still not available nonethelesssuch a benchmark seems to be important in the presenceof the remarkable development of liver surgery observedin recent decades14 Results from this study could suggeststandards for evaluating individuals in this specific surgicalarea The literature research and the author selection for theh-index calculation of the present study identified surgeonscientists on the basis of their h-index and their scientificage in a growth chart useful for the assessment of scientificresearch in the field of hepatic surgery worldwide

There are some significant results from this study thatdeserve particular attention Attention must be first focusedon the difference observed in h-indexes obtained fromScopus and Google Scholar databases in relationship withscientific age A weak monotonic relationship was foundbetween scientific age and h-index in fact after a scientificage of about 15 to 20 years Google Scholar gives higher

h-indexes than the Scopus database This is a result of theliterature availability being from only 1996 for the h-indexcalculation of Scopus This means that comparisons can bereliably performed between the 2 databases only for younger

Figure 3 Relationship observed between the age-weighted ci-tation rate (AWCR) and scientific age (A rho 358) numberof citations (B rho 960) and the Hirsch index (h-index)(C rho 912)

6 The American Journal of Surgery Vol - No - - 2013

surgeon scientists on the contrary this approach will give anincorrect comparison in older authors These observationssupport the suggestion of using the same database foracademic research comparison especially for older authorsLooking at the growth h-index curves provided the findingthat there are some younger authors (around the 95thpercentile) who have an h-index that grossly correspondsto the 25th percentile of older ones is of interest This findingindicates that there are relatively young surgeons who havealready produced a relevant amount of scientific workscomparable or even higher to that of older surgeons Thusthe chronological age of a scientist should not represent anobstacle for employment in the academic field fundingallocation or career progression This observation supportsthe need to assess academic activity on the number of high-quality research reports rather than on academic ageApplication of a global surgical benchmark would allowthe constructive appraisal of academic surgery to achieve itsfuture goals especially among young scientists14

The second result that should attract attention is therelationship observed between the various bibliometricmeasures considered and the databases used In fact goodcorrelations were found between the number of citeddocuments the number of citations and the h-index inboth Scopus and Google Scholar databases It should alsobe noted that the age-weighted citation was found to bestrictly related to both h-index and the number of citationsThese findings mean that all bibliometric measures that arecommonly used to assess relevance of a body of work havesimilar accuracy however in a short period of time the h-index has become a widely recognized measure of quan-tifying an individualrsquos research output and to compareacademic and public institutions with one another4ndash11

Given its widespread use the proposed h-index benchmarkprovides a useful metric in the comparison of research inthe present surgical field with important practical aspectsin the evaluation of the academic career of surgeons Incomparing the h-index of candidates by means of the pre-sent growth curves institutions would be able to identifythe candidate who is best qualified for career advancementfor example in Italy the National Agency for the Evalua-tion of Universities and Research Institutes established in2012 a national scientific evaluation for career advance-ment toward associate professor or full professor basedon the h-index15 Similarly centers of academic surgicalexcellence can be recognized and targeted as sites of signif-icant academic recruitment and research output14

Some comments should be reserved for the limits of theh-index16 First this metric may not fully capture the qual-ity of a scientific research a published article could be citednot because of its quality but rather because of its poorvalue that raised harsh criticism Unfortunately this repre-sents an unavoidable limit of this citation metric Secondthe typical interfield differences in the h-index valuescaused by differences among fields in productivity and ci-tation practices can also be found when the field was

A Cucchetti et al Hirsch index in hepatic surgery research 7

selected as was found in this analysis In fact in the area ofhepatic surgery intrafield differences could probably be ex-pected among surgeons who perform only general hepaticsurgery and surgeons involved in liver transplantationand this is an aspect that probably requires further dedi-cated investigations Third the h-index can be affected byjournal impact factor or citation index In fact it can be ex-pected that the higher the rank of a scientific journal thehigher the probability would be of a specific article pub-lished in such a journal being cited by other authors be-cause such a journal would be considered moreauthoritative in the field An indirect measure of such a re-lationship can probably be obtained from calculating the g-index which aims to improve the h-index by giving moreweight to highly cited articles which articles published inmore authoritative scientific journals can be This repre-sents another aspect that deserves further analysis4 Allthese observations highlight the need for caution in theuse of the h-index avoiding indiscriminate use because re-search performance has to be considered a complex multi-faceted endeavor that should not be underestimated17

The literature research and the author selection used inthe present study can have potential shortcomings It ispossible that some authors who have significantly contrib-uted to the growth of liver surgery were left out from thesearch criteria It is possible in fact that some surgeonswho have contributed to liver transplantation developmentmay not have been included because they published injournals other than those included for example transplan-tation targeted journals The exclusion of transplantationjournals in the inclusion criteria was determined a priori toavoid the inclusion of other physician figures such asanesthesiologists or hepatologists11 It should be notedhowever that is fairly unlikely that a surgeon who has re-ally contributed to the development of liver surgery hasnever published a scientific article in 1 of the 4 most impor-tant surgical journals This does not mean that these figureshave not played a decisive role in the growth of liver sur-gery but it contrasts with the aim of this study whichwas to identify surgeons who have shaped liver surgeryas we now know it Attention should also be given to theother databases for h-index calculation such as ISIrsquos Webof Knowledge database12 or Firefoxrsquos add-in for GoogleScholar18 Although it was not the aim of this study to com-pare h-index calculation to other databases this featureprobably represents an issue that deserves further investiga-tion Another item to note would be the potential bias rep-resented by homonyms which in this study were kept to aminimum because the Scopus identification number wasapplied for author identification and all references wereaccurately checked and any incorrect references wereremoved

Conclusion

This study provides an original first benchmark useful toassess surgeon scientistsrsquo productivity in the field of liversurgery The h-index can be considered an accurate tool inthe determination of the quality and quantity of publishedwork Because the h-index calculation is influenced by thedatabase used the report of a scientistrsquos h-index mustinclude the database used for computation

Acknowledgment

The authors thank Susan West for her writing assistanceThe h-index of the corresponding Author calculated withGoogle Scholar can be found here httpscholargoogleitcitationsuser=6V_7iUEAAAAJamphl=it

References

1 Ball P Index aims for fair ranking of scientists Nature 2005436900

2 Hirsch JE Does the H index have predictive power Proc Natl Acad

Sci USA 200710419193ndash8

3 Hirsch JE An index to quantify an individualrsquos scientific research out-

put Proc Natl Acad Sci USA 200510216569ndash72

4 Egghe L Theory and practice of the g-index Scientometrics 200669

131ndash52

5 Jin B The AR-index complementing the h-index ISSI Newsletter

199736

6 Lee J Kraus KL Couldwell WT Use of the h index in neurosurgery

J Neurosurg 2009111387ndash92

7 Horne R Petrie KH Wessely S H-index pathology implications for

medical researchers and practitioners BMJ 2009339b5356

8 Ball P Achievement index climbs the ranks Nature 2007448737

9 Fersht A The most influential journals impact factor and Eigenfactor

Proc Natl Acad Sci USA 20091066883ndash4

10 Podlubny I Kassayova K Law of the constant ratio Towards a better

list of citation superstars compiling a multidisciplinary list of highly

cited researchers Res Eval 200615154ndash62

11 Poynard T Thabut D Jabre P et al Ranking hepatologists which

Hirschrsquos h-index to prevent the lsquolsquoe-crise de foi-ersquorsquo Clin Res Hepatol

Gastroenterol 201135375ndash86

12 Thomson Reuters Community Forum Available at httpcommunity

thomsonreuterscom Accessed January 2 2012

13 Harzing AW Publish or Perish 2007 Available at httpwww

harzingcompophtm Accessed June 5 2013

14 Ashrafian H Rao C Darzi A Benchmarking in surgical research Lan-

cet 20093741045ndash7

15 Agenzia Nazionale di Valutazione del Sistema Universitario e della Ri-

cerca Available at httpwwwanvurorgq5en Accessed June 5 2013

16 Costas R Bordons M The h-index advantages limitations and its re-

lation with other bibliometric indicators at the micro level J Informetr

20071193ndash203

17 Martin BR The use of multiple indicators in the assessment of basic

research Scientometrics 199636343ndash62

18 Mozilla Available at httpswwwmatunicalitianniwiki

ScholarHIndexCalculator Accessed January 10 2011

000

The Use of the Hirsch Index in BenchmarkingHepatic Surgery Research

Alessandro Cucchetti Federico Mazzotti Sara PellegriniMatteo Cescon Lorenzo Maroni Giorgio Ercolani andAntonio Daniele Pinna

Rapid progress in hepatic surgery over the pastdecades has been accompanied by vast increases inresearch publications The citation metric that hasbeen most actively adopted is the h-index (Hirschindex) defined as the number of h publications citedat least h-times in the literature The scientific ageof surgeon scientists working in the field of hepaticsurgery was plotted against h-index percentiles inan h-index growth chart for both the Scopus data-base and Google Scholar providing a first bench-mark to assess surgeon scientistsrsquo productivity inthe field of liver surgery

Table 1 Distributions of the bibliographic measures

Bibliographic measureAuthors considered(N 5 3251)

Scopus Cited DocumentsMedian (range) 13 (1ndash938)25th75th95th percentile 437130

Scopus Number of CitationsMedian (range) 229 (0ndash18004)25th75th95th percentile 627422754

Scopus H-IndexMedian (range) 6 (0ndash81)25th75th95th percentile 31326

Google Scholar Cited DocumentsMedian (range) 12 (1ndash1000)25th75th95th percentile 538172

Google Scholar Number of CitationsMedian (range) 313 (0ndash26377)25th75th95th percentile 889684006

Google Scholar H-IndexMedian (range) 6 (0ndash117)25th75th95th percentile 21432

Google Scholar G-IndexMedian (range) 11 (0ndash182)25th75th95th percentile 42658

Google Scholar AWCRMedian (range) 353 (0ndash3016)25th75th95th percentile 969984151

Scientific Age (years)Median (range) 19 (0ndash57)25th75th95th percentile 1228460ndash10 years 677 (208)11ndash20 years 1137 (350)21ndash30 years 729 (224)30 years 708 (218)

AWCR5 age-weighted citation rate g-index5--- h-index5Hirsch index

Figure 1 Relationships observed between Scopus and GoogleScholar bibliometric measures numbers of cited documents (A)numbers of citations (B) and Hirsch index (h-index) (C)

A Cucchetti et al Hirsch index in hepatic surgery research 3

reports the relationships observed between numbers ofcited documents (Fig 1A) numbers of citations(Fig 1B) and h-indexes (Fig 1C) obtained from Scopusand Google Scholar All 3 bibliographic measures showedgood correlations when results from Scopus and GoogleScholar were compared in particular the number of citeddocuments had an rho of 821 (P 5 001) the number ofcitations an rho of 815 (P 5 001) and the h-index anrho of 838 (P 5 001) Table 2 reports the correlations be-tween each bibliographic measure considered and h-in-dexes calculated with both Scopus and Google ScholarOf note results from the same database demonstratedvery good correlations whereas scientific age showed aweak monotonic relationship with h-index (an rho of345 vs Scopus h-index and 500 vs Google h-index)

The role of scientific age

Results were found to be modified in different ways byscientific age as can be observed from Table 3 Regarding

Table 2 Relationships observed between each bibliographicmeasure considered and Google Scholar and Scopus Hirschindexes

Bibliographicmeasure

Scopush-index

Google Scholarh-index

Scopus total cited documents 945 827Scopus total citation 917 762Google Scholar totalcited documents

780 958

Google Scholar total citation 770 914Google Scholar ACWR 768 912Scientific age 345 500

Values represent Spearman correlation (rho)

AWCR 5 age-weighted citation rate h-index 5 Hirsch index

4 The American Journal of Surgery Vol - No - - 2013

Scopus results the median number of cited documentsranged from 5 for authors with a scientific age less than10 years to 23 for authors with a scientific age greaterthan 30 years (P 5 001) with the median number of cita-tions from 59 to 602 (P 5 001) and the median h-indexfrom 3 to 9 (P 5 001) A similar relationship was observed

Table 3 Relationships observed between each bibliographic measurscientific age

Bibliographic measure Scopus

Scientific age 10 years (n 5 677)Median cited documents (range) 5 (1ndash164)25th75th95th percentile 21238

Median number of citations 59 (0ndash2437)25th75th95th percentile 20175547

Median h-index 3 (0ndash29)25th75th95th percentile 2511

Scientific age 11ndash20 years (n 5 1137)Median cited documents (range) 12 (1ndash423)25th75th95th percentile 33093

Median number of citations 273 (0ndash8875)25th75th95th percentile 846571833

Median h-index 6 (0ndash47)25th75th95th percentile 31221

Scientific age 21ndash30 years (n 5 729)Median cited documents (range) 22 (1ndash535)25th75th95th percentile 860169

Median number of citations 437 (0ndash14039)25th75th95th percentile 14411814045

Median h-index 9 (0ndash64)25th75th95th percentile 51734

Scientific age 30 years (n 5 708)Median cited documents (range) 23 (1ndash938)25th75th95th percentile 969194

Median number of citations 602 (0ndash18004)25th75th95th percentile 10911764348

Median h-index 9 (0ndash81)25th75th95th percentile 41733

h-index 5 Hirsch index

P values are referred to the Wilcoxon signed-rank testdaggerRho is the results of Spearman correlation between Scopus and Google Sc

for Google Scholar results the median number of citeddocuments ranged from 4 to 465 (P 5 001) the mediannumber of citations from 52 to 1017 (P 5 001) and themedian h-index from 2 to 15 (P 5 001) respectivelyThese 3 bibliographic measures change according to the da-tabase used for calculation and the scientific age In partic-ular it should be noted that for authors with a scientificage less than 10 years the number of cited documents(P 5 001) number of citations (P 5 019) and h-index(P 5 001) are higher even if only slightly with Scopuswhen compared with Google Scholar results These differ-ences become less pronounced for authors with a scientificage between 10 and 20 years (P 5 049 001 and 020 re-spectively) and from the 21st year onward Google Scholargives higher values than Scopus database results (P 5001in all cases)

The Hirsch index growth curve

In Fig 2 scientific age was plotted against h-indexes andvalues interpolated to obtain an h-index growth chart forboth the Scopus database (Fig 2A) and Google Scholar(Fig 2B) These growth charts consist of a series of

e considered and Google Scholar and Scopus databases by

Google Scholar P Rhodagger

4 (1ndash90) 001 817292652 (0ndash2065) 019 810141585352 (0ndash20) 001 7851510

11 (1ndash296) 049 80552369240 (0ndash14802) 001 609127717204255 (0ndash57) 020 82921121

23 (1ndash906) 001 831957185617 (0ndash19846) 001 6412141409449310 (0ndash75) 001 83851834

465 (1ndash1000) 001 883121204041017 (0ndash26377) 001 6592902464814915 (0ndash117) 001 78662444

holar results

Figure 2 Hirsch-index (h-index) growth chart for Scopus data-base (A) and Google Scholar (B) (C) Reports the difference inthe median curve (50th percentile) between the 2 databases

A Cucchetti et al Hirsch index in hepatic surgery research 5

percentile curves that illustrate the distribution of the h-in-dex obtained with Scopus or Google Scholar in relationshipwith scientific age in order to track how the h-index canchange over time Fig 2C reports the difference in the me-dian curve (50th percentile) between the 2 databases Con-sidering an author with a scientific age of 5 years and anh-index of 5 his productivity can be placed above the 75thpercentile of the hepatic surgery scientific community withboth Scopus and Google Scholar methods meaning thatthis author had a very good scientific production Consider-ing an author with a scientific age of 10 years and an h-indexof 18 his productivity can be placed above the 95th percen-tile of the hepatic surgery community meaning that this au-thor had an excellent scientific production Of particular noteis that percentile curves were different between Scopusrsquos andGoogle Scholarrsquos h-indexes In fact as can be observed fromFig 2C the difference between the 50th percentile h-indexesof Scopus and Google Scholar was minimal within the first10 to 15 years after which Google Scholar provided higherh-indexes than the Scopus database

The age-weighted citation rate

The possibility of younger and yet less cited articlesbeing included in a different bibliographic measure wasexplored by analyzing the age-weighted citation rate Thescatter-plot of Fig 3A effectively shows that the AWCRcan handle scientific age in a single measure (rho 358)However as can be observed from Fig 3BC the AWCRwas found to be strictly related to both the Google Scholarnumber of citations (rho 960) and the h-index (rho 912)

Comments

The assessment of academic research represents acritical issue of investigation because the determinationof the quality and quantity of published work is ofparamount importance both for employment purposes andfor funding allocation1ndash3 In the field of hepatic surgery ametric of comparison is still not available nonethelesssuch a benchmark seems to be important in the presenceof the remarkable development of liver surgery observedin recent decades14 Results from this study could suggeststandards for evaluating individuals in this specific surgicalarea The literature research and the author selection for theh-index calculation of the present study identified surgeonscientists on the basis of their h-index and their scientificage in a growth chart useful for the assessment of scientificresearch in the field of hepatic surgery worldwide

There are some significant results from this study thatdeserve particular attention Attention must be first focusedon the difference observed in h-indexes obtained fromScopus and Google Scholar databases in relationship withscientific age A weak monotonic relationship was foundbetween scientific age and h-index in fact after a scientificage of about 15 to 20 years Google Scholar gives higher

h-indexes than the Scopus database This is a result of theliterature availability being from only 1996 for the h-indexcalculation of Scopus This means that comparisons can bereliably performed between the 2 databases only for younger

Figure 3 Relationship observed between the age-weighted ci-tation rate (AWCR) and scientific age (A rho 358) numberof citations (B rho 960) and the Hirsch index (h-index)(C rho 912)

6 The American Journal of Surgery Vol - No - - 2013

surgeon scientists on the contrary this approach will give anincorrect comparison in older authors These observationssupport the suggestion of using the same database foracademic research comparison especially for older authorsLooking at the growth h-index curves provided the findingthat there are some younger authors (around the 95thpercentile) who have an h-index that grossly correspondsto the 25th percentile of older ones is of interest This findingindicates that there are relatively young surgeons who havealready produced a relevant amount of scientific workscomparable or even higher to that of older surgeons Thusthe chronological age of a scientist should not represent anobstacle for employment in the academic field fundingallocation or career progression This observation supportsthe need to assess academic activity on the number of high-quality research reports rather than on academic ageApplication of a global surgical benchmark would allowthe constructive appraisal of academic surgery to achieve itsfuture goals especially among young scientists14

The second result that should attract attention is therelationship observed between the various bibliometricmeasures considered and the databases used In fact goodcorrelations were found between the number of citeddocuments the number of citations and the h-index inboth Scopus and Google Scholar databases It should alsobe noted that the age-weighted citation was found to bestrictly related to both h-index and the number of citationsThese findings mean that all bibliometric measures that arecommonly used to assess relevance of a body of work havesimilar accuracy however in a short period of time the h-index has become a widely recognized measure of quan-tifying an individualrsquos research output and to compareacademic and public institutions with one another4ndash11

Given its widespread use the proposed h-index benchmarkprovides a useful metric in the comparison of research inthe present surgical field with important practical aspectsin the evaluation of the academic career of surgeons Incomparing the h-index of candidates by means of the pre-sent growth curves institutions would be able to identifythe candidate who is best qualified for career advancementfor example in Italy the National Agency for the Evalua-tion of Universities and Research Institutes established in2012 a national scientific evaluation for career advance-ment toward associate professor or full professor basedon the h-index15 Similarly centers of academic surgicalexcellence can be recognized and targeted as sites of signif-icant academic recruitment and research output14

Some comments should be reserved for the limits of theh-index16 First this metric may not fully capture the qual-ity of a scientific research a published article could be citednot because of its quality but rather because of its poorvalue that raised harsh criticism Unfortunately this repre-sents an unavoidable limit of this citation metric Secondthe typical interfield differences in the h-index valuescaused by differences among fields in productivity and ci-tation practices can also be found when the field was

A Cucchetti et al Hirsch index in hepatic surgery research 7

selected as was found in this analysis In fact in the area ofhepatic surgery intrafield differences could probably be ex-pected among surgeons who perform only general hepaticsurgery and surgeons involved in liver transplantationand this is an aspect that probably requires further dedi-cated investigations Third the h-index can be affected byjournal impact factor or citation index In fact it can be ex-pected that the higher the rank of a scientific journal thehigher the probability would be of a specific article pub-lished in such a journal being cited by other authors be-cause such a journal would be considered moreauthoritative in the field An indirect measure of such a re-lationship can probably be obtained from calculating the g-index which aims to improve the h-index by giving moreweight to highly cited articles which articles published inmore authoritative scientific journals can be This repre-sents another aspect that deserves further analysis4 Allthese observations highlight the need for caution in theuse of the h-index avoiding indiscriminate use because re-search performance has to be considered a complex multi-faceted endeavor that should not be underestimated17

The literature research and the author selection used inthe present study can have potential shortcomings It ispossible that some authors who have significantly contrib-uted to the growth of liver surgery were left out from thesearch criteria It is possible in fact that some surgeonswho have contributed to liver transplantation developmentmay not have been included because they published injournals other than those included for example transplan-tation targeted journals The exclusion of transplantationjournals in the inclusion criteria was determined a priori toavoid the inclusion of other physician figures such asanesthesiologists or hepatologists11 It should be notedhowever that is fairly unlikely that a surgeon who has re-ally contributed to the development of liver surgery hasnever published a scientific article in 1 of the 4 most impor-tant surgical journals This does not mean that these figureshave not played a decisive role in the growth of liver sur-gery but it contrasts with the aim of this study whichwas to identify surgeons who have shaped liver surgeryas we now know it Attention should also be given to theother databases for h-index calculation such as ISIrsquos Webof Knowledge database12 or Firefoxrsquos add-in for GoogleScholar18 Although it was not the aim of this study to com-pare h-index calculation to other databases this featureprobably represents an issue that deserves further investiga-tion Another item to note would be the potential bias rep-resented by homonyms which in this study were kept to aminimum because the Scopus identification number wasapplied for author identification and all references wereaccurately checked and any incorrect references wereremoved

Conclusion

This study provides an original first benchmark useful toassess surgeon scientistsrsquo productivity in the field of liversurgery The h-index can be considered an accurate tool inthe determination of the quality and quantity of publishedwork Because the h-index calculation is influenced by thedatabase used the report of a scientistrsquos h-index mustinclude the database used for computation

Acknowledgment

The authors thank Susan West for her writing assistanceThe h-index of the corresponding Author calculated withGoogle Scholar can be found here httpscholargoogleitcitationsuser=6V_7iUEAAAAJamphl=it

References

1 Ball P Index aims for fair ranking of scientists Nature 2005436900

2 Hirsch JE Does the H index have predictive power Proc Natl Acad

Sci USA 200710419193ndash8

3 Hirsch JE An index to quantify an individualrsquos scientific research out-

put Proc Natl Acad Sci USA 200510216569ndash72

4 Egghe L Theory and practice of the g-index Scientometrics 200669

131ndash52

5 Jin B The AR-index complementing the h-index ISSI Newsletter

199736

6 Lee J Kraus KL Couldwell WT Use of the h index in neurosurgery

J Neurosurg 2009111387ndash92

7 Horne R Petrie KH Wessely S H-index pathology implications for

medical researchers and practitioners BMJ 2009339b5356

8 Ball P Achievement index climbs the ranks Nature 2007448737

9 Fersht A The most influential journals impact factor and Eigenfactor

Proc Natl Acad Sci USA 20091066883ndash4

10 Podlubny I Kassayova K Law of the constant ratio Towards a better

list of citation superstars compiling a multidisciplinary list of highly

cited researchers Res Eval 200615154ndash62

11 Poynard T Thabut D Jabre P et al Ranking hepatologists which

Hirschrsquos h-index to prevent the lsquolsquoe-crise de foi-ersquorsquo Clin Res Hepatol

Gastroenterol 201135375ndash86

12 Thomson Reuters Community Forum Available at httpcommunity

thomsonreuterscom Accessed January 2 2012

13 Harzing AW Publish or Perish 2007 Available at httpwww

harzingcompophtm Accessed June 5 2013

14 Ashrafian H Rao C Darzi A Benchmarking in surgical research Lan-

cet 20093741045ndash7

15 Agenzia Nazionale di Valutazione del Sistema Universitario e della Ri-

cerca Available at httpwwwanvurorgq5en Accessed June 5 2013

16 Costas R Bordons M The h-index advantages limitations and its re-

lation with other bibliometric indicators at the micro level J Informetr

20071193ndash203

17 Martin BR The use of multiple indicators in the assessment of basic

research Scientometrics 199636343ndash62

18 Mozilla Available at httpswwwmatunicalitianniwiki

ScholarHIndexCalculator Accessed January 10 2011

000

The Use of the Hirsch Index in BenchmarkingHepatic Surgery Research

Alessandro Cucchetti Federico Mazzotti Sara PellegriniMatteo Cescon Lorenzo Maroni Giorgio Ercolani andAntonio Daniele Pinna

Rapid progress in hepatic surgery over the pastdecades has been accompanied by vast increases inresearch publications The citation metric that hasbeen most actively adopted is the h-index (Hirschindex) defined as the number of h publications citedat least h-times in the literature The scientific ageof surgeon scientists working in the field of hepaticsurgery was plotted against h-index percentiles inan h-index growth chart for both the Scopus data-base and Google Scholar providing a first bench-mark to assess surgeon scientistsrsquo productivity inthe field of liver surgery

Table 2 Relationships observed between each bibliographicmeasure considered and Google Scholar and Scopus Hirschindexes

Bibliographicmeasure

Scopush-index

Google Scholarh-index

Scopus total cited documents 945 827Scopus total citation 917 762Google Scholar totalcited documents

780 958

Google Scholar total citation 770 914Google Scholar ACWR 768 912Scientific age 345 500

Values represent Spearman correlation (rho)

AWCR 5 age-weighted citation rate h-index 5 Hirsch index

4 The American Journal of Surgery Vol - No - - 2013

Scopus results the median number of cited documentsranged from 5 for authors with a scientific age less than10 years to 23 for authors with a scientific age greaterthan 30 years (P 5 001) with the median number of cita-tions from 59 to 602 (P 5 001) and the median h-indexfrom 3 to 9 (P 5 001) A similar relationship was observed

Table 3 Relationships observed between each bibliographic measurscientific age

Bibliographic measure Scopus

Scientific age 10 years (n 5 677)Median cited documents (range) 5 (1ndash164)25th75th95th percentile 21238

Median number of citations 59 (0ndash2437)25th75th95th percentile 20175547

Median h-index 3 (0ndash29)25th75th95th percentile 2511

Scientific age 11ndash20 years (n 5 1137)Median cited documents (range) 12 (1ndash423)25th75th95th percentile 33093

Median number of citations 273 (0ndash8875)25th75th95th percentile 846571833

Median h-index 6 (0ndash47)25th75th95th percentile 31221

Scientific age 21ndash30 years (n 5 729)Median cited documents (range) 22 (1ndash535)25th75th95th percentile 860169

Median number of citations 437 (0ndash14039)25th75th95th percentile 14411814045

Median h-index 9 (0ndash64)25th75th95th percentile 51734

Scientific age 30 years (n 5 708)Median cited documents (range) 23 (1ndash938)25th75th95th percentile 969194

Median number of citations 602 (0ndash18004)25th75th95th percentile 10911764348

Median h-index 9 (0ndash81)25th75th95th percentile 41733

h-index 5 Hirsch index

P values are referred to the Wilcoxon signed-rank testdaggerRho is the results of Spearman correlation between Scopus and Google Sc

for Google Scholar results the median number of citeddocuments ranged from 4 to 465 (P 5 001) the mediannumber of citations from 52 to 1017 (P 5 001) and themedian h-index from 2 to 15 (P 5 001) respectivelyThese 3 bibliographic measures change according to the da-tabase used for calculation and the scientific age In partic-ular it should be noted that for authors with a scientificage less than 10 years the number of cited documents(P 5 001) number of citations (P 5 019) and h-index(P 5 001) are higher even if only slightly with Scopuswhen compared with Google Scholar results These differ-ences become less pronounced for authors with a scientificage between 10 and 20 years (P 5 049 001 and 020 re-spectively) and from the 21st year onward Google Scholargives higher values than Scopus database results (P 5001in all cases)

The Hirsch index growth curve

In Fig 2 scientific age was plotted against h-indexes andvalues interpolated to obtain an h-index growth chart forboth the Scopus database (Fig 2A) and Google Scholar(Fig 2B) These growth charts consist of a series of

e considered and Google Scholar and Scopus databases by

Google Scholar P Rhodagger

4 (1ndash90) 001 817292652 (0ndash2065) 019 810141585352 (0ndash20) 001 7851510

11 (1ndash296) 049 80552369240 (0ndash14802) 001 609127717204255 (0ndash57) 020 82921121

23 (1ndash906) 001 831957185617 (0ndash19846) 001 6412141409449310 (0ndash75) 001 83851834

465 (1ndash1000) 001 883121204041017 (0ndash26377) 001 6592902464814915 (0ndash117) 001 78662444

holar results

Figure 2 Hirsch-index (h-index) growth chart for Scopus data-base (A) and Google Scholar (B) (C) Reports the difference inthe median curve (50th percentile) between the 2 databases

A Cucchetti et al Hirsch index in hepatic surgery research 5

percentile curves that illustrate the distribution of the h-in-dex obtained with Scopus or Google Scholar in relationshipwith scientific age in order to track how the h-index canchange over time Fig 2C reports the difference in the me-dian curve (50th percentile) between the 2 databases Con-sidering an author with a scientific age of 5 years and anh-index of 5 his productivity can be placed above the 75thpercentile of the hepatic surgery scientific community withboth Scopus and Google Scholar methods meaning thatthis author had a very good scientific production Consider-ing an author with a scientific age of 10 years and an h-indexof 18 his productivity can be placed above the 95th percen-tile of the hepatic surgery community meaning that this au-thor had an excellent scientific production Of particular noteis that percentile curves were different between Scopusrsquos andGoogle Scholarrsquos h-indexes In fact as can be observed fromFig 2C the difference between the 50th percentile h-indexesof Scopus and Google Scholar was minimal within the first10 to 15 years after which Google Scholar provided higherh-indexes than the Scopus database

The age-weighted citation rate

The possibility of younger and yet less cited articlesbeing included in a different bibliographic measure wasexplored by analyzing the age-weighted citation rate Thescatter-plot of Fig 3A effectively shows that the AWCRcan handle scientific age in a single measure (rho 358)However as can be observed from Fig 3BC the AWCRwas found to be strictly related to both the Google Scholarnumber of citations (rho 960) and the h-index (rho 912)

Comments

The assessment of academic research represents acritical issue of investigation because the determinationof the quality and quantity of published work is ofparamount importance both for employment purposes andfor funding allocation1ndash3 In the field of hepatic surgery ametric of comparison is still not available nonethelesssuch a benchmark seems to be important in the presenceof the remarkable development of liver surgery observedin recent decades14 Results from this study could suggeststandards for evaluating individuals in this specific surgicalarea The literature research and the author selection for theh-index calculation of the present study identified surgeonscientists on the basis of their h-index and their scientificage in a growth chart useful for the assessment of scientificresearch in the field of hepatic surgery worldwide

There are some significant results from this study thatdeserve particular attention Attention must be first focusedon the difference observed in h-indexes obtained fromScopus and Google Scholar databases in relationship withscientific age A weak monotonic relationship was foundbetween scientific age and h-index in fact after a scientificage of about 15 to 20 years Google Scholar gives higher

h-indexes than the Scopus database This is a result of theliterature availability being from only 1996 for the h-indexcalculation of Scopus This means that comparisons can bereliably performed between the 2 databases only for younger

Figure 3 Relationship observed between the age-weighted ci-tation rate (AWCR) and scientific age (A rho 358) numberof citations (B rho 960) and the Hirsch index (h-index)(C rho 912)

6 The American Journal of Surgery Vol - No - - 2013

surgeon scientists on the contrary this approach will give anincorrect comparison in older authors These observationssupport the suggestion of using the same database foracademic research comparison especially for older authorsLooking at the growth h-index curves provided the findingthat there are some younger authors (around the 95thpercentile) who have an h-index that grossly correspondsto the 25th percentile of older ones is of interest This findingindicates that there are relatively young surgeons who havealready produced a relevant amount of scientific workscomparable or even higher to that of older surgeons Thusthe chronological age of a scientist should not represent anobstacle for employment in the academic field fundingallocation or career progression This observation supportsthe need to assess academic activity on the number of high-quality research reports rather than on academic ageApplication of a global surgical benchmark would allowthe constructive appraisal of academic surgery to achieve itsfuture goals especially among young scientists14

The second result that should attract attention is therelationship observed between the various bibliometricmeasures considered and the databases used In fact goodcorrelations were found between the number of citeddocuments the number of citations and the h-index inboth Scopus and Google Scholar databases It should alsobe noted that the age-weighted citation was found to bestrictly related to both h-index and the number of citationsThese findings mean that all bibliometric measures that arecommonly used to assess relevance of a body of work havesimilar accuracy however in a short period of time the h-index has become a widely recognized measure of quan-tifying an individualrsquos research output and to compareacademic and public institutions with one another4ndash11

Given its widespread use the proposed h-index benchmarkprovides a useful metric in the comparison of research inthe present surgical field with important practical aspectsin the evaluation of the academic career of surgeons Incomparing the h-index of candidates by means of the pre-sent growth curves institutions would be able to identifythe candidate who is best qualified for career advancementfor example in Italy the National Agency for the Evalua-tion of Universities and Research Institutes established in2012 a national scientific evaluation for career advance-ment toward associate professor or full professor basedon the h-index15 Similarly centers of academic surgicalexcellence can be recognized and targeted as sites of signif-icant academic recruitment and research output14

Some comments should be reserved for the limits of theh-index16 First this metric may not fully capture the qual-ity of a scientific research a published article could be citednot because of its quality but rather because of its poorvalue that raised harsh criticism Unfortunately this repre-sents an unavoidable limit of this citation metric Secondthe typical interfield differences in the h-index valuescaused by differences among fields in productivity and ci-tation practices can also be found when the field was

A Cucchetti et al Hirsch index in hepatic surgery research 7

selected as was found in this analysis In fact in the area ofhepatic surgery intrafield differences could probably be ex-pected among surgeons who perform only general hepaticsurgery and surgeons involved in liver transplantationand this is an aspect that probably requires further dedi-cated investigations Third the h-index can be affected byjournal impact factor or citation index In fact it can be ex-pected that the higher the rank of a scientific journal thehigher the probability would be of a specific article pub-lished in such a journal being cited by other authors be-cause such a journal would be considered moreauthoritative in the field An indirect measure of such a re-lationship can probably be obtained from calculating the g-index which aims to improve the h-index by giving moreweight to highly cited articles which articles published inmore authoritative scientific journals can be This repre-sents another aspect that deserves further analysis4 Allthese observations highlight the need for caution in theuse of the h-index avoiding indiscriminate use because re-search performance has to be considered a complex multi-faceted endeavor that should not be underestimated17

The literature research and the author selection used inthe present study can have potential shortcomings It ispossible that some authors who have significantly contrib-uted to the growth of liver surgery were left out from thesearch criteria It is possible in fact that some surgeonswho have contributed to liver transplantation developmentmay not have been included because they published injournals other than those included for example transplan-tation targeted journals The exclusion of transplantationjournals in the inclusion criteria was determined a priori toavoid the inclusion of other physician figures such asanesthesiologists or hepatologists11 It should be notedhowever that is fairly unlikely that a surgeon who has re-ally contributed to the development of liver surgery hasnever published a scientific article in 1 of the 4 most impor-tant surgical journals This does not mean that these figureshave not played a decisive role in the growth of liver sur-gery but it contrasts with the aim of this study whichwas to identify surgeons who have shaped liver surgeryas we now know it Attention should also be given to theother databases for h-index calculation such as ISIrsquos Webof Knowledge database12 or Firefoxrsquos add-in for GoogleScholar18 Although it was not the aim of this study to com-pare h-index calculation to other databases this featureprobably represents an issue that deserves further investiga-tion Another item to note would be the potential bias rep-resented by homonyms which in this study were kept to aminimum because the Scopus identification number wasapplied for author identification and all references wereaccurately checked and any incorrect references wereremoved

Conclusion

This study provides an original first benchmark useful toassess surgeon scientistsrsquo productivity in the field of liversurgery The h-index can be considered an accurate tool inthe determination of the quality and quantity of publishedwork Because the h-index calculation is influenced by thedatabase used the report of a scientistrsquos h-index mustinclude the database used for computation

Acknowledgment

The authors thank Susan West for her writing assistanceThe h-index of the corresponding Author calculated withGoogle Scholar can be found here httpscholargoogleitcitationsuser=6V_7iUEAAAAJamphl=it

References

1 Ball P Index aims for fair ranking of scientists Nature 2005436900

2 Hirsch JE Does the H index have predictive power Proc Natl Acad

Sci USA 200710419193ndash8

3 Hirsch JE An index to quantify an individualrsquos scientific research out-

put Proc Natl Acad Sci USA 200510216569ndash72

4 Egghe L Theory and practice of the g-index Scientometrics 200669

131ndash52

5 Jin B The AR-index complementing the h-index ISSI Newsletter

199736

6 Lee J Kraus KL Couldwell WT Use of the h index in neurosurgery

J Neurosurg 2009111387ndash92

7 Horne R Petrie KH Wessely S H-index pathology implications for

medical researchers and practitioners BMJ 2009339b5356

8 Ball P Achievement index climbs the ranks Nature 2007448737

9 Fersht A The most influential journals impact factor and Eigenfactor

Proc Natl Acad Sci USA 20091066883ndash4

10 Podlubny I Kassayova K Law of the constant ratio Towards a better

list of citation superstars compiling a multidisciplinary list of highly

cited researchers Res Eval 200615154ndash62

11 Poynard T Thabut D Jabre P et al Ranking hepatologists which

Hirschrsquos h-index to prevent the lsquolsquoe-crise de foi-ersquorsquo Clin Res Hepatol

Gastroenterol 201135375ndash86

12 Thomson Reuters Community Forum Available at httpcommunity

thomsonreuterscom Accessed January 2 2012

13 Harzing AW Publish or Perish 2007 Available at httpwww

harzingcompophtm Accessed June 5 2013

14 Ashrafian H Rao C Darzi A Benchmarking in surgical research Lan-

cet 20093741045ndash7

15 Agenzia Nazionale di Valutazione del Sistema Universitario e della Ri-

cerca Available at httpwwwanvurorgq5en Accessed June 5 2013

16 Costas R Bordons M The h-index advantages limitations and its re-

lation with other bibliometric indicators at the micro level J Informetr

20071193ndash203

17 Martin BR The use of multiple indicators in the assessment of basic

research Scientometrics 199636343ndash62

18 Mozilla Available at httpswwwmatunicalitianniwiki

ScholarHIndexCalculator Accessed January 10 2011

000

The Use of the Hirsch Index in BenchmarkingHepatic Surgery Research

Alessandro Cucchetti Federico Mazzotti Sara PellegriniMatteo Cescon Lorenzo Maroni Giorgio Ercolani andAntonio Daniele Pinna

Rapid progress in hepatic surgery over the pastdecades has been accompanied by vast increases inresearch publications The citation metric that hasbeen most actively adopted is the h-index (Hirschindex) defined as the number of h publications citedat least h-times in the literature The scientific ageof surgeon scientists working in the field of hepaticsurgery was plotted against h-index percentiles inan h-index growth chart for both the Scopus data-base and Google Scholar providing a first bench-mark to assess surgeon scientistsrsquo productivity inthe field of liver surgery

Figure 2 Hirsch-index (h-index) growth chart for Scopus data-base (A) and Google Scholar (B) (C) Reports the difference inthe median curve (50th percentile) between the 2 databases

A Cucchetti et al Hirsch index in hepatic surgery research 5

percentile curves that illustrate the distribution of the h-in-dex obtained with Scopus or Google Scholar in relationshipwith scientific age in order to track how the h-index canchange over time Fig 2C reports the difference in the me-dian curve (50th percentile) between the 2 databases Con-sidering an author with a scientific age of 5 years and anh-index of 5 his productivity can be placed above the 75thpercentile of the hepatic surgery scientific community withboth Scopus and Google Scholar methods meaning thatthis author had a very good scientific production Consider-ing an author with a scientific age of 10 years and an h-indexof 18 his productivity can be placed above the 95th percen-tile of the hepatic surgery community meaning that this au-thor had an excellent scientific production Of particular noteis that percentile curves were different between Scopusrsquos andGoogle Scholarrsquos h-indexes In fact as can be observed fromFig 2C the difference between the 50th percentile h-indexesof Scopus and Google Scholar was minimal within the first10 to 15 years after which Google Scholar provided higherh-indexes than the Scopus database

The age-weighted citation rate

The possibility of younger and yet less cited articlesbeing included in a different bibliographic measure wasexplored by analyzing the age-weighted citation rate Thescatter-plot of Fig 3A effectively shows that the AWCRcan handle scientific age in a single measure (rho 358)However as can be observed from Fig 3BC the AWCRwas found to be strictly related to both the Google Scholarnumber of citations (rho 960) and the h-index (rho 912)

Comments

The assessment of academic research represents acritical issue of investigation because the determinationof the quality and quantity of published work is ofparamount importance both for employment purposes andfor funding allocation1ndash3 In the field of hepatic surgery ametric of comparison is still not available nonethelesssuch a benchmark seems to be important in the presenceof the remarkable development of liver surgery observedin recent decades14 Results from this study could suggeststandards for evaluating individuals in this specific surgicalarea The literature research and the author selection for theh-index calculation of the present study identified surgeonscientists on the basis of their h-index and their scientificage in a growth chart useful for the assessment of scientificresearch in the field of hepatic surgery worldwide

There are some significant results from this study thatdeserve particular attention Attention must be first focusedon the difference observed in h-indexes obtained fromScopus and Google Scholar databases in relationship withscientific age A weak monotonic relationship was foundbetween scientific age and h-index in fact after a scientificage of about 15 to 20 years Google Scholar gives higher

h-indexes than the Scopus database This is a result of theliterature availability being from only 1996 for the h-indexcalculation of Scopus This means that comparisons can bereliably performed between the 2 databases only for younger

Figure 3 Relationship observed between the age-weighted ci-tation rate (AWCR) and scientific age (A rho 358) numberof citations (B rho 960) and the Hirsch index (h-index)(C rho 912)

6 The American Journal of Surgery Vol - No - - 2013

surgeon scientists on the contrary this approach will give anincorrect comparison in older authors These observationssupport the suggestion of using the same database foracademic research comparison especially for older authorsLooking at the growth h-index curves provided the findingthat there are some younger authors (around the 95thpercentile) who have an h-index that grossly correspondsto the 25th percentile of older ones is of interest This findingindicates that there are relatively young surgeons who havealready produced a relevant amount of scientific workscomparable or even higher to that of older surgeons Thusthe chronological age of a scientist should not represent anobstacle for employment in the academic field fundingallocation or career progression This observation supportsthe need to assess academic activity on the number of high-quality research reports rather than on academic ageApplication of a global surgical benchmark would allowthe constructive appraisal of academic surgery to achieve itsfuture goals especially among young scientists14

The second result that should attract attention is therelationship observed between the various bibliometricmeasures considered and the databases used In fact goodcorrelations were found between the number of citeddocuments the number of citations and the h-index inboth Scopus and Google Scholar databases It should alsobe noted that the age-weighted citation was found to bestrictly related to both h-index and the number of citationsThese findings mean that all bibliometric measures that arecommonly used to assess relevance of a body of work havesimilar accuracy however in a short period of time the h-index has become a widely recognized measure of quan-tifying an individualrsquos research output and to compareacademic and public institutions with one another4ndash11

Given its widespread use the proposed h-index benchmarkprovides a useful metric in the comparison of research inthe present surgical field with important practical aspectsin the evaluation of the academic career of surgeons Incomparing the h-index of candidates by means of the pre-sent growth curves institutions would be able to identifythe candidate who is best qualified for career advancementfor example in Italy the National Agency for the Evalua-tion of Universities and Research Institutes established in2012 a national scientific evaluation for career advance-ment toward associate professor or full professor basedon the h-index15 Similarly centers of academic surgicalexcellence can be recognized and targeted as sites of signif-icant academic recruitment and research output14

Some comments should be reserved for the limits of theh-index16 First this metric may not fully capture the qual-ity of a scientific research a published article could be citednot because of its quality but rather because of its poorvalue that raised harsh criticism Unfortunately this repre-sents an unavoidable limit of this citation metric Secondthe typical interfield differences in the h-index valuescaused by differences among fields in productivity and ci-tation practices can also be found when the field was

A Cucchetti et al Hirsch index in hepatic surgery research 7

selected as was found in this analysis In fact in the area ofhepatic surgery intrafield differences could probably be ex-pected among surgeons who perform only general hepaticsurgery and surgeons involved in liver transplantationand this is an aspect that probably requires further dedi-cated investigations Third the h-index can be affected byjournal impact factor or citation index In fact it can be ex-pected that the higher the rank of a scientific journal thehigher the probability would be of a specific article pub-lished in such a journal being cited by other authors be-cause such a journal would be considered moreauthoritative in the field An indirect measure of such a re-lationship can probably be obtained from calculating the g-index which aims to improve the h-index by giving moreweight to highly cited articles which articles published inmore authoritative scientific journals can be This repre-sents another aspect that deserves further analysis4 Allthese observations highlight the need for caution in theuse of the h-index avoiding indiscriminate use because re-search performance has to be considered a complex multi-faceted endeavor that should not be underestimated17

The literature research and the author selection used inthe present study can have potential shortcomings It ispossible that some authors who have significantly contrib-uted to the growth of liver surgery were left out from thesearch criteria It is possible in fact that some surgeonswho have contributed to liver transplantation developmentmay not have been included because they published injournals other than those included for example transplan-tation targeted journals The exclusion of transplantationjournals in the inclusion criteria was determined a priori toavoid the inclusion of other physician figures such asanesthesiologists or hepatologists11 It should be notedhowever that is fairly unlikely that a surgeon who has re-ally contributed to the development of liver surgery hasnever published a scientific article in 1 of the 4 most impor-tant surgical journals This does not mean that these figureshave not played a decisive role in the growth of liver sur-gery but it contrasts with the aim of this study whichwas to identify surgeons who have shaped liver surgeryas we now know it Attention should also be given to theother databases for h-index calculation such as ISIrsquos Webof Knowledge database12 or Firefoxrsquos add-in for GoogleScholar18 Although it was not the aim of this study to com-pare h-index calculation to other databases this featureprobably represents an issue that deserves further investiga-tion Another item to note would be the potential bias rep-resented by homonyms which in this study were kept to aminimum because the Scopus identification number wasapplied for author identification and all references wereaccurately checked and any incorrect references wereremoved

Conclusion

This study provides an original first benchmark useful toassess surgeon scientistsrsquo productivity in the field of liversurgery The h-index can be considered an accurate tool inthe determination of the quality and quantity of publishedwork Because the h-index calculation is influenced by thedatabase used the report of a scientistrsquos h-index mustinclude the database used for computation

Acknowledgment

The authors thank Susan West for her writing assistanceThe h-index of the corresponding Author calculated withGoogle Scholar can be found here httpscholargoogleitcitationsuser=6V_7iUEAAAAJamphl=it

References

1 Ball P Index aims for fair ranking of scientists Nature 2005436900

2 Hirsch JE Does the H index have predictive power Proc Natl Acad

Sci USA 200710419193ndash8

3 Hirsch JE An index to quantify an individualrsquos scientific research out-

put Proc Natl Acad Sci USA 200510216569ndash72

4 Egghe L Theory and practice of the g-index Scientometrics 200669

131ndash52

5 Jin B The AR-index complementing the h-index ISSI Newsletter

199736

6 Lee J Kraus KL Couldwell WT Use of the h index in neurosurgery

J Neurosurg 2009111387ndash92

7 Horne R Petrie KH Wessely S H-index pathology implications for

medical researchers and practitioners BMJ 2009339b5356

8 Ball P Achievement index climbs the ranks Nature 2007448737

9 Fersht A The most influential journals impact factor and Eigenfactor

Proc Natl Acad Sci USA 20091066883ndash4

10 Podlubny I Kassayova K Law of the constant ratio Towards a better

list of citation superstars compiling a multidisciplinary list of highly

cited researchers Res Eval 200615154ndash62

11 Poynard T Thabut D Jabre P et al Ranking hepatologists which

Hirschrsquos h-index to prevent the lsquolsquoe-crise de foi-ersquorsquo Clin Res Hepatol

Gastroenterol 201135375ndash86

12 Thomson Reuters Community Forum Available at httpcommunity

thomsonreuterscom Accessed January 2 2012

13 Harzing AW Publish or Perish 2007 Available at httpwww

harzingcompophtm Accessed June 5 2013

14 Ashrafian H Rao C Darzi A Benchmarking in surgical research Lan-

cet 20093741045ndash7

15 Agenzia Nazionale di Valutazione del Sistema Universitario e della Ri-

cerca Available at httpwwwanvurorgq5en Accessed June 5 2013

16 Costas R Bordons M The h-index advantages limitations and its re-

lation with other bibliometric indicators at the micro level J Informetr

20071193ndash203

17 Martin BR The use of multiple indicators in the assessment of basic

research Scientometrics 199636343ndash62

18 Mozilla Available at httpswwwmatunicalitianniwiki

ScholarHIndexCalculator Accessed January 10 2011

000

The Use of the Hirsch Index in BenchmarkingHepatic Surgery Research

Alessandro Cucchetti Federico Mazzotti Sara PellegriniMatteo Cescon Lorenzo Maroni Giorgio Ercolani andAntonio Daniele Pinna

Rapid progress in hepatic surgery over the pastdecades has been accompanied by vast increases inresearch publications The citation metric that hasbeen most actively adopted is the h-index (Hirschindex) defined as the number of h publications citedat least h-times in the literature The scientific ageof surgeon scientists working in the field of hepaticsurgery was plotted against h-index percentiles inan h-index growth chart for both the Scopus data-base and Google Scholar providing a first bench-mark to assess surgeon scientistsrsquo productivity inthe field of liver surgery

Figure 3 Relationship observed between the age-weighted ci-tation rate (AWCR) and scientific age (A rho 358) numberof citations (B rho 960) and the Hirsch index (h-index)(C rho 912)

6 The American Journal of Surgery Vol - No - - 2013

surgeon scientists on the contrary this approach will give anincorrect comparison in older authors These observationssupport the suggestion of using the same database foracademic research comparison especially for older authorsLooking at the growth h-index curves provided the findingthat there are some younger authors (around the 95thpercentile) who have an h-index that grossly correspondsto the 25th percentile of older ones is of interest This findingindicates that there are relatively young surgeons who havealready produced a relevant amount of scientific workscomparable or even higher to that of older surgeons Thusthe chronological age of a scientist should not represent anobstacle for employment in the academic field fundingallocation or career progression This observation supportsthe need to assess academic activity on the number of high-quality research reports rather than on academic ageApplication of a global surgical benchmark would allowthe constructive appraisal of academic surgery to achieve itsfuture goals especially among young scientists14

The second result that should attract attention is therelationship observed between the various bibliometricmeasures considered and the databases used In fact goodcorrelations were found between the number of citeddocuments the number of citations and the h-index inboth Scopus and Google Scholar databases It should alsobe noted that the age-weighted citation was found to bestrictly related to both h-index and the number of citationsThese findings mean that all bibliometric measures that arecommonly used to assess relevance of a body of work havesimilar accuracy however in a short period of time the h-index has become a widely recognized measure of quan-tifying an individualrsquos research output and to compareacademic and public institutions with one another4ndash11

Given its widespread use the proposed h-index benchmarkprovides a useful metric in the comparison of research inthe present surgical field with important practical aspectsin the evaluation of the academic career of surgeons Incomparing the h-index of candidates by means of the pre-sent growth curves institutions would be able to identifythe candidate who is best qualified for career advancementfor example in Italy the National Agency for the Evalua-tion of Universities and Research Institutes established in2012 a national scientific evaluation for career advance-ment toward associate professor or full professor basedon the h-index15 Similarly centers of academic surgicalexcellence can be recognized and targeted as sites of signif-icant academic recruitment and research output14

Some comments should be reserved for the limits of theh-index16 First this metric may not fully capture the qual-ity of a scientific research a published article could be citednot because of its quality but rather because of its poorvalue that raised harsh criticism Unfortunately this repre-sents an unavoidable limit of this citation metric Secondthe typical interfield differences in the h-index valuescaused by differences among fields in productivity and ci-tation practices can also be found when the field was

A Cucchetti et al Hirsch index in hepatic surgery research 7

selected as was found in this analysis In fact in the area ofhepatic surgery intrafield differences could probably be ex-pected among surgeons who perform only general hepaticsurgery and surgeons involved in liver transplantationand this is an aspect that probably requires further dedi-cated investigations Third the h-index can be affected byjournal impact factor or citation index In fact it can be ex-pected that the higher the rank of a scientific journal thehigher the probability would be of a specific article pub-lished in such a journal being cited by other authors be-cause such a journal would be considered moreauthoritative in the field An indirect measure of such a re-lationship can probably be obtained from calculating the g-index which aims to improve the h-index by giving moreweight to highly cited articles which articles published inmore authoritative scientific journals can be This repre-sents another aspect that deserves further analysis4 Allthese observations highlight the need for caution in theuse of the h-index avoiding indiscriminate use because re-search performance has to be considered a complex multi-faceted endeavor that should not be underestimated17

The literature research and the author selection used inthe present study can have potential shortcomings It ispossible that some authors who have significantly contrib-uted to the growth of liver surgery were left out from thesearch criteria It is possible in fact that some surgeonswho have contributed to liver transplantation developmentmay not have been included because they published injournals other than those included for example transplan-tation targeted journals The exclusion of transplantationjournals in the inclusion criteria was determined a priori toavoid the inclusion of other physician figures such asanesthesiologists or hepatologists11 It should be notedhowever that is fairly unlikely that a surgeon who has re-ally contributed to the development of liver surgery hasnever published a scientific article in 1 of the 4 most impor-tant surgical journals This does not mean that these figureshave not played a decisive role in the growth of liver sur-gery but it contrasts with the aim of this study whichwas to identify surgeons who have shaped liver surgeryas we now know it Attention should also be given to theother databases for h-index calculation such as ISIrsquos Webof Knowledge database12 or Firefoxrsquos add-in for GoogleScholar18 Although it was not the aim of this study to com-pare h-index calculation to other databases this featureprobably represents an issue that deserves further investiga-tion Another item to note would be the potential bias rep-resented by homonyms which in this study were kept to aminimum because the Scopus identification number wasapplied for author identification and all references wereaccurately checked and any incorrect references wereremoved

Conclusion

This study provides an original first benchmark useful toassess surgeon scientistsrsquo productivity in the field of liversurgery The h-index can be considered an accurate tool inthe determination of the quality and quantity of publishedwork Because the h-index calculation is influenced by thedatabase used the report of a scientistrsquos h-index mustinclude the database used for computation

Acknowledgment

The authors thank Susan West for her writing assistanceThe h-index of the corresponding Author calculated withGoogle Scholar can be found here httpscholargoogleitcitationsuser=6V_7iUEAAAAJamphl=it

References

1 Ball P Index aims for fair ranking of scientists Nature 2005436900

2 Hirsch JE Does the H index have predictive power Proc Natl Acad

Sci USA 200710419193ndash8

3 Hirsch JE An index to quantify an individualrsquos scientific research out-

put Proc Natl Acad Sci USA 200510216569ndash72

4 Egghe L Theory and practice of the g-index Scientometrics 200669

131ndash52

5 Jin B The AR-index complementing the h-index ISSI Newsletter

199736

6 Lee J Kraus KL Couldwell WT Use of the h index in neurosurgery

J Neurosurg 2009111387ndash92

7 Horne R Petrie KH Wessely S H-index pathology implications for

medical researchers and practitioners BMJ 2009339b5356

8 Ball P Achievement index climbs the ranks Nature 2007448737

9 Fersht A The most influential journals impact factor and Eigenfactor

Proc Natl Acad Sci USA 20091066883ndash4

10 Podlubny I Kassayova K Law of the constant ratio Towards a better

list of citation superstars compiling a multidisciplinary list of highly

cited researchers Res Eval 200615154ndash62

11 Poynard T Thabut D Jabre P et al Ranking hepatologists which

Hirschrsquos h-index to prevent the lsquolsquoe-crise de foi-ersquorsquo Clin Res Hepatol

Gastroenterol 201135375ndash86

12 Thomson Reuters Community Forum Available at httpcommunity

thomsonreuterscom Accessed January 2 2012

13 Harzing AW Publish or Perish 2007 Available at httpwww

harzingcompophtm Accessed June 5 2013

14 Ashrafian H Rao C Darzi A Benchmarking in surgical research Lan-

cet 20093741045ndash7

15 Agenzia Nazionale di Valutazione del Sistema Universitario e della Ri-

cerca Available at httpwwwanvurorgq5en Accessed June 5 2013

16 Costas R Bordons M The h-index advantages limitations and its re-

lation with other bibliometric indicators at the micro level J Informetr

20071193ndash203

17 Martin BR The use of multiple indicators in the assessment of basic

research Scientometrics 199636343ndash62

18 Mozilla Available at httpswwwmatunicalitianniwiki

ScholarHIndexCalculator Accessed January 10 2011

000

The Use of the Hirsch Index in BenchmarkingHepatic Surgery Research

Alessandro Cucchetti Federico Mazzotti Sara PellegriniMatteo Cescon Lorenzo Maroni Giorgio Ercolani andAntonio Daniele Pinna

Rapid progress in hepatic surgery over the pastdecades has been accompanied by vast increases inresearch publications The citation metric that hasbeen most actively adopted is the h-index (Hirschindex) defined as the number of h publications citedat least h-times in the literature The scientific ageof surgeon scientists working in the field of hepaticsurgery was plotted against h-index percentiles inan h-index growth chart for both the Scopus data-base and Google Scholar providing a first bench-mark to assess surgeon scientistsrsquo productivity inthe field of liver surgery

A Cucchetti et al Hirsch index in hepatic surgery research 7

selected as was found in this analysis In fact in the area ofhepatic surgery intrafield differences could probably be ex-pected among surgeons who perform only general hepaticsurgery and surgeons involved in liver transplantationand this is an aspect that probably requires further dedi-cated investigations Third the h-index can be affected byjournal impact factor or citation index In fact it can be ex-pected that the higher the rank of a scientific journal thehigher the probability would be of a specific article pub-lished in such a journal being cited by other authors be-cause such a journal would be considered moreauthoritative in the field An indirect measure of such a re-lationship can probably be obtained from calculating the g-index which aims to improve the h-index by giving moreweight to highly cited articles which articles published inmore authoritative scientific journals can be This repre-sents another aspect that deserves further analysis4 Allthese observations highlight the need for caution in theuse of the h-index avoiding indiscriminate use because re-search performance has to be considered a complex multi-faceted endeavor that should not be underestimated17

The literature research and the author selection used inthe present study can have potential shortcomings It ispossible that some authors who have significantly contrib-uted to the growth of liver surgery were left out from thesearch criteria It is possible in fact that some surgeonswho have contributed to liver transplantation developmentmay not have been included because they published injournals other than those included for example transplan-tation targeted journals The exclusion of transplantationjournals in the inclusion criteria was determined a priori toavoid the inclusion of other physician figures such asanesthesiologists or hepatologists11 It should be notedhowever that is fairly unlikely that a surgeon who has re-ally contributed to the development of liver surgery hasnever published a scientific article in 1 of the 4 most impor-tant surgical journals This does not mean that these figureshave not played a decisive role in the growth of liver sur-gery but it contrasts with the aim of this study whichwas to identify surgeons who have shaped liver surgeryas we now know it Attention should also be given to theother databases for h-index calculation such as ISIrsquos Webof Knowledge database12 or Firefoxrsquos add-in for GoogleScholar18 Although it was not the aim of this study to com-pare h-index calculation to other databases this featureprobably represents an issue that deserves further investiga-tion Another item to note would be the potential bias rep-resented by homonyms which in this study were kept to aminimum because the Scopus identification number wasapplied for author identification and all references wereaccurately checked and any incorrect references wereremoved

Conclusion

This study provides an original first benchmark useful toassess surgeon scientistsrsquo productivity in the field of liversurgery The h-index can be considered an accurate tool inthe determination of the quality and quantity of publishedwork Because the h-index calculation is influenced by thedatabase used the report of a scientistrsquos h-index mustinclude the database used for computation

Acknowledgment

The authors thank Susan West for her writing assistanceThe h-index of the corresponding Author calculated withGoogle Scholar can be found here httpscholargoogleitcitationsuser=6V_7iUEAAAAJamphl=it

References

1 Ball P Index aims for fair ranking of scientists Nature 2005436900

2 Hirsch JE Does the H index have predictive power Proc Natl Acad

Sci USA 200710419193ndash8

3 Hirsch JE An index to quantify an individualrsquos scientific research out-

put Proc Natl Acad Sci USA 200510216569ndash72

4 Egghe L Theory and practice of the g-index Scientometrics 200669

131ndash52

5 Jin B The AR-index complementing the h-index ISSI Newsletter

199736

6 Lee J Kraus KL Couldwell WT Use of the h index in neurosurgery

J Neurosurg 2009111387ndash92

7 Horne R Petrie KH Wessely S H-index pathology implications for

medical researchers and practitioners BMJ 2009339b5356

8 Ball P Achievement index climbs the ranks Nature 2007448737

9 Fersht A The most influential journals impact factor and Eigenfactor

Proc Natl Acad Sci USA 20091066883ndash4

10 Podlubny I Kassayova K Law of the constant ratio Towards a better

list of citation superstars compiling a multidisciplinary list of highly

cited researchers Res Eval 200615154ndash62

11 Poynard T Thabut D Jabre P et al Ranking hepatologists which

Hirschrsquos h-index to prevent the lsquolsquoe-crise de foi-ersquorsquo Clin Res Hepatol

Gastroenterol 201135375ndash86

12 Thomson Reuters Community Forum Available at httpcommunity

thomsonreuterscom Accessed January 2 2012

13 Harzing AW Publish or Perish 2007 Available at httpwww

harzingcompophtm Accessed June 5 2013

14 Ashrafian H Rao C Darzi A Benchmarking in surgical research Lan-

cet 20093741045ndash7

15 Agenzia Nazionale di Valutazione del Sistema Universitario e della Ri-

cerca Available at httpwwwanvurorgq5en Accessed June 5 2013

16 Costas R Bordons M The h-index advantages limitations and its re-

lation with other bibliometric indicators at the micro level J Informetr

20071193ndash203

17 Martin BR The use of multiple indicators in the assessment of basic

research Scientometrics 199636343ndash62

18 Mozilla Available at httpswwwmatunicalitianniwiki

ScholarHIndexCalculator Accessed January 10 2011

000

The Use of the Hirsch Index in BenchmarkingHepatic Surgery Research

Alessandro Cucchetti Federico Mazzotti Sara PellegriniMatteo Cescon Lorenzo Maroni Giorgio Ercolani andAntonio Daniele Pinna

Rapid progress in hepatic surgery over the pastdecades has been accompanied by vast increases inresearch publications The citation metric that hasbeen most actively adopted is the h-index (Hirschindex) defined as the number of h publications citedat least h-times in the literature The scientific ageof surgeon scientists working in the field of hepaticsurgery was plotted against h-index percentiles inan h-index growth chart for both the Scopus data-base and Google Scholar providing a first bench-mark to assess surgeon scientistsrsquo productivity inthe field of liver surgery

000

The Use of the Hirsch Index in BenchmarkingHepatic Surgery Research

Alessandro Cucchetti Federico Mazzotti Sara PellegriniMatteo Cescon Lorenzo Maroni Giorgio Ercolani andAntonio Daniele Pinna

Rapid progress in hepatic surgery over the pastdecades has been accompanied by vast increases inresearch publications The citation metric that hasbeen most actively adopted is the h-index (Hirschindex) defined as the number of h publications citedat least h-times in the literature The scientific ageof surgeon scientists working in the field of hepaticsurgery was plotted against h-index percentiles inan h-index growth chart for both the Scopus data-base and Google Scholar providing a first bench-mark to assess surgeon scientistsrsquo productivity inthe field of liver surgery