The FASEB JournalVolume 22, Issue 8 p. 2623-2628
Life Sciences Forum
Free Access
Comparison of SCImago journal rank indicator with journal impact factor
Matthew E. Falagas, Vasilios D. Kouranos, Ricardo Arencibia-Jorge, Drosos E. Karageorgopoulos
First published: 11 April 2008
https://doi.org/10.1096/fj.08-107938
Citations: 2
ABSTRACT
The application of currently available sophisticated algorithms of citation analysis allows for the incorporation of the “quality” of citations in the evaluation of scientific journals. We sought to compare the newly introduced SCImago journal rank (SJR) indicator with the journal impact factor (IF). We retrieved relevant information from the official Web sites hosting the above indices and their source databases. The SJR indicator is an open-access resource, while the journal IF requires paid subscription. The SJR indicator (based on Scopus data) lists considerably more journal titles published in a wider variety of countries and languages, than the journal IF (based on Web of Science data). Both indices divide citations to a journal by articles of the journal, during a specific time period. However, contrary to the journal IF, the SJR indicator attributes different weight to citations depending on the “prestige” of the citing journal without the influence of journal self-citations; prestige is estimated with the application of the PageRank algorithm in the network of journals. In addition, the SJR indicator includes the total number of documents of a journal in the denominator of the relevant calculation, whereas the journal IF includes only “citable” articles (mainly original articles and reviews). A 3-yr period is analyzed in both indices but with the use of different approaches. Regarding the top 100 journals in the 2006 journal IF ranking order, the median absolute change in their ranking position with the use of the SJR indicator is 32 (1st quartile: 12; 3rd quartile: 75). Although further validation is warranted, the novel SJR indicator poses as a serious alternative to the well-established journal IF, mainly due to its openaccess nature, larger source database, and assessment of the quality of citations.—Falagas, M. E., Kouranos, V. D., Arencibia-Jorge, R., Karageorgopoulos, D. E. Comparison of SCImago journal rank indicator with journal impact factor. FASEB J. 22, 2623–2628 (2008)
THE EVALUATION OF THE QUALITY of research is important for various professional societies, individual scientists, scholarly institutions, and funding organizations (1). The quality of a scientific contribution is primarily estimated from the long-term impact that it has in science. The latter can be inferred from the citations in scientific articles that a contribution receives. These principles have been applied in the evaluation of scientific journals (2). The journal impact factor (IF), first conceived in 1955 by Eugene Garfield, the founder of the Institute for Scientific Information (ISI), has been extensively used in the past decades as an index of quality of scientific journals (3) and is based on citation analysis (4).
Although the journal impact factor has been widely regarded as the best instrument for the evaluation of the quality of scientific journals, it has not been spared from criticism(58). Main points of consideration regarding methodological aspects in the calculation of this index include the lack of assessment of the quality of citations (9), the inclusion of self-citations (1012), the poor comparability between different scientific fields (13), and the analysis of mainly English-language publications (1416).
In fact, many researchers have proposed different approaches in the evaluation of the quality of scientific journals. The common point in most of these approaches is the assessment of the quality of citations received by a journal (1720). The quality of citations can be estimated analyzing the networks of scientific papers with sophisticated mathematical algorithms (21). The PageRank algorithm, used in the evaluation of webpages by the popular Google search engines, has been proposed as an appropriate model for the evaluation of the quality of citations in scientific journals (22, 23). In fact, a group of researchers at the University of Washington developed a similar algorithm for the evaluation of the influence of scientific journals included in the Thompson Scientific Journal Citation Reports (JCR) dataset (24). Furthermore, another research group from Spanish Universities developed an indicator, named SCImago Journal Rank (SJR) indicator, for the assessment of the quality of scientific journals, applying the PageRank algorithm on the Scopus database (25). In this evolving context, we sought to identify and evaluate the main characteristics and differences between the widely used journal IF by ISI and the newly introduced SJR indicator.
MATERIALS AND METHODS
We searched (January 2008) in the official relevant Web sites for information regarding the main characteristics of the journal IF, provided by JCR through ISI Web of Science, Thomson Scientific, and the SJR indicator, provided by the SCImago journal and country rank Web site, and developed by the SCImago research group. We also searched for information pertinent to the characteristics of these two indices of quality of scientific journals in the official Web sites hosting the databases used by each one of the indices (ISI Web of Science and Scopus for journal IF and SJR indicator, respectively).
In addition, we listed the journals with the top 100 journal IFs and retrieved information regarding their ranking in the SJR indicator list by matching their international standard serial number (ISSN). We also listed the journals with the top 100 SJR indicators and found their ranking in the list of journal IFs. Finally, we calculated the median value as well as the first and third quartile values of the absolute change in ranking order of the journals in the two top 100 lists (with the use of each one of the compared indices vs. the other).
RESULTS
The main characteristics of the evaluation of scientific journals in JCR and in SCImago journal rank are summarized in Table 1. The journal IF of a specific journal for a specific calendar year is defined by the total number of citations (references) by articles published during the specific year in “source” journals (potentially including the respective journal) to any article of the specific journal that was published during the preceding 2 yr, divided by the total number of “citable” articles published in the respective journal during the preceding 2 yr. Articles regarded as citable are mainly original research and review articles.
Table 1. Main characteristics of the evaluation of scientific journals by journal citation reports and SCImago journal and country rank.
CharacteristicJournal citation reportsImago journal and country rank
OrganizationThomson ScientificSCImago research group
Source databaseInstitute for Scientific Information (ISI) Web of ScienceScopus (Elsevier B.V.)
Number of journals<7934a13,208
Languages of publication of journals3050
Countries of publication of journals7197
Countries of research originNot available229
UpdateWeeklyDaily
Main indicator of quality of journalsJournal impact factor (IF)SCImago journal rank (SJR) indicator
Reference period1 calendar year3 calendar years
Citation window2 preceding years3 past years
Journals providing citations“Source” journals (“cited-only” journals excluded)All other journals
Weight of citationsEqualDepending on the “prestige” of the citing journal
Journal self-citationsIncludedNot included
Articles considered to receive citations“Citable” (research and review articles)All types
Subject classification2 Editions (science and social sciences), 224 subject categories27 subject areas, 295 subject categories
AccessRestricted (paid subscription required after 1 month free use)Open
Secondary indices, utilitiesJournal immediacy index, journal cited half-life, unified impact factor, 5-yr impact factor, self-cites, graphical representationsH Index, self-cites, country indicators, graphical representations
a 6166 science journals, 1768 social science journals.
The SJR indicator of a specific journal for a three-calendar-year period is calculated through an iteration process that computes the “prestige” gained by the journal through the transfer of prestige from all the other journals included in the network of journals, by their citations during the past 3 yr to all articles of the specific journal published in the past 3 yr, divided by the total number of articles of the specific journal during the 3 yr period in regard. The amount of prestige of each journal transferred to another journal in the network is computed by considering the percentage of citations of the former journal that are directed to articles of the latter journal (26).
Table 2 presents the first 20 ranked journals of all categories with each one of the compared indices and their corresponding rank using the other index. Of the 20 journals with the highest journal IFs, 13 retain a position in the top 20 journals with the use of the SJR indicator, and vice versa. Regarding the top 100 journals in the 2006 journal IF ranking order, the median absolute change in their ranking position with the use of the SJR indicator is 32 (1st quartile: 12; 3rd quartile: 75). Conversely, regarding the top 100 journals with the current SJR indicator, the median absolute change in their ranking position with the use of the journal IF is 29 (1st quartile: 10.5; 3rd quartile: 65.5).
Table 2. Comparative rankings of the top 20 journals by journal impact factor and SCImago journal rank indicator
Journal impact factorSCImago journal rank indicator
RankValueJournal titleValueRank
163,342Ca-A Cancer Journal of Clinicians7,27519
251,296New England Journal of Medicine3,64951
347,237Annual Review of Immunology22,4391
436,525Annual Review of Biochemistry16,1002
533,508Reviews of Modern Physics2,68979
631,583Nature Reviews Cancer9,1599
731,441Physiological Reviews7,86616
831,354Nature Reviews Molecular Cell Biology12,2406
930,028Science5,33830
1029,194Cell15,2243
1128,697Nature Reviews Immunology11,1017
1228,588Nature Medicine7,22620
1328,533Annual Review of Neuroscience8,67811
1427,596Nature Immunology12,4845
1526,681Nature6,20323
1626,576Annual Review of Cell and Developmental Biology14,1934
1726,054Chemical Reviews2,24593
1825,800Lancet1,652134
1924,370Briefings in Bioinformatics2,53584
2024,176Nature Genetics9,08310
347,237Annual Review of Immunology22,4391
436,525Annual Review of Biochemistry16,1002
1029,194Cell15,2243
1626,576Annual Review of Cell and Developmental Biology14,1934
1427,596Nature Immunology12,4845
831,354Nature Reviews Molecular Cell Biology12,2406
1128,697Nature Reviews Immunology11,1017
3318,306Immunity9,3378
631,583Nature Reviews Cancer9,1599
2024,176Nature Genetics9,08310
1328,533Annual Review of Neuroscience8,67811
3119,098Annual Review of Genetics8,58312
2124,077Cancer Cell8,21413
5614,033Molecular Cell8,18514
4515,050Genes and Development8,08615
731,441Physiological Reviews7,86616
5314,299Current Opinion in Cell Biology7,39917
3218,485Nature Cell Biology7,36718
163,342Ca-A Cancer Journal of Clinicians7,27519
1228,588Nature Medicine7,22620
The journals with the greatest relative increase in their ranking order in the SJR indicator compared to the journal IF were the journals Immunity, Molecular Cell, Annual Review of Cell and Developmental Biology, Cell, and Current Opinion in Cell Biology. The journals with the greatest relative stability in their ranking order from the journal IF to the SJR classification were Annual Review of Physiology, Annual Review of Biophysics and Biomolecular Structure, PLoS Biology, An-
nual Review of Plant Biology,andBlood. The journals that exhibited the greatest relative decrease in their ranking order in the SJR indicator compared to the journal IF were Annual Review of Astronomy and Astrophysics, JAMA-Journal of the American Medical Association, Nature Physics, Advances in Catalysis, and Behavioral and Brain Sciences. It should be mentioned that all five of the latter journals exhibited a fall of more than 4000 positions in their ranking order with the use of the SJR indicator compared to the journal IF. Exploring the considerable discrepancy in the rankings of the latter five journals, we noted as potential causes a computational error, the presence of another journal with the same title but different ISSN, lack of source data, and a divergence between the number of citable articles and of total articles or documents. Furthermore, one of the top 100 journals according to the journal IF classification (Trends in Ecology & Evolution) was not included in the SJR database.
DISCUSSION
The main differences between the journal IF and the SJR indicator derive mainly from differences in the scientific databases used as the sources of citations, as well as from differences in the methodology of estimation of these indices. The latter primarily regard the weight attributed to citations, the way of handling self-citations, the temporal window analyzed, and the type and number of the articles of a journal considered in the denominator of calculation ofthe aforementioned indices. In terms of utility, the main novelty introduced by the SJR indicator is open access.
Regarding the differences in the scientific databases, on which the compared indices of evaluation of scientific journals are applied, Scopus includes a substantially larger collection of journals, originating from remarkably more countries and published in a greater variety of languages (27). Thus, in this regard it can be assumed that SJR may provide a more comprehensive estimation of the scientific value of journals, particularly so for those published in non-English languages (16). This is why the latter category of journals receives a great percentage of the overall incoming citations from non-English journals (14, 15). The latter are rather underrepresented in the Web of Science database. Furthermore, the Web of Science takes into consideration citations originating from a subset of source journals (28), potentially excluding some journals published in non-English languages, a factor that may influence the evaluation of this category of journals (14). Yet, it should be mentioned that although Scopus includes a larger collection of non-English journals, the latter are still underrepresented, constituting ~15% of the total number of included journals (29). Moreover, the apparent advantage of Scopus in citation analysis originating from the breadth of its database is limited to the time period after 1996 for which citation analysis is available (30).
Regarding the methodology of the calculation of the two compared indices, the most significant difference lies in the fact that the SJR indicator takes into account not only the absolute number but also the “quality” of citations received by a journal, whereas the journal IF considers incoming citations only in a quantitative manner. It is plausible that the articles of a journal have a greater impact in science if they are cited by journals of higher scientific quality. Such an analysis may not have been feasible in the past, though in today's electronic era, powerful computational systems provide the opportunity to apply sophisticated algorithms for the evaluation of interactions between journals in a huge network or universe of scientific publications. However, the simple and comprehensible methodology used in the calculation of the journal IF, despite the controversy raised over the years for the output data (31, 32), has been regarded as one ofthe most favorable attributes of this long-used standard of reference in the field of analysis of scientific citations.
It should be noted though that the simple methodology used in the calculation of the journal IF has allowed editors to use various practices aiming to increase the impact factor of their journals (5, 33, 34). The principal ones are probably the promotion of self-citations (1012, 35), the predilection for review articles (36), and the decrease in the total number of included articles (37). Notably, the use of the SJR indicator allows for the estimation of a journal's impact without the influence of self-citations, since prestige can be transferred to a journal by all other journals but not by itself. Instead, in JCR providing the journal IFs, self-citation analysis for each journal can be separately performed. However, this factor is not incorporated in the calculation of the journal IFs. Regarding the weight of the different types of articles in the process of calculating the two compared indices of scientific journals, no provision has been made for differentiating between the value assigned to original research articles compared to review articles in any of the indices.
One of the major shortcomings of the SJR indicator may be the fact that it divides the prestige gained by a journal, through the citations of its articles, to the total number of articles included, rather than to the number of citable articles, as is used in the calculation of the journal IF. Although the strategy used in the calculation of the SJR indicator may be mathematically valid, since in theory any published article can be cited and all citations are taken into account in the numerator of the fraction, in practice, article types such as correspondence articles, letters to the editor, commentaries, perspectives, news, obituaries, editorials, interviews, and tributes are unlikely to receive a significant number of citations (3). In this regard, the scientific quality of journals that contain a large number of the latter types of articles, which may otherwise be of interest to the reader, are expected to be appreciably underestimated with the SJR indicator. Yet, it should be mentioned that neither is the journal IF an optimal index of quality of scientific journals regarding this issue, since it does not adjust for the fact that a great number of total citations addressed to a journal are received by a relatively short portion of the included articles (3, 38, 39).
The effect of the differences in the time periods used for the assessment of the two herein compared indices of quality of scientific journals on the accuracy of the estimates cannot be directly inferred in the context of the present study. It should be noted though that both of the indices refer to the past three calendar-year periods, although through a different approach. This temporal window may be relatively short, particularly for journals with an appreciable interval between receipt of an article and publication (3, 40).
An indisputable advantage of journal IF over any new index of evaluation of quality of scientific journals is tradition. On the one hand, authors compete for publishing the products of their research in highly ranked journals (41). On the other hand, journal editors elect to publish scientific articles that are expected to be highly cited (37). Thus, it can be assumed that journal IF rankings would have influenced accordingly the quality of the journals over the years in which they have been considered as the sole standard of reference (3).
Although our study did not aim to systematically assess the comparability in journal rankings between the novel SJR indicator and the well-established journal IF index, we observed that the introduction of the SJR indicator does not bring about radical changes in this regard. In fact, half of the journals in the top 100 journal IF list are placed within a reasonable range of 32 ranking places in the SJR indicator journal list. Some striking inconsistencies observed in the rankings of the new SJR indicator, compared to the journal impact factor, should be attributed to the “precocity” of the new indicator rather than to systematic differences in the methodologies used. In fact, the two methodologies have been systematically compared in previous studies. A weighted PageRank algorithm applied in the same dataset used for the derivation of the journal impact factor provided results that correlated significantly with the later index (23). However, this correlation was moderately strong and differed between various scientific fields, for instance, for medical journals (9, 23).
The SCImago journal rank indicator is a novel instrument for the evaluation of scientific journals that may challenge the established premiership of the journal IF in ranking scientific journals. It provides unrestricted (open) access, is based on a larger source journal database, and focuses on the quality of citations that a journal receives by other journals, rather than the absolute number. However, the sophisticated methodology used in the calculation of the SJR indicator needs to be adequately validated, and certain characteristics may need to be reconsidered before definitive conclusions for its applicability could be drawn. It appears, though, that the election of one index or the other would be mostly a matter of whether the popularity or the quality of a journal is considered as the primary criterion for the evaluation of scientific journals.
The authors have no conflicts of interest to declare. No funding was received for this study.
REFERENCES
Citing Literature
Download PDF
Journal
Articles
Actions
AboutINFORMATION
Metrics
Citations: 2
Details
© FASEB
Keywords
bibliometric analysis
quality of publications
bibliographic databases
mathematical computing
scientometrics
Publication History
Issue Online:
11 April 2008
Version of Record online:
11 April 2008
© 2021 Federation of American Societies for Experimental Biology (FASEB)
About Wiley Online Library
Privacy Policy
Terms of Use
Cookies
Accessibility
Help & Support
Contact Us
Training and Support
DMCA & Reporting Piracy
Opportunities
Subscription Agents
Advertisers & Corporate Partners
Connect with Wiley
The Wiley Network
Wiley Press Room
Copyright © 1999-2021 John Wiley & Sons, Inc. All rights reserved
ReadSubmit a manuscriptAuthor GuidelinesEditorial Board Read (Open access) Submit a manuscriptAuthor GuidelinesEditorial BoardVisit FASEB
ReferencesRelatedInformationPDFABSTRACT MATERIALS AND METHODS RESULTS DISCUSSIONREFERENCESCiting LiteraturePDFRequest permissionExport citationAdd to favoritesTrack citationTerms and Conditions of UseLearn more.