The Correlation between Altmetric Attention Score and Traditional Bibliometrics in Top Nursing Journal Articles

Chen, Lingmin; Yang, Mutong; Li, Nian; He, Ying; Zhang, Yonggang

doi:https://doi.org/10.1155/2023/2789960

Journal of Nursing Management

On this page

Abstract Introduction Methods Results Discussion Data Availability Conflicts of Interest Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2023 | Article ID 2789960 | https://doi.org/10.1155/2023/2789960

The Correlation between Altmetric Attention Score and Traditional Bibliometrics in Top Nursing Journal Articles

Lingmin Chen,¹Mutong Yang,²Nian Li,³Ying He,⁴and Yonggang Zhang^5,6,7

Academic Editor: Fiona Timmins

Received07 Nov 2022

Revised20 Nov 2022

Accepted02 Dec 2022

Published22 Feb 2023

Abstract

Background. Altmetric Attention Score (AAS) is a quantitative measurement of the online impact of research and has a potential correlation with traditional bibliometrics. However, the correlation for nursing journal articles is still unknown. The objective of the study was to analyze the correlation between AAS and traditional bibliometrics in the top nursing journal articles. Materials and Methods. Articles published in top nursing journals (the journals with the top 20 5-year impact factors) from 2010 to 2019 were included. The correlations between AAS and citations, AAS and Relative Citation Ratio (RCR) score, AAS and Category Normalized Citation Impact (CNCI) value, and AAS and impact factors were analyzed. Statistical analyses were performed using Stata 25.0 software. Results. A total of 15,212 journal articles were included in the study. Very weak correlations were found between AASs and citations [0.124 (95% CI, 0.108–0.14)], AASs and RCRs [0.26 (95% CI, 0.244–0.275)], and AASs and CNCIs [0.207 (95% CI, 0.192–0.223)]. The weak correlations were also found between AASs and impact factors in several journals. The weak correlations between AASs and citations, AASs and CNCIs, and AASs and RCRs were also found for most journals based on subgroup analysis. Conclusions. There is very weak correlations between AASs and traditional bibliometrics in top nursing journal articles. More studies should be conducted to assess how AAS influence bibliometrics, and how they can help manage nursing journal articles and research.

1. Introduction

Bibliometrics is the use of statistical methods to analyze publications, especially for scientific contents [1, 2]. It is used to justify the role of the researchers and the research team [3] and also to identify key partners [4] and the progress of a specific research field [5]. The most widely used bibliometrics method is based on citations [6], such as citation times, impact factors [7], H index [8], Eigenfactor score [9], and CiteScore [9]. However, these metrics have limitations. It would take time to accumulate citation times [10], and different journal articles could not be compared directly even in the same research field [11, 12]. Absolute citation times favor older journal articles and have limited utility in comparing journal articles from different fields [12, 13]. Thus, new metrics have been developed. RCR [14], developed by NIH, is defined as the total number of citations that a paper received per year divided by the average field-specific citation rate for a peer companion group [14, 15]. It allows the comparison of articles in different fields. CNCI is another metrics to assess the paper's impact and has been analyzed in different studies [16, 17], which can also be used to assess scientific articles from different research fields [16, 18].

In recent years, with the development of digital technology and the use of online platforms to discuss research, alternative-level metrics (altmetrics) have been introduced in research fields [12–14]. They were used to measure the journal articles with attention, dissemination, overall influence, and impacts [14]. The main altmetrics platform is Altmetric [19, 20], which compiles the number of mentions of a paper across the most used social media platforms such as Twitter, Facebook, and LinkedIn, and public policy documents, mainstream media, online reference managers, and other online platforms, to generate a weighted score, Altmetric Attention Score (AAS) [14]. AAS is a dynamic bibliometric which captures the online impact of a paper. It has been used to assess the online impact of journal articles across different fields [7, 14, 21] and provide potential evidence for research impact or journal strategy.

Nursing is an important discipline, and research helps the progress of the discipline by improving nursing practice and finally helping nursing management. Factors that influence the characteristics of nursing journal articles could further help journal articles to be quickly transformed into nursing practice and applied to nursing management. It has been reported that bibliometric analysis might help nursing management and nursing practice, and thus some studies have been performed in the field of nursing management [2, 16]. However, these studies did not analyze the important impact of online platforms on nursing management. It is still unclear whether the online impact would be associated with traditional bibliometrics and whether the online impact would help nursing paper citations. Furthermore, it is also unclear whether the nursing manager could use social media, public research platforms, or online platfroms to help nursing research impact. Although several articles have used bibliometrics methods to assess the top cited [2, 16] or the top impact nursing articles [2, 16], whether impact from social media, public research platforms, or online platforms will help higher citations is a mystery. Therefore, in the current study, we analyzed the Altmetric and bibliometric data of journal articles published in nursing journals, to assess the correlation between AASs and the citation metrics.

2. Methods

2.1. Inclusion and Exclusion Criteria for the Selection of Nursing Journal Articles

The inclusion criteria were as follows: (1) the journal articles should be published in top nursing journals. Five-year impact factors were used to identify the top nursing journals, and thus the top 20 5-year impact factor journals in the nursing category in the JCR report in 2021 were used in the current study. (2) The journals should be indexed by PubMed. Thus, the journals included in the study were as follows: Australian Critical Care, Birth-Issues in Perinatal Care, BMC Nursing, European Journal of Cardiovascular Nursing, Intensive And Critical Care Nursing, International Journal of Mental Health Nursing, International Journal of Nursing Studies, International Nursing Review, Journal of Advanced Nursing, Journal of Clinical Nursing, Journal of Nursing Management, Journal of Nursing Scholarship, Nurse Education Today, Nursing Ethics, Nursing Outlook, Research in Nursing & Health, Seminars in Oncology Nursing, Women and Birth, Worldviews on Evidence-Based Nursing, and Asian Nursing Research. (3) The paper type should be an article or review. (4) The paper could be found in Incites, iCite, Altmetric, and PubMed. The following exclusion criteria were used: (1) duplicated journal articles; (2) mismatched journal articles; (3) editorial articles, corrections, and letters.

2.2. Paper Searching

The journal articles were searched by the selected journal names in the three databases (Incites [16], iCite [14], and Altmetric [12]). All search results were downloaded. If any data for an article was missing, the article was further searched in PubMed or the specific journal websites.

2.3. Paper Screening and Data Extraction

All articles were screened according to the inclusion and exclusion criteria. Only articles which had all the required data (RCR, citation time, AAS, and CNCI) were included. The following information was extracted from each article: journal, author, impact factor, RCR score, citation time, CNCI value, AAS, etc. The RCR scores were extracted from the downloaded file from the iCite database, the CNCI values were extracted from the downloaded file from the Incites database, and the AASs were extracted from the downloaded file from the Altmetric database. The journals' impact facotrs were downloaded from the Web of Science.

2.4. Statistical Analysis

The characteristics of the included studies were analyzed. The correlations between AASs and citations, AASs and RCRs, AASs and CNCIs, and AASs and impact factors, were analyzed using Spearman correlation coefficients [12]. Subgroup analyses were performed according to journals, and years. All analyses were performed by SPSS 25.0 software.

3. Results

3.1. The Main Characteristics of Included Journal Articles

The data were collected by September 9^th, 2022. After screening all data of the journal articles according to the inclusion and exclusion criteria, a total of 15,212 journal articles were included. The impact factors for all the selected 20 journals from 2010 to 2019 are shown in Table 1, and the impact factors of most journals increased from 2010 to 2019. The average citation of all journal articles was 17.36 times, and the median citation was 11 times. A total of 318 journal articles were cited 0 times, 14,059 journal articles were cited 1 to 50 times, 660 were cited 51 to 100 times, and 175 journal articles were cited more than 100 times.

3.2. The Performance of Included Journal Articles

The average RCR score of included articles was 1.71, and the median RCR score was 1.14. The RCR scores of 258 articles were 0; 6,576 articles had RCR scores between 0 and 1; 8,236 journal articles had RCR scores between 1 and 10; the RCR scores of 142 journal articles were higher than 10. The average CNCI value was 1.52, and the median CNCI value was 1.05. The CNCI values of 318 journal articles were 0; 7,017 journal articles had CNCI values between 0 and 1; 7,785 journal articles had CNCI values between 1 and 10; the CNCI values of 92 journal articles were higher than 10. The average AAS was 8.92, and the median AAS was 3. The AASs of 2264 journal articles were 0; 9,950 journal articles had AASs between 0 and 10; 2,866 journal articles had AASs between 10 and 100; the AASs of 132 journal articles were higher than 100. The citations, RCRs, CNCIs, and AASs of all journal articles in the 20 journals were summarized in listed in Table 2.

3.3. The Contribution of Different Sources to AAS

The sources that contributing to the AAS were individually analyzed from 2010 to 2019. For all years, Twitter was the most frequent contributor to AAS followed by news outlets and Policy (Table 3).

3.4. The Correlation of AASs with Citations, RCRs, CNCIs

The correlations of AASs with citations, CNCIs, and RCRs are shown in Figures 1–3. In summary, the correlation coefficient was 0.124 (95% CI, 0.108–0.14) between AASs and citations, 0.26 (95% CI, 0.244–0.275) between AASs and RCRs, and 0.207 (95% CI, 0.192–0.223) between AASs and CNCIs. The correlations for each journal are shown in Table 4.

For the International Journal of Nursing Studies, which ranked first in the JCR report (2021) with an impact factor of 6.612, a total of 1,437 journal articles were included. The total citation times was 43,729 (ranging from 0 to 582 by paper, median 19), the total AAS was 12,975 (ranging from 0 to 703 by paper, median 3), and the total RCR score was 3,985.92 (ranging from 0 to 35.94 by paper, median 1.79). Very weak correlations were found: 0.207 (95% CI, 0.158–0.258) between AASs and citations, 0.371 (95% CI, 0.325–0.418) between AASs and RCRs, and 0.267 (95% CI, 0.219–0.317) between AASs and CNCIs.

The Journal of Clinical Nursing ranked fourth in the JCR report (2021) with an impact factor of 4.423, from which we collected the highest number of journal articles in the current study. The total number of citations was 48,236 (ranging from 0 to 262 by paper, median 10), the total AASs was 17,842 (ranging from 0 to 251 by paper, median 2), and the total RCR score was 4,709.72 (ranging from 0 to 21.32 by paper, median 0.98). Very weak correlations were found: 0.133 (95% CI, 0.097–0.166) between AASs and citations, 0.312 (95% CI, 0.28–0.346) between AASs and RCRs, and 0.23 (95% CI, 0.199–0.264) between AASs and CNCIs.

The Journal of Nursing Management was the only management journal in the field of nursing, from which a total of 783 journal articles were included. The total number of citations was 13,906 (ranging from 0 to 203 by paper, median 11), the total AASs was 3,007 (ranging from 0 to 228 by paper, median 1), and the total RCR score was 1,593.84 (ranging from 0 to 17.58 by paper, median 1.43). Very weak correlations were found: 0.158 (95% CI, 0.086–0.229) between AASs and citations, 0.239 (95% CI, 0.162–0.304) between AASs and RCRs, and 0.206 (95% CI, 0.137–0.274) between AASs and CNCIs.

Nurse Education Today was the only education journal in the field of nursing, from which a total of 1,566 journal articles were analyzed. The total citation was 26,555 (ranging from 0 to 770 by paper, median 11), the total AAS was 15,837 (ranging from 0 to 373, median 3), and the total RCR score was 3,104.38 (ranging from 0 to 81.92, median 1.34). Very weak correlations were found: 0.086 (95% CI, 0.038–0.134) between AASs and citations, 0.269 (95% CI, 0.223–0.317) between AASs and RCRs, and 0.188 (95% CI, 0.139–0.234) between AASs and CNCIs.

3.5. The Correlation Analysis by Years

Spearman rank correlation coefficient was calculated between citations and the AASs by years (Table 5). The results showed very weak correlations between AASs and citations, AASs and CNCIs, and AASs and RCRs in all years.

3.6. The Correlation between AASs and Impact Factors

The correlations between AASs and impact factors were individually analyzed based on years. Weak correlations were found in most of the years (Table 6). Spearman rank correlation coefficient was calculated between the impact factors and AASs based on journals (Table 7). Weak correlations were found for the Australian Critical Care, International Journal of Mental Health Nursing, International Journal of Nursing Studies, Journal of Advanced Nursing, and Nurse Education Today.

4. Discussion

In the current study, the Altmetric and traditional bibliometric of journal articles published in 20 top nursing journals were analyzed. A very weak correlation between AASs and citations was found. AAS was also found to be correlated with RCR and CNCI, which were novel metrics of research influence based on citation times. The results suggest that traditional bibliometrics and AAS cannot be used interchangeably but rather complementarily when assessing the impact of journal articles in the nursing field.

Further analysis by journals and publication years found that the International Journal of Nursing Studies (IJNS) had more correlation between the AASs and citations, and no correlation was found in the Nursing Ethics. This difference might be due to the difference in the breadth of journal topics since one is a comprehensive nursing journal and the other is a subspecificity nursing journal. Articles from IJNS with high online attention might also impact the scientific community and thus increase citations. However, articles from Nursing Ethics, which are likely to be controversial articles for the public to garner public interest, thereby increasing AAS, while they might not impact the scientific community in the same manner [12].

With the COVID-19 pandemic [22], more training programs in the nursing practice field have been performed via the social media platforms and other online platforms. That should be of interest to nursing managers, nurses, and journals. It indicates that the social media and other online platforms would not only help the scientific community but also help the public community, especially during the COVID-19 pandemic period [23]. Implementing new research works on the online platforms will contribute to these results and further help the scientific community.

The study has several strengths. First, the current study was a study with large sample size analyzing alternative metrics and bibliometrics based on articles in top nursing journals. Second, it analyzed the AASs with 3 citation-based metrics and found weak correlations between these parameters. Third, we analyzed the correlation by journals, which was a unique contribution to the nursing field. The paper also has several limitations. First, to analyze the precision correlations, we excluded journal articles that missed any data of the four variables (AASs, citations, RCRs, and CNCIs). Second, the correlations might be different when analyzing at a different time. Third, the study only analyzed the journal articles from top nursing journals, and nursing-associated journal articles published in other journals could have different kinds of correlations, especially for those articles published in low-impact factor journals. Fourth, the study did not include other bibliometrics [24, 25], such as the Eigenfactor score and CiteScore; in the future, a new study should be performed to assess these correlations.

In conclusion, the study of articles published in high-impact nursing journals between 2010 to 2019, finds very weak correlations between AAS and citation-based metrics, which suggests promoting journal articles on online platforms may help research, journals, and nurses. Future studies are needed to assess the long-term correlations among these metrics for nursing journal articles.

5. Implication to Nursing Management

Understanding the correlation of online impact with traditional bibliometrics of research is critical to nursing practitioners, which in turn helps manage nursing journal articles and research. Nursing managers should develop targeted strategies to increase the online impact of research or nursing practice and increase research impact.

Data Availability

The data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest

The authors declare that there are no conflicts of interest.

Acknowledgments

We thank Enliven for improving the grammar and readability. This study was partly supported by the Post Doctor Research Project of West China Hospital of Sichuan University (Grant no. 19HXBH071), the China Postdoctoral Science Foundation (Grant no: 2021M692274), and the Postdoctoral Research Project of Sichuan University (Grant no. 2021SCU12001).

References

É de Andrade Vieira and A. M. Tribuzy de Magalhães Cordeiro, “Bioprospecting and potential of cactus mucilages: a bibliometric review,” Food Chemistry, vol. 401, Article ID 134121, 2023.
View at: Publisher Site | Google Scholar
W. S. Su, G. J. Hwang, and C. Y. Chang, “Bibliometric analysis of core competencies associated nursing management publications,” Journal of Nursing Management, vol. 30, no. 7, pp. 2869–2880, 2022.
View at: Publisher Site | Google Scholar
Y. Huang, X. Wang, Y. Zhang, D. Chiavetta, and A. L. Porter, ““Big data” driven tech mining and ST&I management: an introduction,” Scientometrics, vol. 127, no. 9, pp. 5227–5231, 2022.
View at: Publisher Site | Google Scholar
S. Zhu, Y. Liu, Z. Gu, and Y. Zhao, “Research trends in biomedical applications of two-dimensional nanomaterials over the last decade—a bibliometric analysis,” Advanced Drug Delivery Reviews, vol. 188, Article ID 114420, 2022.
View at: Publisher Site | Google Scholar
R. A. Farahat and M. Elsaid, “Monkeypox and its research trends in Arab countries: a brief bibliometric analysis,” Travel Medicine and Infectious Disease, vol. 49, Article ID 102413, 2022.
View at: Publisher Site | Google Scholar
A. N. Bernstein, P. Filippou, and A. B. Smith, “Publication productivity and bibliometric profiling-a potentially gender-biased approach,” JAMA Surgery, vol. 157, no. 3, pp. 277-278, 2022.
View at: Publisher Site | Google Scholar
R. Ladeiras-Lopes, R. Vidal-Perez, D. Santos-Ferreira et al., “Twitter promotion is associated with higher citation rates of cardiovascular articles: the ESC journals randomized sstudy,” European Heart Journal, vol. 43, no. 19, pp. 1794–1798, 2022.
View at: Publisher Site | Google Scholar
R. K. Saad, M. Al Nsour, Y. Khader, and M. Al Gunaid, “Public health surveillance systems in the eastern mediterranean region: bibliometric analysis of scientific literature,” JMIR Public Health and Surveillance, vol. 7, no. 11, Article ID e32639, 2021.
View at: Publisher Site | Google Scholar
E. Roldan-Valadez, S. Y. Salazar-Ruiz, R. Ibarra-Contreras, and C. Rios, “Current concepts on bibliometrics: a brief review about impact factor, Eigenfactor score, CiteScore, SCImago journal rank, source-normalised impact per paper, H-index, and alternative metrics,” Irish Journal of Medical Science, vol. 188, no. 3, pp. 939–951, 2019.
View at: Publisher Site | Google Scholar
B. D. Haislup, W. R. Rate, M. D. Civilette, A. S. Cohen, B. M. Bodendorfer, and H. P. Gould, “The 100 most impactful articles on the rotator cuff: an altmetric analysis of online media,” Journal of Experimental Orthopaedics, vol. 9, no. 1, p. 92, 2022.
View at: Publisher Site | Google Scholar
I. Ramos-Vielba, N. Robinson-Garcia, and R. Woolley, “A value creation model from science-society interconnections: archetypal analysis combining publications, survey and altmetric data,” PLoS One, vol. 17, no. 6, Article ID e0269004, 2022.
View at: Publisher Site | Google Scholar
C. S. Collins, N. P. Singh, S. Ananthasekar, C. J. Boyd, E. Brabston, and T. W. King, “The correlation between altmetric score and traditional bibliometrics in orthopaedic literature,” Journal of Surgical Research, vol. 268, pp. 705–711, 2021.
View at: Publisher Site | Google Scholar
D. L. Dixon and W. L. Baker, “Long-term association of altmetric attention scores with citations in selected major pharmacy journals,” American Journal of Pharmaceutical Education, vol. 86, no. 2, Article ID ajpe8606, 2022.
View at: Publisher Site | Google Scholar
S. Grover, A. D. Elwood, J. M. Patel, C. V. Ananth, and J. S. Brandt, “Altmetric and bibliometric analysis of obstetrics and gynecology research: influence of public engagement on citation potential,” American Journal of Obstetrics and Gynecology, vol. 227, no. 2, pp. 300.e1–300.e44, 2022.
View at: Publisher Site | Google Scholar
V. Reddy, A. Gupta, M. D. White et al., “Assessment of the NIH-supported relative citation ratio as a measure of research productivity among 1687 academic neurological surgeons,” Journal of Neurosurgery, vol. 134, no. 2, pp. 638–645, 2021.
View at: Publisher Site | Google Scholar
R. Zhu, Y. Wang, R. Wu, X. Meng, S. Han, and Z. Duan, “Trends in high-impact papers in nursing research published from 2008 to 2018: a web of science-based bibliometric analysis,” Journal of Nursing Management, vol. 28, no. 5, pp. 1041–1052, 2020.
View at: Publisher Site | Google Scholar
L. Chen, N. Li, and Y. Zhang, “High-impact papers in the field of anesthesiology: a 10-year cross-sectional study,” Canadian Journal of Anesthesia/Journal canadien d’anesthésie, vol. 23, pp. 1–8, 2022.
View at: Publisher Site | Google Scholar
L. Chen, Y. Yang, J. Fan, Y. Zhang, and N. Li, “Trends of high-impact studies in pharmacology and pharmacy: a cross-sectional study,” Frontiers in Pharmacology, vol. 12, Article ID 726668, 2021.
View at: Publisher Site | Google Scholar
B. F. Kocyigit and A. Akyol, “Bibliometric and altmetric analyses of publication activity in the field of behcet’s disease in 2010–2019,” Journal of Korean Medical Science, vol. 36, no. 32, p. e207, 2021.
View at: Publisher Site | Google Scholar
C. Slovacek, “Examining the correlation between altmetric score and citations in the plastic surgery literature,” Plastic and Reconstructive Surgery, vol. 148, no. 2, pp. 322e–323e, 2021.
View at: Publisher Site | Google Scholar
M. Djulbegovic, K. Kalahasty, A. Watane, S. K. Jabori, H. Al-Khersan, and J. Sridhar, “Correlation between altmetric attention scores and citations for articles published in high-impact factor ophthalmology journals from 2018 to 2019,” JAMA Ophthalmol, vol. 140, no. 6, pp. 623–627, 2022.
View at: Publisher Site | Google Scholar
C. Li, X. Zhu, Y. Zhang, J. Zhang, C. O. Jeon, and B. Jia, “COVID-19 influences both physical and mental health: lessons from bibliometric analysis,” Travel Medicine and Infectious Disease, vol. 49, Article ID 102405, 2022.
View at: Publisher Site | Google Scholar
C. H. Basch, C. E. Basch, G. C. Hillyer, and Z. C. Meleo-Erwin, “Social media, public health, and community mitigation of COVID-19: challenges, risks, and benefits,” Journal of Medical Internet Research, vol. 24, no. 4, Article ID e36804, 2022.
View at: Publisher Site | Google Scholar
M. Villasenor-Almaraz, J. Islas-Serrano, C. Murata, and E. Roldan-Valadez, “Impact factor correlations with scimago journal rank, source normalized impact per paper, eigenfactor score, and the CiteScore in radiology, nuclear medicine & medical imaging journals,” La Radiologia Medica, vol. 124, no. 6, pp. 495–504, 2019.
View at: Publisher Site | Google Scholar
E. Roldan-Valadez, U. Orbe-Arteaga, and C. Rios, “Eigenfactor score and alternative bibliometrics surpass the impact factor in a 2-years ahead annual-citation calculation: a linear mixed design model analysis of radiology, nuclear medicine and medical imaging journals,” La Radiologia Medica, vol. 123, no. 7, pp. 524–534, 2018.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2023 Lingmin Chen et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies