Abstract
Objectives. Acupuncture has increasingly been used for insomnia relief after stroke. We aimed to evaluate the methodological quality and summarize the evidence regarding the effectiveness of acupuncture for poststroke insomnia (PSI) from systematic reviews/meta-analyses (SRs/MAs). Methods. Eight databases were searched from inception through August 23, 2020. SRs/MAs on acupuncture treatment for PSI were included. Methodological quality assessment was performed using Assessing the Methodological Quality of Systematic Reviews 2 (AMSTAR-2), and evidence quality assessment was performed using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE). Results. Six SRs/MAs on acupuncture treatment for PSI were included. The AMSTAR-2 showed that the methodological quality of all included SRs/MAs was rated as critically low. According to the evaluation results of GRADE, 38.9% (7/18) of outcomes were rated as very low-quality evidence, 22.2% (4/18) were low-quality evidence, and 8.9% (7/18) were moderate-quality evidence. Descriptive analysis results showed that acupuncture was an effective treatment modality for PSI. Conclusions. All included reviews indicated that acupuncture was more effective than the control group for the treatment of PSI, but the credibility of the results is limited owing to the generally low methodological and evidence quality of the included SRs/MAs. More high-quality evidence is needed to determine whether acupuncture is more effective than other treatments.
1. Introduction
Worldwide, stroke is the second leading cause of death and the third-largest source of disability [1]. Insomnia is one of the most common poststroke complications, affecting approximately 40% to 60% of stroke patients [2]. Although a large number of chronic poststroke insomnia (PSI) patients are produced, insomnia continues to be the most unrecognized modifiable risk factor for stroke [3]. Untreated PSI causes cognitive dysfunction, altered mood, and fatigue and may impede stroke rehabilitation, lengthen hospital stay, influence stroke outcomes, and increase the possibility of stroke recurrence [4]. The primary pharmacological treatments for PSI are benzodiazepines, nonbenzodiazepine sedatives, and melatonin agonists. Although effective pharmacological treatments are available, their significant side effects have limited their clinical applications and long-term use [5]. The mainstay of nonpharmacological interventions for insomnia is cognitive behavioral therapy. However, cognitive behavioral therapy often results in an acute reduction in sleep time during the first few weeks of treatment, which means that sleep quality improvement with cognitive behavioral therapy requires long-term implementation [6]. Therefore, faced with the limitations of currently available PSI treatments, many people are interested in complementary and alternative medicine.
Among the complementary treatment modalities, acupuncture is one of the safe and most popular treatments [7]. According to the theory of traditional Chinese medicine (TCM), acupuncture provides overall coordination to help the body achieve a state of relative equilibrium of yin and yang, restoring its physiological function and regulating the sleep-wake cycle [8]. Systematic reviews (SRs)/meta-analyses (MAs) are considered the gold standard for assessing the effects of health care interventions, but their methodology must comply strictly with a series of guidelines, so as to minimize the possibility of bias in answering a specific research question [9]. Furthermore, the generation of assessment tools such as the Assessing the Methodological Quality of Systematic Reviews 2 (AMSTAR-2) [10] and the Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) [11] have also played prominent roles in designing, reporting, and assessing high-quality SRs/MAs. However, not all authors of SRs/MAs have maintained strict adherence to the criteria, which has led to low-quality reviews and difficulty in providing convincing conclusions [12]. A literature search yielded several published systematic reviews (SRs)/meta-analyses (MAs) that have reported that acupuncture is efficacious for PSI, but their methodological and evidence quality have not been evaluated. Therefore, we composed an overview to summarize the evidence on the effectiveness of acupuncture for PSI.
2. Materials and Methods
2.1. Inclusion and Exclusion Criteria
The inclusion criteria were as follows: (a) study design: SRs/MAs based on randomized controlled trials (RCTs); (b) participants: the participants had PSI diagnosed according to any authoritative diagnostic criteria, no restrictions on sex, age, race, onset time, or the source of cases; (c) intervention: acupuncture therapy (manual acupuncture, electroacupuncture, or auricular acupuncture, etc.) versus conventional medication (CM) or acupuncture therapy combined with CM versus CM alone; and (d) outcomes: effective rate, sleep parameters, sleep efficiency, scales, or index for sleep quality evaluation, adverse effects.
The exclusion criteria were as follows: (a) animal studies; (b) network MAs, overviews, and narrative reviews; and (c) studies in which the required data were unavailable.
2.2. Search Strategy
Literature searches were conducted in PubMed, EMBASE, the Cochrane Library, Web of Science, China National Knowledge Infrastructure (CNKI), Wanfang Database, Chongqing VIP, and SinoMed from their inceptions to March 28, 2020, and we conducted an updated search on August 23, 2020, to provide more up-to-date and comprehensive evidence. Details of the search strategy for each database are presented in Additional file 1. There were no restrictions on language, publication status (including gray literature), or publication country.
2.3. Eligibility Assessment and Data Extraction
Two reviewers (SM-L and M-S) independently screened the titles and abstracts of the retrieved citations, evaluated the potential full texts, and determined the eligibility of the reviews. Disagreements were resolved by consensus or arbitration by a third reviewer (Y-H).
Two independent reviewers (SM-L and M-S) extracted the following data: the first author, publication year, country, numbers of included trials and participants, quality assessment methods, interventions, outcomes, and the main results of each included review.
2.4. Methodological Quality Assessment
The Assessing the Methodological Quality of Systematic Reviews 2 (AMSTAR-2) tool [10] was used by two independent reviewers (XH-Q and M-S) to assess the methodological quality of the included SRs/MAs. The AMSTAR-2 tool contains 16 items, and seven of them are critical items. Details of the 16 items are presented in Additional file 2. The quality of each item (especially the critical items) should be fully considered and categorized into four levels, namely, high, moderate, low, and critically low. Disagreements were resolved by consensus or arbitration by a third reviewer (Y-H).
2.5. GRADE Scoring
The Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) [11] system based on the GRADEpro software (http://ims.cochrane.org/revman/gradepro) was used to assess the evidence quality of the outcome measures of the included SRs/MAs. The following criteria were taken into account: the risk of bias, inconsistencies, indirectness, inaccuracy, and publication bias. The evidence quality was evaluated by two reviewers (XH-Q and M-S) independently, and disagreements were resolved by consensus or arbitration by a third reviewer (Y-H).
3. Results
3.1. Literature Selection
The literature search retrieved 94 potentially relevant studies, and after removing the duplicates and performing titles/abstracts screening, we excluded 84 studies. The remaining ten studies were retrieved for full-text assessment. We excluded four studies because two of these did not focus on acupuncture and two did not focus on PSI. Finally, six studies [13–18] were included in this overview. Details of the literature selection can be found in Figure 1.
3.2. Study Characteristics
The descriptive characteristics of the included SRs/MAs are presented in Table 1. The SRs/MAs were published between 2015 and 2020 and included between seven and 35 trials each with sample sizes ranging from 435 to 2866 subjects. All SRs/MAs included only RCTs. These SRs/MAs were mainly published by authors from East Asia (five [14–18] from China and one [13] from Korea); five [14–18] were published in Chinese and one [13] in English; four [13–15, 17] were published in journals, and two [16, 18] were master’s degree papers in China. The intervention measures were mostly acupuncture therapy or acupuncture plus other therapies (e.g., auricular acupressure) in the treatment group and CM, and sham acupuncture in the control group. Five SRs/MAs [13–17] used the Cochrane risk of bias criteria to evaluate the quality of the trials, and the remaining one [18] used the Jadad scale.
3.3. Methodological Evaluation
After evaluation of the methodological quality by the AMSTAR-2 tool, all included SRs/MAs were rated to be of critically low quality. The critical items that were most frequently lacking were item 2 (lack of the protocol being registered before review commencement (6/6, 100%)), item 4 (lack of a search strategy for each database (2/6, 33.3%)), item 7 (lack of a list of excluded trails (2/6, 33.3%)), and item 15 (lack of assessment of the present and likely impact of publication bias (1/6, 16.7%)). The breakdown of the AMSTAR-2 assessment for each review is presented in Table 2.
3.4. Evidence Quality Evaluation
After evaluation of the evidence quality of the 18 outcome measures by the GRADE system, 38.9% (7/18), 22.2% (4/18), and 38.9% (7/18) were rated to be of moderate, low, and critically low quality, respectively. The key factors affecting the evidence quality included the study limitations within the original trials, inconsistency, imprecision, and the possibility of publication bias. The details of the GRADE evaluation are shown in Table 3.
3.5. Outcomes and Efficacy Evaluation
3.5.1. Effective Rate
Five reviews [14–18] analyzed the effective rate of acupuncture for PSI. In four reviews [14–16, 18], acupuncture appeared to be more effective than CM for the treatment of PSI. Additionally, according to the results of other reviews [14, 17, 18], acupuncture plus other therapy was more effective than CM alone. In one review that used the cure rate to compare the effects of acupuncture with those of CM, acupuncture was observed to have a significant difference in reducing PSI.
3.5.2. PSQI Score
Two reviews [13, 15] used the Pittsburgh Sleep Quality Index (PSQI) score to compare the effects of acupuncture with those of CM. It was found that acupuncture was more effective than CM for the treatment of PSI. In one review [17] that used the PSQI score to compare the effects of acupuncture plus auricular acupressure with those of CM, the results indicated that the combined treatment was superior to CM alone in terms of improvement in daytime dysfunction, but there was no statistical difference between different treatments in terms of improvement in sleep onset latency and sleep time.
3.5.3. ISI and AIS Score
One review [13] comparing the effects of intradermal acupuncture with those of sham acupuncture used the Athens Insomnia Scale (ISI)and the Insomnia Severity Index (AIS). In this review, acupuncture had a significant effect on PSI relative to sham, as assessed by both by the ISI and the AIS.
3.5.4. TCM Syndrome Score
Two reviews [13, 15] comparing the effects of acupuncture with those of CM used the TCM syndrome score. Of these reviews, the results of Lee and Lim [13] showed that acupuncture had a better effect than CM, while the other review [15] reported there was no statistical difference between the different treatments.
3.5.5. Depression and Anxiety
One review [13] comparing the effects of acupuncture with those of CM used the Self-Rating Anxiety Scale (SAS) or the Self-Rating Depression Scale (SDS). According to the results, no significant differences in scores for depression and anxiety were found between the different treatments.
4. Discussion
Insomnia is a common complication following stroke, often interfering with activity, recovery, and rehabilitation. As a widely used therapy in China for poststroke rehabilitation, the effect of acupuncture in improving sleep quality in stroke patients has attracted the attention of researchers. Currently, there have been multiple SRs/MAs of relevant clinical trials to explore the effect of acupuncture for PSI. Based on a comprehensive research method, we conducted a comprehensive assessment of SRs/MAs with varying quality to further explore the reliability of the results, in order to provide a basis for future related research by clinical and scientific researchers.
4.1. Summary of Quality
The results of the AMSTAR-2 evaluation show that each SR/MA had different deficiencies in terms of its design, registration, data extraction, and analysis. None of the SRs/MAs had registered a preliminary design protocol, which may have resulted in a larger adjustment of the study process than expected, increasing the risk of bias and affecting the rigor of the SR/MA. For an SR/MA, a protocol should be designed and registered in advance to ensure that the study implementation process is methodical. Although all of the reviews had conducted literature searches in more than one database, two [14, 17] only provided search keywords but no specific search strategy, which may result in publication bias and affect the conclusion’s reliability. A full and comprehensive literature search is quite critical for secondary literature studies like SRs/MAs. None of the SRs/MAs provided a complete list of potential studies with reasons for the exclusion of each, which may affect the reliability of the results. Although the majority of the published SRs/MAs were journal articles with word count and format restrictions, providing a list of excluded articles could more strongly prove the rigor of the literature screening process. Two [16, 18] SRs/MAs did not report funding resources or state their conflicts of interest, which may increase study reporting bias because the results of business-funded studies might be biased towards the funder. One review [17] did not consider publication bias and provided explanations for its possible sources when the authors interpreted or discussed the study results, which may affect the authenticity of the final results. The deficiencies mentioned above are the main reasons for the critically low methodological quality rating of the included SRs/MAs.
The results of the GRADE evaluation indicated that the current quality of evidence was not satisfying. Of these 18 outcome measures, seven, four, and seven were rated to be of moderate, low, and critically low quality, respectively. The key factors affecting the evidence quality included limitations, inconsistency, imprecision, and the possibility of publication bias. The study limitations within the original trials were the main factors in the low quality of all of the evidence. Most of the RCTs included in the SRs/MAs had an unclear risk of bias for blinding, random sequence generation, and allocation concealment. Although blinding of the therapists who perform acupuncture would be difficult, blinding of patients, other care providers, and outcome assessors should be attempted in order to minimize the performance and assessment bias. Reduced evidence quality due to inconsistencies may originate from substantial clinical and methodological variations in the included RCTs, which can be avoided by standardizing the inclusion and exclusion criteria and standardizing the literature screening process. Additionally, imprecision and publication bias also lead to downgrading of the quality of evidence, which may be related to implausible study designs. These deficiencies in the evidence quality indicate that the conclusions of the SRs/MAs may differ from the true results and thus cannot provide a scientific basis for clinicians.
Assessment of various aspects of the quality of the included SRs/MAs identified common areas for improvement, and thus, the practice of using the widely accepted tools like AMSTAR-2 and GRADE for designing, reporting, and assessing SRs/MAs needs to be encouraged and advocated, thereby providing more convincing evidence based on their findings. The core reason for the low quality of the SRs/MAs may be that a significant proportion of acupuncture SRs/MAs researchers may not receive sufficient evidence-based medicine education [19]. Therefore, we should strengthen and improve evidence-based medicine education in university and medical school and continuing education through TCM, especially regarding the use of the Cochrane Handbook and some widely accepted tools like AMSTAR-2 and GRADE. Furthermore, AMSTAR-2 and GRADE were introduced in 2004 and 2017, respectively. It is clear that they both existed prior to the publication of the included SRs/MAs. This phenomenon suggests a lack of normative awareness among the authors of SRs/MAs, as most weaknesses can be avoided or reduced if authors of SR/MAs could increase their awareness of standard methodology and learn how to use the AMSTAR-2 and GRADE tools before the commencement of SRs/MAs.
4.2. Limitations
This study is the first overview of SRs/MAs using AMSTAR-2 and GRADE to explore the evidence about using acupuncture for PSI. Based on the current results, the methodological and evidence quality of the included SRs/MAs were not satisfying, so our study cannot draw firm conclusions about the use of acupuncture for PSI. Additional high-quality randomized trials and MAs are necessary to determine whether acupuncture is more effective than other treatments.
5. Conclusion
All included SRs/MAs suggested that acupuncture was more effective than CM or sham acupuncture in the treatment of PSI; however, this conclusion was limited due to the generally low methodological and evidence quality of the included studies. The practice of using widely accepted tools like AMSTAR-2 and GRADE for designing, reporting, and assessing SRs/MAs needs to be encouraged and advocated, thereby providing more convincing evidence based on their findings.
Abbreviations
| PSI: | Poststroke insomnia |
| SR: | Systematic review |
| MA: | Meta-analysis |
| AMSTAR-2: | Assessing the Methodological Quality of Systematic Reviews 2 |
| GRADE: | Grading of Recommendations, Assessment, Development, and Evaluation |
| RCTs: | Randomized clinical trials |
| CNKI: | China National Knowledge Infrastructure |
| CM: | Conventional medication |
| TCM: | Traditional Chinese medicine |
| PSQI: | Pittsburgh Sleep Quality Index |
| ASI: | Athens Insomnia Scale |
| ISI: | Insomnia Severity Index |
| SAS: | Self-Rating Anxiety Scale |
| SDS: | Self-Rating Depression Scale. |
Data Availability
All data generated or analyzed during this study are included within the article.
Disclosure
Jinke Huang and Manli Wu are the co-first authors.
Conflicts of Interest
The authors declare that there are no conflicts of interest.
Authors’ Contributions
Jinke Huang and Manli Wu planned and designed the study, and they contributed equally. Simin Liang, Min Shen, and Xiaohui Qin screened potential studies and extracted data from the included studies. Min Shen and Xiaohui Qin assessed the reviews, Yong Huang provided guidance on the overview methodology, and Jinke Huang drafted the manuscript. All authors read, critically reviewed, and approved the final manuscript as submitted.
Acknowledgments
The authors thank all investigators and supporters involved in this study.