Abstract

Objectives. The aim of this study was to investigate the validity and reliability of the Turkish version of the National Comprehensive Cancer Network/Functional Assessment of Cancer Therapy Ovarian Symptom Index-18 (NFOSI-18) and Functional Assessment of Cancer Therapy/Gynecologic Oncology Group Neurotoxicity 4-item (NTX-4) in patients with advanced ovarian cancer (OC). Methods. Ninety-four women with OC were included. Pearson correlation coefficients were used to examine the convergent validity between the European Quality of Life Survey-5 Dimension-3 Level (EQ-5D-3L) and the Turkish NFOSI-18 and NTX-4. The internal consistencies of the Turkish NFOSI-18 and NTX-4 were calculated using Cronbach’s alpha. Turkish NFOSI-18 and Turkish NTX-4 were readministered to 62 (67.4%) patients with OC after 14–21 days to evaluate test-retest reliability.Results. Turkish NFOSI-18 and Turkish NTX-4 showed excellent internal consistency (Cronbach’s alpha: 0.919 and 0.917, respectively). The test-retest reliability of Turkish NFOSI-18 and Turkish NTX-4 was detected as good to excellent for total score (ICC [95%] = 0.93 [0.88-0.95] and ICC [95%] = 0.90 [0.85-0.94], respectively). Significant correlations were detected between the EQ-5D-3L total score, NFOSI-18 (r = 0.648,  < 0.01), and NTX-4 (r = 0.694,  < 0.01) indicating sufficient convergent validity. Conclusion. The Turkish NFOSI-18 and Turkish NTX-4 are reliable and valid tools to assess disease-related symptoms in patients with advanced OC.

1. Introduction

Ovarian cancer (OC) is the most life-threatening malignancy with a high mortality rate among gynecologic cancers [1]. Only 45.6% of patients with OC survive 5 years [2] and about 70% of patients have a relapse within three years, despite initial treatment yielding significant treatment outcomes [3]. Almost 60% of women with OC are not diagnosed until the disease has progressed to stage III or IV due to a lack of effective screening and the absence of specific symptoms [4]. In fact, the rate of early diagnosis for patients with OC is reported as low as 20% [2]. Patients with OC experience a variety of physical symptoms due to chronic disease processes and side effects of the treatment, including neuropathy, fatigue, alopecia, pain, swelling, anemia, nausea, and vomiting [5]. Moreover, patients with advanced OC may present emotional manifestations such as anxiety and depression due to repetitive cycles of long-duration treatments [6]. Additionally, the primary aim of clinical management of advanced OC is generally based on symptom management, functional preservation, and maintaining or enhancing the quality of life of a patient [7, 8]. Thus, evaluating health-related quality of life (HRQoL) which is a biopsychosocial concept emerges as important as a clinical and research endpoint [9, 10].

The Quality of Life Subcommittee of the Food and Drug Administration (FDA) Oncologic Drugs Advisory Committee has emphasized that HRQoL should be assessed for the clinical benefit of patients in oncology drug trials [11]. Functional Assessment of Cancer Therapy-Ovarian (FACT-O) questionnaire which was developed based on expert opinion is frequently used to evaluate HRQoL in patients with OC [12]. The FACT-O is a reliable and valid instrument for assessing HRQoL in women with OC [13]. European Organization for Research and Treatment of Cancer the Ovarian Cancer Module (EORTC-QLQ-OV28) is also a psychometrically accepted questionnaire designed to assess HRQoL in patients with OC [14]. However, FDA Oncologic Drugs Advisory Committee recommends including items regarding particular symptoms depending on the stage of the disease for measurement tools developed for cancer patients [11]. Besides, recent findings highlight the importance of involving patients during the development of disease-specific patient-reported outcome measure (PROM) as patients’ opinions and the importance given to particular topics by them may differ from oncology experts [15, 16].

The National Comprehensive Cancer Network Functional Assessment of Cancer Therapy Ovarian Symptom Index-18 (NFOSI-18) was developed as a PROM to assess the impact of the symptoms and concerns of advanced patients with OC by gathering opinions from both experts and patients [17]. The original NFOSI-18 demonstrated good internal consistency (Cronbach’s alpha: 0.80) [17] and content validity [18]. However, there are no items in NFOSI-18 evaluating neuropathy even though neurotoxicity was frequently reported by patients with OC [19]. Thus, either the 11-item (NTX-11) or 4-item (NTX-4) Functional Assessment of Cancer Therapy/Gynecologic Oncology Group Neurotoxicity subscales should be used alongside NFOSI-18 to evaluate the HRQoL in a more comprehensive manner by assessing the symptoms of neurotoxicity [20, 21]. The original version of NFOSI-18 and NTX-4 was developed in English, and their validity and reliability were assessed in Korean according to the Functional Assessment of Chronic Illness Therapy (FACIT) organization guidelines [22]. The aim of this study was to investigate the validity and reliability of Turkish forms of NFOSI-18 and NTX-4.

2. Materials and Methods

2.1. Study Design

The present study was carried out as a multicenter, cross-sectional study that consists of three institutions (Dokuz Eylul University Hospital, Izmir Katip Celebi University Atatürk Training and Research Hospital, and Izmir Health Sciences University Tepecik Training and Research Hospital). The ethical approval was obtained from by the Noninvasive Research Ethics Board of Dokuz Eylul University (decision no: 2021/17-15 and date: 0.2.06.2021). The study was performed following the ethical standards of the Declaration of Helsinki.

2.2. Participants

This study was conducted between November 2020 and February 2021. The purpose of the study and the procedure to be carried out were described to 156 patients with advanced OC who were invited to participate in the study. The inclusion criteria were (1) being 18 years or older; (2) diagnosis of stage 2 and greater OC according to International Federation of Gynecology and Obstetrics (FIGO) classification; (3) being able to read and understand Turkish; and (4) currently receiving chemotherapy or radiotherapy. Refusing to participate in the study was the exclusion criterion. Signed informed consent was obtained from all the participants before the study.

2.3. Study Sample

GPower software (version 3.1.9.2, Düsseldorf University, Germany) was used for the calculation of the sample size. The minimum sample size was estimated as 93 patients with a 5% type I error rate, a minimum power of 80%, the hypothesized minimum correlation coefficient of 0.3 [23], and the assumed correlation coefficient of 0.5 (the value was reported in a previous study investigating NFOSI-18 Korean language validation) [22].

2.4. Procedures

The Turkish version of the NFOSI-18 and NTX-4 available at https://FACIT.org was used with the permission from FACIT Group [24, 25]. A detailed medical history including demographic information (i.e., age, height, weight, body mass index (BMI), and disease-related characteristics (OC subtype, OC stage, adjuvant treatment, and disease duration)) of each participant was recorded. Then, the patients filled out the HRQoL questionnaires. All assessments lasted approximately 30–45 minutes. All assessments were conducted by the same assessor to ensure consistency and reduce interassessor variability. A test-retest reliability analysis was performed with 62 (66% of total sample) patients for a second time within a 14 to 21-day period [26].

2.5. Measurement Tools
2.5.1. The National Comprehensive Cancer Network Functional Assessment of Cancer Therapy Ovarian Symptom Index-18 (NFOSI-18)

NFOSI-18 comprises 18 items consisting of four subscales: disease-related symptoms-physical (DRS-P; nine items), disease-related symptoms-emotional (one item), treatment side effects (five items), and general function/well-being (three items) [17]. Each item is assessed using a five-point Likert scale, with 0 “not at all” and 4 “very much.” The individual item scores were summed to calculate the final score or related subscale scores. The total score range between 0 and 72, and higher scores indicate better HRQoL [17].

2.5.2. Functional Assessment of Cancer Therapy/Gynecologic Oncology Group Neurotoxicity 4-Item (NTX-4)

NTX-4 was used for the assessment of peripheral neuropathy symptoms, including sensory, motor, and auditory problems and sensitivity to cold. It consists of four items: numbness or tingling in the feet, numbness or tingling in the hands, discomfort in the feet, and discomfort in the hands [22]. NTX-4 is five-point Likert-type scale, and each item gets a score between 0 “not at all” and 4 “very much.” The total score is calculated by summing up the scores of each item, which ranges from 0 to 16. A higher score indicates higher level of neurotoxicity [22].

2.5.3. European Quality of Life Survey-5 Dimensions-3 Levels (EQ-5D-3L)

Generic HRQoL was evaluated using the EQ-5D-3L which essentially consists of two sections: the EQ-5D-3L descriptive system and the visual analogue scale (VAS) [27]. The EQ-5D-3L comprises five dimensions: mobility, self-care, usual activities, pain/discomfort, and anxiety/depression. The VAS records the patient’s self-rated health on a vertical visual analogue scale, where the endpoints are captioned “The best health you can imagine” and “The worst health you can imagine.” The final scores vary from 0 to 100, and higher score indicates a good HRQoL EQ-5D-3L has shown high reliability, content validity, construct validity, and responsiveness for oncologic patients [28]. It has been established that the Turkish version of EQ-5D-3L is a valid and reliable instrument [29].

2.6. Statistical Analysis

Statistical analyses were carried out using IBM SPSS Statistics for Windows (version 20.0. Armonk, NY: IBM Corp.). For all analyses, the cutoff for statistical significance was set at  < 0.05. The Kolmogorov–Smirnov test, kurtosis-skewness statistics, detrended Q-Q plots, and histograms were used to screen the normal distribution of the data. Descriptive statistics were calculated as mean and standard deviation (SD) for continuous data for normally distributed variables and n (%) for categorical variables.

2.6.1. Internal Consistency

Cronbach’s alpha coefficients were determined to assess the internal consistency of NFOSI-18 and NTX-4, with values greater than 0.7 presumed for sufficient internal consistency [30]. A change more than 10% variance in Cronbach’s alpha with the removal of any item was accepted as a cutoff for removing an item. Adjusted correlations following the deletion of individual items (corrected item-total) were also calculated to represent the homogeneity of scales. A correlation coefficient higher than 0.3 was assumed to keep items for further analyses [23].

2.6.2. Reliability of Items

The reliability of the individual’s items was explored using weighted kappa analysis; the obtained kappa coefficients were interpreted as follows: values ≤ 0 as indicating no agreement and 0.01–0.20 as none to slight, 0.21–0.40 as fair, 0.41– 0.60 as moderate, 0.61–0.80 as substantial, and 0.81–1.00 as almost excellent agreement [31].

2.6.3. Test-Retest Reliability

The intraclass correlation coefficient (ICC) values in 95% CI were calculated using two-way mixed effects and an absolute agreement to evaluate test-retest reliability. The obtained ICC values were interpreted as follows: very low (ICC ≤ 0.25), low (ICC = 0.26–0.49), moderate (ICC = 0.50–0.69), high (ICC = 0.70–0.89), and very high (ICC ≥ 0.90). Standard error of measurement (SEM) which corresponds to the amount of variance due to measurement error when a test is repeated was determined using the formula “” [32].

2.6.4. Convergent Validity

The construct validity of NFOSI-18 (total score and subscores) and NTX-4 was evaluated by calculating the Pearson correlation coefficients (r) between the NFOSI-18, NTX-4, and EQ-5D-3L. The strength of correlation was interpreted as follows: negligible (r ≤ 0.299), poor (r = 0.300 and 0.499), moderate (r = 0.500 and 0.699), strong (r = 0.700 and 0.899), and excellent (r ≥ 0.900) [23]. Coefficient values of 0.3 or higher were assumed to establish convergent validity for Turkish NFOSI-18 and NTX-4.

2.6.5. Floor-Ceiling Effects

To examine floor/ceiling effects, the percentage of participants that received minimum and maximum scores is used. Values greater than 15% were interpreted as indicating the presence of a floor or ceiling effect [33]. Additionally, the presence of the floor ceiling effect was expected to have an absolute skewness value (γ1) > 1. As a result, γ1 > +1 was thought to be an indicator of the floor effect, whereas γ|<-1 was assumed to be an indicator of the ceiling effect [34].

3. Results

The study was completed with 94 women with OC. The physical characteristics, disease-related characteristics, and generic HRQoL of participants are displayed in Table 1.

3.1. Internal Consistency

Turkish NFOSI-18 and Turkish NTX-4 had excellent internal consistency (Cronbach’s alpha: 0.919 and 0.917, respectively). Variation in internal consistency was acceptable with the exclusion of individual items (lower than 10%), and corrected item-total correlations were higher than 0.3 for both questionnaires (Table 2). Thus, all items, in both questionnaires, were retained for further analysis.

3.2. Reliability of Items

The reliability of individual items was detected as substantial to excellent for the Turkish NFOSI-18 (0.612 to 0.850) and as excellent for the Turkish NTX-4 (0.805 to 0.848) (Table 2).

3.3. Test-Retest Reliability

Turkish NFOSI-18 and Turkish NTX-4 showed good to excellent test-retest reliability (Table 3). The SEM in 95% CI for the Turkish NFOSI-18 and NTX-4 was calculated as 3.54 (2.99–4.64) and 1.32 (1.62–1.02), respectively (Table 3).

3.3.1. Convergent Validity

Statistically significant poor to moderate correlations were detected between Turkish NFOSI-18 scores and EQ-5D-3L scores (Table 4) ( < 0.05). Statistically significant poor to moderate correlations were found between Turkish NTX-4 scores and EQ-5D-3L scores (Table 4).

3.3.2. Floor-Ceiling Effects

Only one patient (1.06% of the study sample) had a total NFOSI-18 score of 0, and none of the patients had a total NFOSI-18 score of 72. For the total NTX-4 score, no patient had a score of 0, and three patients (3.20% of the study sample) had a score of 16. These scores were significantly less than the pre-decided cut-off scores of 15% for floor and ceiling effects. For NFOSI-18 and NTX-4 total scores, γ1 was calculated as −0.048 and −0.827, respectively, indicating absence of floor/ceiling effect.

4. Discussion

Our results demonstrated that the Turkish NFOSI-18 and Turkish NTX-4 had adequate internal consistency, item reliability, test-retest reliability, and convergent validity. Turkish NFOSI-18 and NTX-4 did not demonstrate any floor or ceiling effects.

Turkish NFOSI-18 and Turkish NTX-4 questionnaires showed high ICC values in terms of total scores indicating sufficient test-retest reliability for these questionnaires. These findings are in line with the results for Korean version of NFOSI-18 (ICC = 0.77) and NTX-4 (ICC = 0.84) [22]. The test-retest reliability of Treatment Side Effects and Functioning/Well-Being subscales of NFOSI-18 were shown to be lower than acceptable levels (ICC < 0.70) [35] while the other subscores had adequate test-retest reliability. The possible reason may be the reduced side effects associated with cycle-specific chemotherapy treatment. As the treatment parameters change during the course of chemotherapy, this may have led to alterations in patients’ perceptions in terms of burden of the treatment and general well-being [36]. Dose adjustments due to the toxicity, patient’s tolerance, or adverse effects are common in clinical practice and may cause variations in experienced complications or present symptoms [37]. As the interpretation of these subparameters in patient follow-up may be unreliable, using the changes in other subscores or total scores may be more beneficial. Another possible reason for obtaining low ICC values in some subscales may be due to the 14–21-day interval between the test and retest measurements. This relatively long duration was used to prevent the learning effect; however, some aspects related to disease status may have changed during this time and affected the reliability outcomes.

Our results of SEM value were less than 10% of the NFOSI-18 total score and were similar to the results by Trigg et al. [38] which yielded SEM values of 4.93 [36]. Additionally, previous research found a minimal clinical important difference of NFOSI-18 in the range of 5 to 7. Correspondingly, the calculated SEM values for the Turkish version of NFOSI-18 were lower than the minimal clinical important difference.

Two previous studies that investigated the internal consistency of original NFOSI-18 calculated Cronbach’s alpha coefficient as 0.80 [17] and 0.82 [38]. Korean NFOSI-18 (α = 0.84) and Turkish NTX-4 (α = 0.89) were also shown to have sufficient internal consistency [22]. The present research demonstrated the Turkish NFOSI-18 and Turkish NTX-4 to have adequate internal consistency similar to the outcomes of prior research as all questions contributed to total score and all original items were also retained in the Turkish version.

The EQ-5D-3L developed by World Health Organization is a generic HRQoL questionnaire frequently used to assess patients with OC by health professionals [27]. Lee et al. [22] employed the EQ-5D-3L to validate the Korean NFOSI-18 and Korean NTX-4. They detected poor-to-moderate correlations between NFOSI-18, NTX-4, and EQ-5D-3L [22]. Similarly, we obtained significant correlations between all EQ-5D-3L scores and NTX-4 and NFOSI-18 scores/subscores. The patients with OC have been reported to experience difficulties in the activities of daily living due to side effects of adjuvant therapy effects such as nausea, vomiting, loss of appetite, and fatigue [39]. The EQ-5D-3L consists of subscales that affect the patient’s ability to sustain the activities of daily living. Therefore, our results are no surprise as NFOSI-18 and EQ-5D-3L both interrogate the potential impact of the disease on the activities of daily living. Neurotoxicity occurs in approximately half of the patients with OC during the first two cycles of chemotherapy [40]. Age and low HRQoL scores have been identified as potential risk factors for the development of chemotherapy-induced peripheral neuropathy in patients with OC [40]. Neurotoxicity may lead to motor and sensory loss in hands and feet which may impair the daily living activities [41]. In this context, a correlation may be expected between the EQ-5D-3L and the NTX-4.

Our results of the NFOSI-18 mean score were higher than the results of the study by Trigger et al. [38]. This discrepancy might be explained by the difference between study samples. Trigger et al. [38] only included patients with stage 3-4 OC while the present study included patients with stage 2–4 OC according to FIGO. Recent FIGO consensus report suggests that patients with stage 2 OC cannot be classified as early stage due to high risk of recurrence or metastasis [42]. Additionally, adjuvant chemotherapy does not provide additional benefits in patients with stage 1 OC and is not indicated [43]. Besides, all patients in the present study underwent primary cytoreduction surgery and received chemotherapy and may have still been affected by the side effects of adjuvant/surgical treatments and experience similar symptoms with stage 3-4 patients as previously reported [43].

The strength of this study is rooted in its multicenter design, which may increase the generalizability of the findings. The most important limitation of this study is the insufficient number of patients to conduct more comprehensive analyses such as Rasch analysis or factor analysis. In the study by Trigger et al. [38], the validation of the NFOSI-18 was assessed by a multiple attribute analysis including 897 participants. However, our study did not achieve an adequate sample size for these analyses. Smaller representation of patients with stage 4 OC was another limitation of this study.

Data Availability

The study data are available from the corresponding author on reasonable request.

Ethical Approval

This study was performed in line with the principles of the Declaration of Helsinki. Approval was granted by the Noninvasive Research Ethics Board of Dokuz Eylul University (date: 02.06.2021; decision no: 2021/17-15).

All participants gave their written consent for participation and publishing of the study results. Informed consent was obtained from all participants included in the study.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Authors’ Contributions

All authors contributed to the study conception and design. Material preparation and data collection were performed by Husnu Tore Yavuzsen, Karya Polat, Murat Keser, Zeynep Gulsum Guc, and Merve Keskinkılıc. Analysis was performed by Sukriye Cansu Gultekin. The first draft of the manuscript was written by Sukriye Cansu Gultekin. Critical review of the first draft of the manuscript was performed by Husnu Tore Yavuzsen, Tugba Yavuzsen, and Didem Karadibak. All authors have read and approved the final version of the manuscript.