An Analysis of Predictive Sample-to-Cutoff Index for HIV Infection Confirmation Using Elecsys® HIV Combi PT Assay

Xia, Xiaohong; Zhang, Xiang; Zhou, Jun; Zhang, Mengying

doi:https://doi.org/10.1155/2022/5097189

International Journal of Clinical Practice

On this page

Abstract Introduction Materials and Methods Results Discussion Data Availability Conflicts of Interest Authors’ Contributions References Copyright Related Articles

Research Article | Open Access

Volume 2022 | Article ID 5097189 | https://doi.org/10.1155/2022/5097189

An Analysis of Predictive Sample-to-Cutoff Index for HIV Infection Confirmation Using Elecsys® HIV Combi PT Assay

Xiaohong Xia,^1,2Xiang Zhang,³Jun Zhou,^1,2and Mengying Zhang^1,2

Academic Editor: Dawei Cui

Received20 Jan 2022

Accepted05 Jul 2022

Published10 Aug 2022

Abstract

Background. Early and rapid diagnosis is crucial in HIV preventing and treatment. However, the false-positive rate (FPR) by 4-th generation detection assays was high in low-HIV-prevalence regions. Objectives. To analyze the relation between sample-to-cutoff index (COI) and HIV confirmatory results, and to explore a new COI threshold in our own laboratory to predict HIV infection. Methods. We retrospectively analyzed primarily reactive results by Elecsys® HIV combi PT assays and their confirmatory results by western blot (WB) at Nanjing Center for Disease Control and Prevention (CDC). The mean COI values of true positive (TP), false positive (FP), and indeterminate groups were compared, and receiver operating characteristic curve (ROC) analysis was performed to determine the optimal COI value for predicting HIV infection. Results. Totally 150,980 HIV serological results were reviewed, and 305 (0.2%) were primarily reactive. There are 82 (26.89%) true positives, 210 (71.92%) false positives, and 11 indeterminate samples confirmed by WB tests, and another 2 patients rejected WB tests. Mean COI values of TP (643.5) were greatly higher than that of FP (3.174) (), but there is no significant difference between FP and indeterminate groups. Combining the requirement of HIV diagnosis and ROC analysis, 9.87 was established as the optimal threshold to predict the infection, with 100% sensitivity and 99.99% specificity. Conclusions. By adjusting the COI threshold, the FP samples can be reduced and the efficiency of screening assays can be increased, which can save much additional reagent and staff costs and much time for delivery of HIV test results.

1. Introduction

Human immunodeficiency virus (HIV) infection, which can progress to acquired immune deficiency syndrome (AIDS), is becoming a global pandemic [1]. It was reported that the number of people living with HIV/AIDS (PLWH) in China at the end of 2014 is nearly 501,000 [2]. Most recent estimation results indicated that the number was already beyond one million by the end of 2018 and would keep growing, causing much pressure of HIV/AIDS prevention and control [3]. Compared with established infections, new and recent HIV infections are more infectious. And acute infection contributes significantly to HIV transmission, accounting for 10% to 50% of new cases [4–9], and usually the median time interval between infection and diagnosis is 3.3 years [10]. Nearly 40% of PLWH are unaware of their HIV status, which greatly increases the potential risk of transmission. Thus, early and rapid identification of HIV infection is crucial to prevent its further transmission, and HIV testing algorithms aim to minimize the time between infection and confirmatory diagnosis with maximum sensitivity and specificity [11–13]. Simultaneously detecting the HIV-1/2 antibodies and the P24 antigen, 4th-generation assays allow for more sensitive detection of early acute HIV infection cases and reduction of the window period by 4 to 8 days compared with 3rd-generation assays, which only detect HIV antibodies [14–18]. Hence, in current diagnostic algorithms, 4th-generation assays are gradually replacing 3rd-generation assays and being implemented worldwide as screening tests [19]. Elecsys® HIV combi PT assays (Roche Diagnostics GmbH, Penzberg, Germany) are performed to screen HIV infection, and the results are showed as sample-to-cutoff index (COI), which is calculated automatically by comparing the chemiluminescence signal magnitude of the sample with the cutoff value determined by calibration. According to the manufacturer’s specification, samples are defined as reactive if the COI ≥1.0, and nonreactive when COI is less than 0.9. Samples (0.9 ≤ COI < 1.0) were defined as borderline cases, processed as reactive specimens [20,21]. Despite having higher sensitivity, 4th-generation assays are found to have a higher false-positive rate (FPR). Due to a variety of factors, weakly positive COI results are commonly observed and often followed by a nonreactive or indeterminate confirmatory test result in uninfected population [22–25]. It is difficult to differentiate early seroconversion from false positives, especially for those slightly positive samples. In our hospital, according to the Chinese Centers for Disease Control and Prevention (CDC) guideline, a specimen with COI ≥0.9 has to be retested after long and high-speed centrifugation. When the retest result is still reactive, a different method test should be used before submitting for confirmation at Nanjing CDC. These primary and confirmatory tests significantly largely increase reagent and staff costs and also slow the delivery of the patient’s test results to the clinician. In addition, HIV false-positive results may cause inappropriate anxiety to patients or even undesirable consequences [13].

Recently, many studies show that there was an association between test values and subsequent confirmatory results. As COI values increased, the number of FP decreased, and the possibility of positive results was higher [13, 26, 27]. Thus, the present study was carried out to evaluate the performance of Elecsys® HIV combi PT assay in a large cohort of patients from 2018 to 2019 in Nanjing, a low-HIV-prevalence region of East China, to analyze the sensitivity, specificity, and positive predictive value (PPV) at different COI, and to seek for an optimal cutoff that can predict HIV infection at early period.

2. Materials and Methods

2.1. Study Population

We retrospectively collected testing episodes from the First Affiliated Hospital of Nanjing Medical University between January 2018 and January 2019. A total of 150, 980, including all repeatedly reactive tests, HIV screening results were analyzed by Elecsys® HIV combi PT assays. All the data were from the laboratory information system (LIS) and hospital information system (HIS) of the First Affiliated Hospital of Nanjing Medical University.

2.2. HIV Diagnostic Algorithm

All specimens were centrifuged for 10 minutes at 3500 rpm and then detected by Elecsys® HIV combi PT assay on the Cobas e601 analyzer (Roche Diagnostics GmbH, Mannheim, Germany). Any sample with COI ≥0.9 was retested in duplicate after 10000 , 10-min centrifugation. For samples still reactive, a 3rd-generation gold-labeled immune assay (GIA) kit, XinChuang HIV-1/2Ab (InTec, INC, Xiamen, China), was used to perform the supplemental tests for initially screening reactive cases. Initially, screening reactive cases should be submitted for confirmatory tests (WB, nucleic acid, or p24 antigen tests). Currently, western blot (WB) analysis (HIV1/2 BLOT 2.2; MP Biomedicals, Singapore) is taken for confirmation at Nanjing CDC; nucleic acid and p24 antigen tests were not applied (Figure 1). Thus, for the non-positive by WB, timely follow-up (week 2, week 4, month 3, and month 6) is required for HIV infection exclusion. According to the Chinese CDC criteria [28], HIV WB results interpretation is as follows: HIV-1 Positive: presence of at least two env bands plus one gap or one pol band; Indeterminate: reactive to any specific band but not compatible with positive criteria; Negative: absence of any of the specific bands; and HIV-2 Positive: presence of gp36 band.

2.3. Statistical Analysis

The continuous variables were expressed by mean, and multiple groups were performed using the one-way ANOVA test and Kruskal–Wallis test. We use the receiver operating characteristic curve (ROC) analysis to determine the optimal cutoff value for predicting true positives (TPs). All statistical analyses were two-sided, and value < 0.05 was considered statistically significant. GraphPad Prism 7.0 (GraphPad Software, San Diego, CA, USA) was used for data analysis.

3. Results

3.1. Testing Episodes

During the study period, a total of 150,980 specimens were tested by using Elecsys® HIV combi PT assay, including 150,675 patients negative and 305 patients initially reactive (Figure 1). 5 tests were negative and 300 were repeatedly reactive after centrifugation (10000 g, 10 min) and HIV-1/2 GIA kit testing. In all 300 repeatedly reactive cases, 82 of 298 cases were found positive, 205 of 298 were negative, and 11 of 298 were indeterminate after WB tests and follow-ups. There are 2 patients who refused to WB test and were lost to follow-up. Overall, according to the HIV identification algorithm in our hospital and Nanjing CDC, HIV prevalence in this study is 0.054% (82/150,980), largely consistent with previous reports in general Chinese population [29], and 82 (29%) and 205 (71%) of 287 cases were identified as true positives (TPs) and false positives (FPs), respectively. The positive rate (PR) of males (63/82) was 76.8%, much higher than 23.2% of females (19/82). False-positive cases accounted for 0.10% of the total.

3.2. COI Distribution

The COI of all samples ranged from 0.1 to 3523, the highest COI for true-negative (TN) samples was 23.77, and the lowest COI for TP was 9.95. As shown in Figure 2, the mean COI values in TP (643.5) were significantly higher than that in FP (3.17) () and indeterminate samples (IN) (6.839) (), while the difference of mean COI in the IN group and FP group was not statistically significant.

There were 10 kinds of WB bands patterns in all 82 WB-positive specimens (Table 1). No relation between WB bands number and COI values was observed, which is different from the previous study by Wang et al. [30].

3.3. ROC Analysis

ROC analysis showed that the sensitivity and specificity were 100% and 99.86%, respectively, when COI >1.0, while the FP number was up to 210. And no loss of sensitivity was observed until the COI threshold was raised to 9.87, while the FP number can be reduced to 14. Table 2 showed different sensitivity, specificity, false-positive situation, and PPV at different COI values. For COI threshold values of 9.87 and 12.79, the sensitivity can achieve 100% and 98.78%, with much lower FP number (14 and 6, respectively). When the COI was raised to 37.75, it can achieve 100% of specificity, while the sensitivity was decreased 97.56%. Considering the clinical utility, 9.87 was recommended as the optimal COI by ROC (with 100% sensitivity and 99.99% specificity).

4. Discussion

Early diagnosis of HIV infection is necessary to facilitate prompt treatment and prevent further transmission, as new and recent infections are more infectious than chronic infections [4]. The Global Health Sector Strategies on HIV (2016–2021) have proposed that 90% of PLWH can be tested [31]. Only 68% of PLWH were diagnosed in China, although it is low HIV prevalence [32]. Hence, HIV screening assay must be highly sensitive and can detect all HIV subtypes. Currently in China, few of new HIV infection can be detected at early period due to rare application of 4-th generation assay and nucleic acid (NAT) and p24 antigen tests in many hospitals [30]. In addition, 4-th generation assay with high sensitivity has a high false-positive rate (FPR = 0.14% in our study), which requires many retests and confirmatory tests subsequently according to the current diagnostic algorithm. These retests and confirmatory tests significantly increase reagent and staff costs, slowing the transmission of the test results to the clinician and further treatment. And it also caused many patients unnecessary anxiety when waiting for the confirmatory results. It was reported by many studies that HIV serology COI values were strongly correlated with the probability of confirmed HIV infection [11,12]. Thus, understanding the performance of fourth-generation assay and establishing a predicted threshold COI for HIV positive results in a local region were pivotal in clinical practice.

The present study retrospectively analyzed 150,980 specimens performed by Elecsys® HIV combi PT assay, in which the prevalence of HIV was 0.054%. Although the 4-th generation Elecsys® HIV combi PT assay was reliable for screening of HIV with high sensitivity (100%) and specificity (99.86%), it resulted in a high false-positive rate (FPR) (0.14%) in Nanjing, close to other studies in the low-HIV-prevalence setting [19]. The mean COI of TP (643.5) was significantly higher than that of FP (3.17). And as COI values increased, the PPV for HIV infection also went up, while FPR decreased. In our study, when COI values were at 1.00, 9.87, and 37.75, the PPV was 28.08%, 85.42%, and 100%, respectively, while FP number was 210, 14%, and 0, respectively. Previous reports have proposed different predicted COI thresholds, while it varied depending on different test assays, patient population, local prevalence, and technical factors [11,19,27,33]. Due to such differences, each institution should have its own COI threshold values. According to the ROC analysis of our study, the COI value with 100% PPV for HIV confirmation was ≥37.75. While considering the requirement of HIV diagnosis for sensitivity, we recommended 9.87 as the optimal COI threshold, which provided a sensitivity and specificity of 100% and 99.99%, respectively, and eliminated 196 FP cases.

Actually, for HIV diagnosis protocol, follow-ups and WB from CDC are still be used as mainly confirmatory tests according to legislation. However, in view of high false-positive rate of HIV screening, which led to great waste of medical resource, we suggest to raise the threshold properly so that the false-positive rate can be reduced in low-HIV-prevalence regions like Nanjing. Also, the COI value can be used to predict infection for those specimens with high COI value so that the clinicians can make some timely and appropriate measures before the confirmatory results, not just waiting for CDC’s feedback. The prediction can also help relieve patients' depression and anxiety.

The limitation of this study involves some unidentified HIV infection status due to refusing WB test and some indeterminate WB results. Most of these patients were lost to follow up, and only 3 indeterminate patients were in contact and found to be positive subsequently. And the nucleic acid test (NAT) and p24 antigen test are also not applied at Nanjing CDC and our hospitals, which resulted in hardness to knowing some seroconversion or acute HIV infection and needed more detailed follow-up. Then, the study was operated in one single medical center, and hence the established COI threshold cannot be applied by other institutions due to differences in patient and technical factors; therefore a multicenter study is required.

In summary, this study showed that the Elecsys® HIV combi PT had excellent sensitivity, but high percent of false-positive results led to various problems. An optimal COI value is necessary for predicting the final status of HIV infection, rapidly and preliminary reportingHIV infection prior to a confirmatory result. In our hospital, COI ≥9.87 was recommended to predict HIV infection at early period, to make up for the lack of high FPR of the 4-th generation HIV screening test.

Data Availability

The data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Authors’ Contributions

Xiaohong Xia and Mengying Zhang conceptualized and designed the study. Administrative support was carried out by Xiang Zhang and Mengying Zhang. Provision of study materials or patients was done by Xiaohong Xia and Mengying Zhang. Collection and assembly of data was performed by Xiaohong Xia, Mengying Zhang, Xiang Zhang, Jun Zhou, and Xiaojie Zhang. Data analysis and interpretation was carried out by Xiaohong Xia and Mengying Zhang. All the authors contributed to manuscript writing and final approval of the manuscript. Xiaohong Xia and Xiang Zhang contributed equally to this work.

References

E. Fanales-Belasio, M. Raimondo, B. Suligoi, and S. Butto, “HIV virology and pathogenetic mechanisms of infection: a brief overview,” Annali dell’Istituto superiore di sanita, vol. 46, no. 1, pp. 5–14, 2010.
View at: Publisher Site | Google Scholar
China NHAF, “China AIDS Response Progress Report,” 2015, https://www.unaids.org/sites/default/files/country/documents/CHN_narrative_report_2015.pdf.
View at: Google Scholar
P. Lyu and F. F. Chen, “National HIV/AIDS epidemic estimation and interpretation in China,” Zhonghua liu xing bing xue za zhi = Zhonghua liuxingbingxue zazhi, vol. 40, no. 10, pp. 1191–1196, 2019.
View at: Publisher Site | Google Scholar
G. M. Shaw and E. Hunter, “HIV transmission,” Cold Spring Harbor perspectives in medicine, vol. 2, no. 11, Article ID a006965, 2012.
View at: Publisher Site | Google Scholar
J. Stekler, J. Maenza, C. E. Stevens et al., “Screening for acute HIV infection: lessons learned,” Clinical Infectious Diseases, vol. 44, no. 3, pp. 459–461, 2007.
View at: Publisher Site | Google Scholar
M. J. Wawer, R. H. Gray, N. K. Sewankambo et al., “Rates of HIV-1 transmission per coital act, by stage of HIV-1 infection, in Rakai, Uganda,” Journal of Infectious Diseases, vol. 191, no. 9, pp. 1403–1409, 2005.
View at: Publisher Site | Google Scholar
B. G. Brenner, M. Roger, J. P. Routy et al., “High rates of forward transmission events after acute/early HIV-1 infection,” Journal of Infectious Diseases, vol. 195, no. 7, pp. 951–959, 2007.
View at: Publisher Site | Google Scholar
W. C. Miller, N. E. Rosenberg, S. E. Rutstein, and K. A. Powers, “Role of acute and early HIV infection in the sexual transmission of HIV,” Current Opinion in HIV and AIDS, vol. 5, no. 4, pp. 277–282, 2010.
View at: Publisher Site | Google Scholar
T. D. Hollingsworth, R. M. Anderson, and C. Fraser, “HIV-1 transmission, by stage of infection,” Journal of Infectious Diseases, vol. 198, no. 5, pp. 687–693, 2008.
View at: Publisher Site | Google Scholar
HIV/AIDS Surveillance in Europe 2018–2017 Data, “European centre for disease prevention and control,” 2018, http://ecdc.europa.eu/en/publications-data/hivaids-surveillance-europe-2018-2017-data.
View at: Google Scholar
T. O. Jensen, P. Robertson, R. Whybin et al., “A signal-to-cutoff ratio in the Abbott architect HIV Ag/Ab Combo assay that predicts subsequent confirmation of HIV-1 infection in a low-prevalence setting,” Journal of Clinical Microbiology, vol. 53, no. 5, pp. 1709–1711, 2015.
View at: Publisher Site | Google Scholar
L. Li, D. Puddicombe, S. Champagne et al., “HIV serology signal-to-cutoff ratio as a rapid method to predict confirmation of HIV infection,” European Journal of Clinical Microbiology & Infectious Diseases, vol. 37, no. 8, pp. 1589–1593, 2018.
View at: Publisher Site | Google Scholar
L. Wang, Y. Xiao, X. D. Tian, Jx Ruan, W. Chen, and Y. Yu, “HIV infection in Xi’an, China: epidemic characterization, risk factors to false positives and potential utility of the sample-to-cutoff index to identify true positives using Architect HIV Ag/Ab combo,” Antimicrobial Resistance and Infection Control, vol. 8, no. 1, p. 9, 2019.
View at: Publisher Site | Google Scholar
A. Alatoom, A. Alatoom, S. Burns et al., “Comparison of 4th-generation HIV antigen/antibody combination assay with 3rd-generation HIV antibody assays for the occurrence of false-positive and false-negative results,” Laboratory Medicine, vol. 46, no. 2, pp. e28–e29, 2015.
View at: Publisher Site | Google Scholar
E. O. Mitchell, G. Stewart, O. Bajzik, M. Ferret, C. Bentsen, and M. K. Shriver, “Performance comparison of the 4th generation bio-rad laboratories GS HIV combo Ag/Ab EIA on the EVOLIS automated system versus Abbott ARCHITECT HIV Ag/Ab combo, ortho Anti-HIV 1+2 EIA on vitros ECi and siemens HIV-1/O/2 enhanced on Advia centaur,” Journal of Clinical Virology, vol. 58, no. Suppl 1, pp. e79–84, 2013.
View at: Publisher Site | Google Scholar
K. Lee, H. D. Park, and E. S. Kang, “Reduction of the HIV seroconversion window period and false positive rate by using ADVIA Centaur HIV antigen/antibody combo assay,” Annals of laboratory medicine, vol. 33, no. 6, pp. 420–425, 2013.
View at: Publisher Site | Google Scholar
B. Weber, E. H. Mbargane Fall, A. Berger, and H. W. Doerr, “Reduction of diagnostic window by new fourth-generation human immunodeficiency virus screening assays,” Journal of Clinical Microbiology, vol. 36, no. 8, pp. 2235–2239, 1998.
View at: Publisher Site | Google Scholar
M. W. Pandori, J. Hackett Jr., B. Louie et al., “Assessment of the ability of a fourth-generation immunoassay for human immunodeficiency virus (HIV) antibody and p24 antigen to detect both acute and recent HIV infections in a high-risk setting,” Journal of Clinical Microbiology, vol. 47, no. 8, pp. 2639–2642, 2009.
View at: Publisher Site | Google Scholar
S. Liang, G. Deng, S. Zhou, J. Zeng, W. Tan, and X. Yuan, “A retrospective analysis of the application of the Elecsys^®HIV combi PT assay in southern China,” Journal of Clinical Laboratory Analysis, vol. 32, no. 3, Article ID e22287, 2018.
View at: Publisher Site | Google Scholar
D. L. Uettwiller-Geiger, M. Lessig, J. An et al., “Analytical and Clinical Performance Evaluation of the Elecsys HIV combi PT Assay on the cobas e 602 Analyzer for the Diagnosis of Human Immunodeficiency Virus,” American Journal of Clinical Pathology, vol. 151, no. 4, pp. 377–385, 2019.
View at: Publisher Site | Google Scholar
E. Y. Song, M. Hur, E. Y. Roh, M. H. Park, H. W. Moon, and Y. M. Yun, “Performances of four fourth-generation human immunodeficiency virus-1 screening assays,” Journal of Medical Virology, vol. 84, no. 12, pp. 1884–1888, 2012.
View at: Publisher Site | Google Scholar
C. P. Erickson, T. McNiff, and J. D. Klausner, “Influenza vaccination and false positive HIV results,” New England Journal of Medicine, vol. 354, no. 13, pp. 1422-1423, 2006.
View at: Publisher Site | Google Scholar
S. Kim, J. H. Lee, J. Y. Choi, J. M. Kim, and H. S. Kim, “False-positive rate of a “fourth-generation” HIV antigen/antibody combination assay in an area of low HIV prevalence,” Clinical and Vaccine Immunology, vol. 17, no. 10, pp. 1642–1644, 2010.
View at: Publisher Site | Google Scholar
V. S. Mahajan, C. A. Pace, and P. Jarolim, “Interpretation of HIV serologic testing results,” Clinical Chemistry, vol. 56, no. 10, pp. 1523–1526, 2010.
View at: Publisher Site | Google Scholar
T. Shima-Sano, R. Yamada, K. Sekita et al., “A human immunodeficiency virus screening algorithm to address the high rate of false-positive results in pregnant women in Japan,” PLoS One, vol. 5, no. 2, Article ID e9382, 2010.
View at: Publisher Site | Google Scholar
L. Wang, K. H. Zhou, H. P. Zhao et al., “The characteristics of screening and confirmatory test results for HIV in Xi’an, China,” PLoS One, vol. 12, no. 7, Article ID e0180071, 2017.
View at: Publisher Site | Google Scholar
L. Chacon, M. L. Mateos, and A. Holguin, “Relevance of cutoff on a 4th generation ELISA performance in the false positive rate during HIV diagnostic in a low HIV prevalence setting,” Journal of Clinical Virology, vol. 92, pp. 11–13, 2017.
View at: Publisher Site | Google Scholar
National guideline for detection of HIV/AIDS, “Chinese center for disease control and prevention,” Chin J Viral Dis, Nov, vol. 6, 2016.
View at: Google Scholar
L. Zhang, E. P. F. Chow, J. Jing et al., “HIV prevalence in China: integration of surveillance data and a systematic review,” The Lancet Infectious Diseases, vol. 13, no. 11, pp. 955–963, 2013.
View at: Publisher Site | Google Scholar
L. Wang, J. Y. Wang, X. D. Tian, J. X. Ruan, Y. Yu, and F. Yan, “Sample-to-cutoff ratios using Architect HIV Ag/Ab Combo: the influence with the results of supplemental tests and optimal cutoff value to predict HIV infection,” Journal of Clinical Laboratory Analysis, vol. 33, no. 5, Article ID e22866, 2019.
View at: Publisher Site | Google Scholar
World Health Organization, “Global Health Sector Strategies on HIV,” 2016, https://www.who.int/westernpacific/health-topics/hiv-aids.
View at: Google Scholar
World Health Organization, “Programmes and Projects-HIV/data and Statistics,” https://www.who.int/hiv/data/en/.
View at: Google Scholar
A. Acar, S. Kemahli, H. Altunay et al., “The significance of repeat testing in Turkish blood donors screened with HBV, HCV and HIV immunoassays and the importance of S/CO ratios in the interpretation of HCV/HIV screening test results and as a determinant for further confirmatory testing,” Transfusion Medicine, vol. 20, no. 3, pp. 152–159, 2010.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2022 Xiaohong Xia et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies