Validation of the Kidney Failure Risk Equation in the Colombian Population

Larrarte, C.; Vesga, J.; Ardila, F.; Aldana, A.; Perea, D.; Sanabria, M.

doi:https://doi.org/10.1155/2024/1282664

International Journal of Nephrology

On this page

Abstract Introduction Materials and Methods Analysis Results Discussion Conclusions Data Availability Disclosure Conflicts of Interest Authors’ Contributions Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2024 | Article ID 1282664 | https://doi.org/10.1155/2024/1282664

Validation of the Kidney Failure Risk Equation in the Colombian Population

C. Larrarte,¹J. Vesga,¹F. Ardila,²A. Aldana,³D. Perea,³and M. Sanabria²

Academic Editor: Jaime Uribarri

Received20 Sept 2023

Revised19 Jan 2024

Accepted27 Jan 2024

Published17 Feb 2024

Abstract

Introduction. Chronic kidney disease prevention programs must identify patients at risk of early progression to provide better treatment and prolong kidney replacement therapy-free survival. Risk equations have been developed and validated in cohorts outside of Colombia, so this study aims to evaluate the discrimination and calibration of the four-variable kidney failure risk equation in a Colombian population where it has yet to be validated. Methods. External validation study of a kidney failure risk equation using a historical cohort of patients with CKD stages 3, 4, and 5, adults without a history of dialysis or kidney transplantation with a two-year follow-up, belonging to the Baxter Renal Care Services Colombia network. The discriminatory capacity of the model was evaluated by the concordance index using Harrell’s C statistic, and the time-dependent area under the receiver operating characteristic (ROC) curve was estimated using the nearest neighbor method, as well as the optimal cut-off point for sensitivity and specificity. Calibration was determined by the degree of agreement between the observed outcome and the probabilities predicted by the model using the Hosmer–Lemeshow statistic. Results. A total of 5,477 patients were included, with a mean age of 72 years, 36.4% diabetic, and a mean baseline eGFR of 36 ml/min/1.73 m². The rate of dialysis initiation was three events per 100 patient-years, 95% CI (2.9–3.6). The optimal cutoff for sensitivity was 0.94, for specificity, 0.76, and the area under the ROC curve was 0.92. Harrell’s C-statistic was 0.88 for the total population, 0.88 for diabetic patients, and 0.93 for those 65 years or older. The validation of the model showed good calibration. Conclusions. In this Colombian cohort, the four-variable KFRE with a two-year prediction horizon has excellent calibration and discrimination, and its use in the care of CKD Colombian patients is recommended.

1. Introduction

Chronic kidney disease (CKD) is a syndrome with multiple pathophysiological features that have been elucidated for decades [1–4]. It is a highly prevalent disease associated with a declining quality of life, morbidity, and higher mortality rates [5, 6]. One of the cornerstones of CKD treatment is based on measures that aim to halt the deterioration of kidney function, prevent the need for dialysis, and reduce the risk of cardiovascular events [7, 8]. Considering that the progression of CKD increases the burden of disease and the cost of care, it is essential to understand the risk of progression to establish preventive or therapeutic measures that can positively influence patient outcomes.

Several demographic and clinical characteristics may be associated with differences in the rate of progression to kidney failure, defined as the need for dialysis or kidney transplant [9–13].

Furthermore, the development of clinical epidemiology has led to an understanding of how multiple risk factors can be combined to generate a prognostic risk model [14]. The use of these models in daily clinical practice has become a valuable tool for making prognostic and therapeutic decisions, although it is not without methodological limitations [15]. Among these, a critical step before the widespread use of a risk model in each jurisdiction is the external validation to assess its predictive performance in different patient populations and, when necessary, adapt the model to local circumstances or include new predictors [16, 17].

Specifically, in patients with CKD, some predictive models for progression to kidney failure were developed using inappropriate methods and were poorly reported [18]. Subsequently, Tangri et al. developed a predictive model for the progression of CKD to kidney failure in the Canadian population. The model, which includes four variables (age, sex, albuminuria, and e-GFR), gave better results than the others [19]. Although the eight-variable equation was the most accurate for predicting the risk of developing kidney failure, not all parameters are available in the initial evaluation of CKD patients; therefore, the four-variable equation may be easier to integrate into clinical practice.

In 2016, Tangri et al. published a multinational validation of the four- and eight-variable equations in 31 international cohorts, demonstrating adequate prediction of the development of kidney failure at 2 and 5 years [20, 21].

In the Colombian context, Baxter Renal Care Services has implemented a preventive program for CKD stages 3 to 5, currently serving around 20,000 patients nationwide. This program integrates the primary and specialized healthcare levels, intervening in an early and multidisciplinary for patients with CKD [22, 23].

To the best of our knowledge, the KFRE has not been validated in Colombia, and there is no scientific information to support the routine use of a CKD progression risk prediction equation that can be applied in this population. In Latin America, this risk equation has been validated recently for the Peruvian population. The authors concluded that the equation needs recalibration for this population [24]. Therefore, a validation study of the four-variable KFRE was conducted on the Colombian population.

2. Materials and Methods

2.1. Study Design and Patients

This is an external validation study of a prediction model using a retrospective multisite cohort. The population object of this study is made up of adult patients with CKD stage 3, 4, or 5 without dialysis, treated in the renal clinic network of Baxter Renal Care Services (BRCS) Colombia, in the period between January 1, 2015, and December 31, 2019, guaranteeing a follow-up of up to 2 years. The inclusion criteria were being 18 years of age or older, having a diagnosis of chronic kidney disease stage 3, 4, or 5 without dialysis (with at least two eGFR measurements according to CKD-EPI 2009 less than 60 ml/min/1.73 m², in a period equal to or greater than 90 days), and having a measurement of the urine albumin-creatinine ratio (uACR). The following were excluded from the study: pregnant women, patients who had previously required dialysis or a kidney transplant, patients with an indication for dialysis without acceptance of dialysis, and patients with a follow-up period of less than two years unless the result of the dialysis initiation, transplantation, or death was presented. The study protocol was approved by the Clinical Research Ethics Committee of Renal Therapy Services Colombia (February 17, 2022, minutes, item number 002), which exempted informed consent since this study does not contain identifiable information and is a retrospective observational study.

3. Data Source and Analysis

3.1. Baseline Characteristics of the Patients

Some demographic variables, such as age, race, and sex, were included. The comorbid conditions described were a history of diabetes and cardiovascular disease (the presence of coronary artery disease and/or peripheral arterial disease). Laboratory parameters included serum creatinine and RAC.

CKD was defined and classified according to the 2012 KDIGO guidelines [7]. GFR was estimated using the CKD-EPI 2009 formula [25]. The KFRE of four variables (age, sex, baseline eGFR, and log uACR) was calculated for the non-North American population as proposed by Tangri, only on the first visit [20].

The outcomes evaluated were the need for kidney replacement therapy (hemodialysis, peritoneal dialysis, or kidney transplantation) and mortality. Data were obtained from the electronic medical records.

3.2. Statistical Analysis

Data are presented as the mean and standard deviation for variables with a normal distribution and as the median and interquartile range for variables with a non-normal distribution. Categorical variables are presented as frequencies and percentages. No data imputation procedure was used to control for missing data; a whole-case analysis approach was used.

Incidence rates with 95% confidence intervals were estimated for initiating kidney replacement therapy and mortality.

A Cox proportional hazards regression model was fitted, and the model’s performance was assessed for discriminatory and calibration ability. Discriminative ability was set by the concordance index with Harrell’s C statistic, and the time-dependent calculation of the area under the receiver operating characteristic (ROC) curve was estimated using the nearest neighbor statistical method. The optimal cut-off points for sensitivity and specificity were also estimated. Calibration was determined by the degree of agreement between the observed outcome and the probabilities predicted by the model using the Hosmer–Lemeshow statistic. Stata 16® (StataCorp. 2019. Stata statistical software: Release 16. College Station, TX: StataCorp LLC.) was used for statistical analyses.

4. Results

4.1. Patients

A total of 5,477 patients were included in the analysis, 4690 (85.6%) completed the 2-year follow-up period (see Figure 1); the mean age was 72 years, the proportion of diabetic patients was 36.4%, and the mean baseline eGFR was 36 ml/min/1.73 m². The dialysis initiation rate was estimated to be 3 events per 100 patient years, and the mean observation period was 683 days. The mortality rate in this population was 4.6 deaths per 100 patient years. Details are shown in Table 1.

Table 2 shows the hazard ratios (HRs) for progression to kidney failure of the four KFRE variables in the original cohort compared with the HRs in the study population, which were very similar.

4.2. Model Discrimination

The model’s discriminative ability was evaluated using Harrell’s C statistic, which was 0.88. In general, the model can adequately discriminate between patients who experience the event of interest and those who do not. See Table 3. Similarly, when discriminating by diabetes, race, and age, the C statistics of the original cohort are similar to those estimated in the Colombian study population.

Model discrimination was also presented graphically, with an area under the time-dependent ROC curve (AUC) of 0.92, which represents adequate discriminatory power, i.e., the AUC expresses the probability of the model to decide whether a patient has the event of interest or not. In addition, optimal points were found for sensitivity = 0.938 and specificity = 0.757. This implies that the model has a high ability to correctly identify people who will need kidney replacement therapy, achieving a remarkable 94% success rate in correctly classifying these patients. On the other hand, when considering specificity, the model demonstrates a moderate ability to correctly recognize patients who do not require KRT. Approximately 75% of patients not requiring KRT were correctly classified by the model. In predialysis care, it is fundamental to detect all cases requiring KRT. Therefore, a high sensitivity proves to be of significant value, ensuring that no potential patients for KRT are overlooked. See Figure 2.

4.3. Model Calibration

The calibration of our model was assessed by comparing expected events with the observed events (see Figure 3) using the Hosmer-Lemeshow statistic; the goodness of fit test was 0.952, reflecting the agreement between the estimated predictions and the observed events.

5. Discussion

In modern healthcare systems, there is a growing interest in developing predictive tools for outcomes of interest for the population. This is the case with CKD and predicting the risk of progression. In this direction, Tangri et al. have developed a risk equation for kidney failure, aligning the risk estimate with the availability of resources in a health system, thus personalizing the care of patients with CKD [26]. As a derived risk prediction model moves from one population to another, it is essential to assess whether the model being validated adequately predicts risk in this new population so that its results can be generalized to similar individuals, even if they differ in many ways from those from which the model was derived [21].

The present study of external validation of the four-variable KFRE in the Colombian population has revealed a remarkable discriminatory capacity for this model, both for the C statistic and the area under the curve consistently being around 0.9, indicating an excellent discriminative ability, very similar to that reported in the meta-analysis for the multinational evaluation of this risk equation [21]. It should be noted that the proportion of patients with micro- or macroalbuminuria in our population is slightly higher than that found in the derivation cohort [20] and in the summary measure of the multinational evaluation with meta-analysis [21]. However, it is also interesting to note that this proportion of patients with albuminuria is very variable in the various studies included in this meta-analysis, an aspect that in our view confers relevance to the validation and does not invalidate the results. Otherwise, in Latin America, Bravo et al. found that the equation has good discrimination but poor calibration in the Peruvian population; that is, the model underestimates the risk of kidney failure in the short term and overestimates it in the long term; these results could be related mainly to the way the population in the study was selected [24].

On the other hand, the calibration of the model yielded a goodness-of-fit test more significant than 0.9, with graphical evidence of excellent calibration in all risk quintiles, a finding somewhat different from that reported in the referenced meta-analysis for non-North American cohorts [21], where there was a tendency to overestimate risk. Consequently, the two-year predictive performance of the four-variable KFRE in Colombian patients with stage 3, 4, or 5 CKD without the need for dialysis could be considered quite adequate.

In discussing the limitations of our study, it is worth noting that patients who were candidates for dialysis but opted for conservative treatment and refused therapy were excluded, although this number was relatively marginal. Furthermore, in the present study, the death event was censored and not considered a competing risk with the initiation of kidney replacement therapy (KRT). It is important to note that some authors suggest treating death as a competing risk only in predictions with a longer follow-up, such as five years, to avoid biased estimates [27].

In 2021, Ramspek et al. performed an external validation of the kidney failure equation that included death as a risk competing with the need for KRT. This study found that models with a short prediction horizon (one or two years) had similar results to models that ignored competing risks. However, models that predict the 5-year risk and do not include death as a competing risk were found to overestimate the risk of progression to kidney failure [28]. To predict the risk of kidney failure in the short term (two years), we, like other authors, recommend the use of a four-variable KFRE without adjustments for competing risks [28].

Implementing this tool in clinical practice will enable risk-based care and optimize outcomes in the CKD population. Moreover, it can also identify groups at high risk of progression, inform public health decisions, and enhance the cost-effectiveness of care for CKD patients.

6. Conclusions

In this Colombian cohort, the four-variable KFRE with a two-year prediction horizon has excellent calibration and discrimination, and its use in the care of CKD Colombian patients is recommended.

Data Availability

The security of the database is consistent with the confidentiality requirements that protect patient privacy as part of the study protocol. Data availability is restricted to ensure patient privacy. The full study protocol and database are available upon request from the principal investigator (carolina_larrarte@baxter.com).

Disclosure

The funder had no role in the study design, data collection, analysis, reporting, or the decision to submit for publication.

Conflicts of Interest

Dr. Larrarte and Ms. Vesga are employees of Baxter Renal Care Services Bucaramanga, Colombia; Drs. Ardila and Sanabria are employees of Baxter Renal Care Services-Latin America, Bogotá DC, Colombia, and Ms. Aldana and Perea are employees of Baxter Renal Care Services Colombia, Bogotá DC, Colombia.

Authors’ Contributions

Dra. Larrarte conceptualized the study and original design of the research and interpreted data for the project. Ms. Vesga was responsible for original conception and design of the research project, data acquisition, statistical analysis, and data interpretation. Dr. Ardila conceptualized the study and original research and interpreted the data project design. Dr. Sanabria conceptualized the study and original design of the research and interpreted data for the project. Ms. Aldana conceptualized the study and original design of the research project, data acquisition, and data interpretation. Ms. Perea was responsible for original conception and design of the research and interpreted the data for the project. All authors have participated in writing the manuscript or in critical review and have provided final approval of the version to be published. All authors verify that they have met all journal authorship requirements. All authors agree to be responsible for all aspects of the work and ensure the publication’s accuracy and integrity. All authors have approved the final version of the manuscript submitted for publication.

Acknowledgments

The authors thank all patients and clinical teams who participated in the study. Baxter Renal Care Services-Colombia, an independent entity of Baxter International, Inc., conducted all activities related to this study. This study was supported by Baxter International, Inc.

References

B. M. Brenner, “Retarding the progression of renal disease,” Kidney International, vol. 64, no. 1, pp. 370–378, 2003.
View at: Publisher Site | Google Scholar
A. S. Levey and J. Coresh, “Chronic kidney disease,” The Lancet, vol. 379, no. 9811, pp. 165–180, 2012.
View at: Publisher Site | Google Scholar
R. C. Harris and E. G. Neilson, “Toward a unified theory of renal progression,” Annual Review of Medicine, vol. 57, no. 1, pp. 365–380, 2006.
View at: Publisher Site | Google Scholar
Y. Liu, “Cellular and molecular mechanisms of renal fibrosis,” Nature Reviews Nephrology, vol. 7, no. 12, pp. 684–696, 2011.
View at: Publisher Site | Google Scholar
A. A. Pagels, B. K. Söderkvist, C. Medin, B. Hylander, and S. Heiwe, “Health-related quality of life in different stages of chronic kidney disease and at initiation of dialysis treatment,” Health and Quality of Life Outcomes, vol. 10, no. 1, p. 71, 2012.
View at: Publisher Site | Google Scholar
Chronic Kidney Disease Prognosis Consortium, K. Matsushita, M. van der Velde et al., “Association of estimated glomerular filtration rate and albuminuria with all-cause and cardiovascular mortality in general population cohorts: a collaborative meta-analysis,” Lancet, vol. 375, no. 9731, pp. 2073–2081, 2010.
View at: Publisher Site | Google Scholar
P. E. Stevens and A. Levin, “Kidney disease: evaluation and management of chronic kidney disease: synopsis of the kidney disease: improving global outcomes 2012 clinical practice guideline,” Annals of Internal Medicine, vol. 158, no. 11, pp. 825–830, 2013.
View at: Publisher Site | Google Scholar
M. T. Yan, C. T. Chao, and S. H. Lin, “Chronic kidney disease: strategies to retard progression,” International Journal of Molecular Sciences, vol. 22, no. 18, Article ID 10084, 2021.
View at: Publisher Site | Google Scholar
B. Hu, C. Gadegbeku, M. S. Lipkowitz et al., “Kidney function can improve in patients with hypertensive CKD,” Journal of the American Society of Nephrology, vol. 23, no. 4, pp. 706–713, 2012.
View at: Publisher Site | Google Scholar
T. Sikaneta, M. Abdolell, H. Taskapan et al., “Variability in CKD stage in outpatients followed in two large renal clinics,” International Urology and Nephrology, vol. 44, no. 5, pp. 1461–1466, 2012.
View at: Publisher Site | Google Scholar
D. Lee, A. Levin, S. D. Roger, and L. P. McMahon, “Longitudinal analysis of performance of estimated glomerular filtration rate as renal function declines in chronic kidney disease,” Nephrology Dialysis Transplantation, vol. 24, no. 1, pp. 109–116, 2008.
View at: Publisher Site | Google Scholar
B. O. Eriksen and O. C. Ingebretsen, “The progression of chronic kidney disease: a 10-year population-based study of the effects of gender and age,” Kidney International, vol. 69, no. 2, pp. 375–382, 2006.
View at: Publisher Site | Google Scholar
D. S. Keith, G. A. Nichols, C. M. Gullion, J. B. Brown, and D. H. Smith, “Longitudinal follow-up and outcomes among a population with chronic kidney disease in a large managed care organization,” Archives of Internal Medicine, vol. 164, no. 6, pp. 659–663, 2004.
View at: Publisher Site | Google Scholar
J. I. Vesga, E. Cepeda, C. E. Pardo, S. Páez, R. Sanchez, and R. M. Sanabria, “Chronic kidney disease progression and transition probabilities in a large preventive cohort in Colombia,” International Journal of Nephrology, vol. 2021, Article ID 8866446, 9 pages, 2021.
View at: Publisher Site | Google Scholar
K. G. M. Moons, A. P. Kengne, M. Woodward et al., “Risk prediction models: I. Development, internal validation, and assessing the incremental value of a new (bio)marker,” Heart, vol. 98, no. 9, pp. 683–690, 2012.
View at: Publisher Site | Google Scholar
K. G. M. Moons, D. G. Altman, Y. Vergouwe, and P. Royston, “Prognosis and prognostic research: application and impact of prognostic models in clinical practice,” British Medical Journal, vol. 338, no. 2, pp. b606–b1490, 2009.
View at: Publisher Site | Google Scholar
K. G. M. Moons, A. P. Kengne, D. E. Grobbee et al., “Risk prediction models: II. External validation, model updating, and impact assessment,” Heart, vol. 98, no. 9, pp. 691–698, 2012.
View at: Publisher Site | Google Scholar
D. B. Toll, K. J. M. Janssen, Y. Vergouwe, and K. Moons, “Validation, updating and impact of clinical prediction rules: a review,” Journal of Clinical Epidemiology, vol. 61, no. 11, pp. 1085–1094, 2008.
View at: Publisher Site | Google Scholar
G. S. Collins, O. Omar, M. Shanyinde, and L. M. Yu, “A systematic review finds prediction models for chronic kidney disease were poorly reported and often developed using inappropriate methods,” Journal of Clinical Epidemiology, vol. 66, no. 3, pp. 268–277, 2013.
View at: Publisher Site | Google Scholar
N. Tangri, L. A. Stevens, J. Griffith et al., “A predictive model for progression of chronic kidney disease to kidney failure,” The Journal of the American Medical Association, vol. 305, no. 15, pp. 1553–1559, 2011.
View at: Publisher Site | Google Scholar
N. Tangri, M. E. Grams, A. S. Levey et al., “Multinational assessment of accuracy of equations for predicting risk of kidney failure. A meta-analysis,” The Journal of the American Medical Association, vol. 315, no. 2, pp. 164–174, 2016.
View at: Publisher Site | Google Scholar
A. Rico-Landazabal, D. Perea, O. Garizabalo et al., “Programa de prevención de la enfermedad renal crónica basado en redes integradas de servicios en Colombia,” Revista de Salud Pública, vol. 19, no. 2, pp. 171–176, 2017.
View at: Publisher Site | Google Scholar
M. Sanabria, D. Espinosa, L. A. Quintero et al., “Chronic kidney disease progression in a large prevention program in Colombia: a cohort study,” Nefrología Latinoamericana, vol. 18, no. 1, pp. 41–49, 2021.
View at: Publisher Site | Google Scholar
J. Bravo-Zúñiga, R. Chávez-Gómez, and P. Soto-Becerra, “Multicentre external validation of the prognostic model kidney failure risk equation in patients with CKD stages 3 and 4 in Peru: a retrospective cohort study,” British Medical Journal Open, vol. 14, no. 1, Article ID 76217, 2024.
View at: Publisher Site | Google Scholar
A. S. Levey, L. A. Stevens, C. H. Schmid et al., “A new equation to estimate glomerular filtration rate,” Annals of Internal Medicine, vol. 150, no. 9, pp. 604–612, 2009.
View at: Publisher Site | Google Scholar
N. Tangri, T. Ferguson, and P. Komenda, “Pro: risk scores for chronic kidney disease progression are robust, powerful, and ready for implementation,” Nephrology Dialysis Transplantation, vol. 32, no. 5, pp. 748–751, 2017.
View at: Publisher Site | Google Scholar
P. Ravani, M. Fiocco, P. Liu et al., “Influence of mortality on estimating the risk of kidney failure in people with stage 4 CKD,” Journal of the American Society of Nephrology, vol. 30, no. 11, pp. 2219–2227, 2019.
View at: Publisher Site | Google Scholar
C. L. Ramspek, M. Evans, C. Wanner et al., “Kidney failure prediction models: a comprehensive external validation study in patients with advanced CKD,” Journal of the American Society of Nephrology, vol. 32, no. 5, pp. 1174–1186, 2021.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2024 C. Larrarte et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

165

Downloads

197

Citations