Abstract
In this paper, the Ridge Regression method is employed to estimate the shape parameter of the Lomax distribution (LD). In addition to that, the approaches of both classical and Bayesian are considered with several loss functions as a squared error (SELF), Linear Exponential (LLF), and Composite Linear Exponential (CLLF). As far as Bayesian estimators are concerned, informative and noninformative priors are used to estimate the shape parameter. To examine the performance of the Ridge Regression method, we compared it with classical estimators which included Maximum Likelihood, Ordinary Least Squares, Uniformly Minimum Variance Unbiased Estimator, and Median Method as well as Bayesian estimators. Monte Carlo simulation compares these estimators with respect to the Mean Square Error criteria (MSE’s). The result of the simulation mentioned that the Ridge Regression method is promising and can be used in a real environment. where it revealed better performance the than Ordinary Least Squares method for estimating shape parameter.
1. Introduction
Ridge Regression is a popular parameter estimation method for analyzing multiple regression data which has multicollinearity. When Multicollinearity happens, Least squares estimates are unbiased, but their variances are large. As a result, the estimator of the Ordinary Least Squares Method (OLS) becomes far from the true value. Ridge Regression decreases the standard errors when a degree of bias is added to the Regression Estimates. Several authors have addressed Ridge Regression as in [1–10].
The Lomax distribution (LD) is a proposed distribution of the Pareto distribution of type II; it was used to obtain a good model for biomedical problems. Also, it is an important model for modeling failure times. The Lomax distribution was used as a stochastic model with a decreasing failure rate for the operating times of the electronic vehicles under study. It was also used in studies related to income and studies related to the size of cities. As well as being a useful model in studying queuing theory and in analyzing data related to biostatistics.
Many theoretical and statistician’s studies have given great interest to estimating the parameter and survival analysis of LD.
Al-Noor and Alwan [11] compared the Nonbayesian, Bayesian, and Empirical Bayes estimate for the parameters of the LD by considering the symmetric and asymmetric loss functions. Al-Duais and Hmood [12] compared the Bayesian estimators and Classical estimators to estimate the parameter and survival analysis depending on record values and by considering the SELF, LLF, and WLLF. Ellah [13] estimated the parameter, reliability, and hazard function of the LD by applying the Bayesian estimators and Classical estimators based on record values by considering the SELF and LLF. Asl et al. [14] applied the Bayesian estimators and Classical estimators of prediction on unidentified parameters of an LD based on a progressively type-I hybrid censoring scheme. Mohie El-Din et al. [15] studied the Classical estimations and Bayesian estimations of the LD based on progressively type-II censored samples and by considering symmetric (SELF) and asymmetric (LLF and GELF), Okasha [16] estimated the parameters, and survival analysis of the LD by applying the E-Bayesian estimators and Bayesian estimators under type-II censored data and by regarding the balanced squared error loss function. Liu and Zhang [17] studied the Bayesian and E-Bayesian estimations of the LD Based on the Generalized Type-I Hybrid Censoring Scheme. to estimate the unknown parameter of LD and by considering SELF and LLF to estimate the parameter and reliability function. Al-Bossly [18] developed a compound LINEX loss function (CLLF) to estimate the shape parameter of the Lomax distribution utilizing the E-Bayes and Bayes estimation methods for the distributional parameters of the LD.
In the current study, Ridge regression was employed to estimate the shape parameter of LD and compare it with the classical estimators which included Maximum Likelihood, Ordinary Least Squares, Uniformly Minimum Variance Unbiased Estimator, and Median Method as well as Bayesian estimators. The uniqueness of this work comes from the fact that, to date, no attempt has been made to estimate the shape parameter of the LD using the method of Ridge regression.
The pdf of LD is given as follows [19]:where is a random variable, and , are the scale and shape parameters, respectively.
The CDF and reliability function of (1) are given by the following equation:
2. Classical Methods of Estimation of Lomax Shape Parameter
The Classical methods selected for the comparative study are (i) Maximum Likelihood Estimator (MLE), (ii) Ordinary Least Squares Method (OLS), (iii) Ridge Regression method, (iv) Uniformly Minimum Variance Unbiased Estimator (UMVUE), and (v) Median Method (M.M).
2.1. MLE’s of the Shape Parameter
Suppose that is a random sample from the LD as in (1), then the for the sample observation will be as follows:where .
Log likelihood function
The MLE’s of denoted by is given as follows:
2.2. Ordinary Least Squares Method (OLS)
The CDF in equation (2) satisfies
Now, suppose that form a random sample from LD defined by (1), and that are the order statistics. With observed ordered observations (2) gives the following equation:
(8) represents a simple linear regression function corresponding to where and i it is a point estimator of many estimators for are used.
For example, the Median Rank estimator or , the mean rank estimator . where denotes the smallest value of , . is the random error with expected value . , , .
The estimates and of the regression parameters, and minimize the function,
Therefore, the estimates of the parameter, is given by the following equation:
The estimate of the parameter is given by the following equation:
2.3. Ridge Regression Method
Ridge Regression estimates can be obtained by minimizing the function.
According to the following constraintwhere is a definite positive constant.
The Lagrange multiples method requires that we derive the following:
Therefore, the estimates and of the parameters, and are given by the following equation:andwhere represents the number of parameter of the distribution and represents the covariance matrix.
NOT when , we get the estimations of the OLS.
The Ridge Regression estimate of denoted by is given as follows:
2.4. UMVUE Estimator of the Shape Parameter
The pdf of LD belongs to the exponential family. Therefore, is a complete sufficient statistic for . Then, depending on the theorem of Lehmann-Scheffe [20], the uniformly minimum variance unbiased estimator of , may be given by the following equation:
2.5. Median Method (M.M)
This method is dependent on the basis that the median divides the data into two equal parts
By substituting into the cumulative distribution function defined by (2). The equation will become
Therefore, the estimates of , can be obtained as follows:where is the median of the data.
3. Prior and Posterior Density Functions
3.1. Prior Distribution
The Bays estimators demand an appropriate selection of priors for the parameter. If we do not have sufficient knowledge about the parameter, in this case, the noninformative priors are better chosen. Or else, it is desirable to use informative priors. In this research, we study both types of priors: informative priors and noninformative priors.
3.1.1. Non-Informative Prior
Let us assume that has noninformative prior density defined as using extended Jeffrey's prior which is given by the following equation:where represented Fisher information matrix which defined as follows:
3.1.2. Informative Priors (The Natural Conjugate Prior)
In this work, three types of prior distributions were used to study the effect of the different prior distributions on a Bayesian estimate of ϑ.(a)Chi-squared prior(b)Inverted levy prior(c)Gamma Prior
3.2. Posterior Density Functions
The posterior distribution for the shape parameter can be expressed as follows:
Combining the in (4) and the prior distribution of extended Jeffrey’s prior (16), chi-square prior (17), inverted Levy prior (18), and gamma prior (19). The posterior density of It can be found on respectively as follows:
4. Loss Functions
In Bayes estimation, we will consider three types of loss functions including SELL, LLF, and CLLF.
4.1. Squared Error Loss Function
The SELF is defined as follows [21]:
The Bayes estimator of relative to SELF, signified by is
4.2. LINEX Loss Function
The LINEX loss function for can be written as follows [22, 23]:
The Bayes estimator of relative to LLF, denoted by is
Provided that exists and is finite, where denotes the expected value.
4.3. Composite LINEX Loss Function
CLLF is given by the following formula [24].
The Bayes estimator of relative to CLLF, denoted by , is
Provided that and exist and are finite.
5. Bayes Estimator
In this part, we estimate , using three various loss functions, including SELF, LLF, and CLLF. We assume four different prior distributions for including; extended Jeffrey's prior, chi-square prior, inverted Levy prior, and gamma prior [25–28].
5.1. Bayesian Estimator of under SELF
The Bayes estimates of relative to SELF depended on which is signified as and can be acquired by using equations (21) and (26) to be
Likewise, we can obtain the Bayesian estimates of relative to SELF depending on , , and , which are signified as , , and by using equations ((22) and (26)), ((23) and (26)) and ((24) and (26)), respectively, to beand
5.2. Bayesian Estimator of under LLF
We can obtain the Bayes estimator of under the LLF depending on signified as by using equations (21) and (28) as follows:
In the same way, the Bayes estimates of relative to LLF depended on , , and , which are signified as , , and by using equations ((22) and (28)) ((23) and (28)), and ((24) and (28)), respectively, to beand
5.3. Bayesian Estimation of under CLLF
The Bayes estimate of under CLLF depended on , which is signified as by using equations (21) and (30) to bewhere:and
Therefore, the Bayes estimation of parameter is
Similarly, the Bayesian estimates of under CLLF depended on , , and , which are signified as , , and by using equations ((22) and (30)), ((23) and (30)), and ((24) and (30)), respectively, to bewhereand
Soandwhereand
Sowhereand
Therefore, the Bayes estimation of parameter is
6. Simulation Study and Results
In this part, a simulation study has been conducted to assess and examine the behavior of Classical methods and Bayes estimators for the shape parameter of LD under different cases. The following steps of the simulation are as follows:(1)Set the true values for the parameters of LD which are varied into three cases to observe their effect on the estimates when , , and “case I , case II and case III ”(2)Determine the sample size and (3)Determine the value = 0.75, , and and (4)For a given sample size , generate by using the following formula: , , where is uniform (5)Classical methods estimation, , , , , and of are computed from equations (6), (10), (13)–(15), respectively(6)Under SELL and based on and priors, Bayesian estimation, , , and , of are computed from equations (31), (32), (33), and (34), respectively(7)Under LLF and based on and priors, Bayesian estimation, , , and , of are computed from equations (35), (36), (37), and (38), respectively(8)Under CLLF and based on and priors, , , and , of are calculated from equations (40), (42), (44), and (46), respectively(9)Steps 4 to 8 are replicated 10,000 times. The (MSE’s) for all Estimates of the parameter are obtained, where
The results are displayed in the following Tables 1–5.
7. Conclusions and Recommendations
In this paper, the Ridge Regression method was employed to estimate the shape parameter of LD. Besides, researchers made a Monte Carlo simulation to test the performance of the Ridge Regression method. Then, compared the Ridge Regression estimator with the other estimators, including MLE, OLS, UMVUE, M.M, and Bayesian estimators based on SELF, LLF, and CLLF. However, the major observations are identified in the following points:(1)Among classical estimators, in Table 1, the performance of the UMVUE was shown as better than other estimators: “MLE, OLS, Ridge, and M.M estimators” in all different cases and all samples sizes. Whereas the performance of Ridge Regression was better than MLE. Estimator especially for a small sample size . In the meanwhile, the results showed that the performance of the Ridge estimator was better than OLS estimator in all different cases and sample sizes.(2)With Bayes estimators, gamma prior records full appearance as best prior based on LLF and CLLF for all different cases and all sample sizes. As well as that is true under SELF with and , while extended Jeffrey’s prior record as best prior based on SELF when .(3)The MSE values associated with each of the classical and Bayes estimate “corresponding to each prior and every loss function” reduces with the increase in the sample size. Also, the results show a convergence between most of the estimators to increase the sample sizes and this conforms to the statistical theory.(4)For all cases and all sample sizes, LLF records full appearance as the best loss function associated with Bayes estimates corresponding to gamma prior.(5)According to the results, MSE values of all classical and Bayes estimators of shape parameters are decreasing as the shape parameter value increase.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
Their authors extend their appreciation to the Deanship of Scientific Research at King Khalid University for funding this work through Larg Groups. (Project under grant number (RGP.2/4/43).