Abstract
This research aims to compare estimating the confidence intervals of variance based on the normal distribution with the primary method and the Bayesian approach. The maximum likelihood is the well-known method to approximate variance, and the Chi-squared distribution performs the confidence interval. The central Bayesian approach forms the posterior distribution that makes the variance estimator, which depends on the probability and prior distributions. Most introductory prior information looks for the availability of the prior distribution, informative prior distribution, and noninformative prior distribution. The gamma, Chi-squared, and exponential distributions are defined in the prior distribution. The informative prior distribution uses the Markov Chain Monte Carlo (MCMC) method to draw the random sample from the posterior distribution. The Fisher information performs the Wald confidence interval as the noninformative prior distribution. The interval estimation of the Bayesian approach is obtained from the central limit theorem. The performance of these methods considers the coverage probability and minimum value of the average width. The Monte Carlo process simulates the data from a normal distribution with the true parameter of mean and several variances and the sample sizes. The R program generates the simulated data repeated 10,000 times in each situation. The results showed that the maximum likelihood method employed on the small sample sizes. The best confidence interval estimation was when sample sizes increased the Bayesian approach with an available prior distribution. Overall, the Wald confidence interval tended to outperform the large sample sizes. For application in real data, we expressed the reported airborne particulate matter of 2.5 in Bangkok, Thailand. We used the 10–1000 records to estimate the confidence interval of variance and evaluated the interval width. The results are similar to those of the simulation study.
1. Introduction
Statistical inference is the process of using detailed statistics on the observation data set (i.e., the mean and the variance), called a statistic, that refer to the properties of the parameter with the population. It assumes that the observed data set is a random sample from a large population. Parameter estimation and hypothesis testing are part of statistical inference related to a statistic and the parameter.
The parameter estimation consists of point and interval estimations. Point estimation contains a single value for estimating the population parameter or called an estimator by using the sampled data, a single number. Some error is associated with the point estimator that may be smaller or larger than the true parameter. Instead of a point estimate, confidence interval estimation uses the sampled data to approximate the range of lower and upper bounds of the population parameter. The desire of confidence interval is computed by the mean of the estimators from point estimation plus and minus the variation in that estimate based on the confidence level.
The normal distribution is a continuous probability distribution consisting of parameters, namely mean and variance, which is often used in the natural and social sciences. One important parameter of the normal distribution is the variance that measures the dispersion of the data set around its mean. In statistics, variance can be computed by taking the sum of the squared differences dispersed around the mean and dividing by the total sample sizes and then squaring the differences to make them positive.
Several tools are available for estimated variance using sample data: the moments method, maximum likelihood method, least-squares method, and Bayesian method. The maximum likelihood method is a primary method for estimating the parameter of probability distribution function given observation. The estimator is approximated by maximizing the likelihood function given the parameter. The maximum likelihood estimator is an interesting method because most estimators present in a class of a minimum variance unbiased estimator [1]. Then, the variance estimator from the maximum likelihood method is used to create the interval estimation from Chi-squared statistics [2]. Smithpreecha and Niwitpong [3] presented four approaches to construct confidence intervals for the common variance of normal distributions and compared the results based on the generalized confidence intervals. Iliopoulos and Kourouklis [4] constructed a confidence interval for the generalized variance of a matrix normal distribution with unknown mean and improved on the usual minimum size.
The Bayesian approach method mentions the probability distribution given the parameter related to the prior distribution. Resolution of the prior distribution impacts obtaining the posterior distribution [5]. The case of the prior distribution is defined as the available prior distribution, informative prior distribution, and noninformative prior distribution. The posterior distribution is estimated by the available prior information shown in the appropriate distribution. So, the Bayesian estimator is obtained from the mean of the posterior distribution. Severini [6] studied the relationship between Bayesian and non-Bayesian confidence intervals by deriving asymptotic extensions of the posterior probability for computing the confidence regions based on the likelihood ratio test statistics. Next, Severini [7] was interested in creating an interval estimate for the population parameter and constructed the posterior probability such that the parameter lays in the interval in some specified value. Ali and Riaz [8] studied the Bayesian methodology for designing Bayesian control charts and noticed that the performance of the Bayesian charts is biased.
The computer algorithm has been developed to draw a random sample from the posterior distribution in an informative prior distribution [9]. Most used the Markov Chain Monte Carlo (MCMC) [10] method as an informative prior distribution because it can draw an approximate sample from the prior distribution. Then, Gibbs sampling [11] is a process of MCMC to generate the sample values from the posterior distribution. The conjugate prior is the same family as the prior and posterior distributions. The construction of the posterior distribution is in a closed form of conjugate distribution that is made to approximate the MCMC estimator. Atchadé [12] constructed the confidence interval for the asymptotic variance and proposed that the confidence interval converges to standard central limit theorems for the Markov chains. Mahmoud et al. [13] proposed Markov Chain Monte Carlo techniques to compute the maximum likelihood estimation and the confidence intervals of coefficient of variation.
The approximation of the confidence interval of variance [14] is studied by using the sample sum of squared deviations from the mean. The confidence interval of minimum length is raised concerning the shortest unbiased interval for the variance of a normal distribution. Typically, the estimated confidence interval of variance is used in the Chi-squared distribution to create the lower and upper bounds. Their importance is partly due to the central limit theorem. Rajić and Stanojević [15] considered the confidence intervals for the ratio of two variances. The F-statistic and central limit theorem proposed the confidence interval. The central limit theorem proposes the Bayesian of the available and informative prior distributions, which performs the confidence interval using the mean and variance from the posterior distribution. Abu-Shawiesh et al. [16] approximated asymptotic confidence interval for the population of standard deviation based on the sample Gini mean difference.
Another problem with the prior information is that if the parameter of the prior distribution is entirely appropriate, it should be incorporated into the posterior distribution. This is not a severe problem since the noninformative prior distribution leads to discussion. Kass and Wasserman [17] stated two different interpretations of noninformative priors. The ignorance of noninformative priors was formal representations, and the prior distribution was defined as the constant value when there is insufficient information. Andrés and Álvarez Hernández [18] evaluated the confidence interval on noninformative prior distribution from Jeffrey’s Bayesian method. The Wald method was determined to create the confidence interval of a binomial proportion. Agresti and Coull [19] suggested the Wald confidence interval based on inverting the binomial test.
We set out to compare the confidence interval of variance using the maximum likelihood method. The Bayesian approach depended on the available prior distribution, informative prior distribution, and noninformative prior distribution. The data are simulated from the normal distribution with varying true parameters and sample sizes.
2. Estimated Methods
The variance of the normal distribution is estimated by the maximum likelihood method and the Bayesian approach as the point estimation. Then, the confidence intervals are computed from the point estimation to construct the lower and upper bounds depending on the significance level. The central limit theorem is the main point in creating the confidence interval of the Bayesian approach.
2.1. Maximum Likelihood Method
The maximum likelihood method is the most popular method to approximate a parameter on several distributions. The concept of the maximum likelihood method is to start from the likelihood function of a random variable . Let be independent and identically distributed (iid) random variables via normal distribution with parameter and ; hence, the probability density function is written by
And it is defined as .
The likelihood function has used the multiplication of probability function as follows:
From (2), take ln on likelihood function following
From (3), differential with respect to parameter ,
We obtain the , so this estimator is a biased estimator, but it is a sufficient and consistent estimator. The performance estimator has supported this property such as unbiased estimator, minimum of variance, consistency, and sufficiency. This estimator is adjusted from the maximum likelihood estimator and defined the property as . We know that is an unbiased estimator, and we get the unbiased estimator in terms of following .
The confidence interval at of is approximated from by using the concept of Chi-squared distribution at degree of freedom [2] following ; then it can be rewritten as
The code for the maximum likelihood method in R programming language is as follows (the example of 90% confidence interval) > MLE = var(x) > l90_percen=(n − 1)/qchisq(1–0.1/2,n − 1) > u90_percen=(n − 1)/qchisq(0.1/2,n − 1) > LCL90_MLE = MLE l90_percen > UCL90_MLE = MLE u90_percen
2.2. Bayesian Approach with Available Prior Distribution
From the maximum likelihood method, we know that the probability density function of random variables is a normal distribution as The estimation of the Bayesian approach is considered by adjusting the parameter of normal distribution by , where and is a constant value; then, the probability distribution function from (1) can be rewritten by
From (2), the likelihood function is changed parameter by
The prior distribution of is considered in form of a gamma distribution with parameter denoted by or rewritten as
The resulting posterior distribution is a gamma distribution that is implied by the conjugate distribution from normal distribution on (6) and gamma distribution on (8) as
Recognizing that this result is displayed in the form of gamma distribution:where and . is approximated by using ; then, it can be rewritten as [5].
With the parameterization, the posterior distribution of is , where is called the inverse gamma distribution with parameters and .
The inverse gamma distribution [20] is the same distribution as the reciprocal of a gamma distribution and the relationship of , is defined to the inverse gamma distribution as . The inverse gamma distribution in the form of random variable is written as
The mean of inverse gamma distribution is , and the variance is .
A similar way to estimate the variance of the Bayesian method is defined asand the variance of Bayesian estimator is calculated as
To evaluate our proposed method, the prior distribution of is proposed in the form of gamma distribution with parameters and or denoted by . Then, the parameter of gamma distribution is an available prior distribution as the gamma, Chi-squared, and exponential distributions by , , and , shown in Figure 1.
From central limit theorem [15] with , it can approximate in the form of standard normal distribution by
This is an asymptotic confidence interval that will only give an approximation as the large sample size, which has better coverage rates for small samples [21].
The definition of the confidence interval can be expressed as
Putting these together, (14) and (15), it is often written asThen,
The confidence interval at of is approximated bywhere and .
The code of R programming language for the Bayesian approach with available prior distribution is set as the constant values of prior distribution as follows (the example of 90% confidence interval). > lamma_bayes.g = 2; alpha_bayes.g = 1 > lamma_bayes.c = 2; alpha_bayes.c = 0.5 > lamma_bayes.e = 1; alpha_bayes.e = 0.2 > B.g = lamma_bayes.g + var(x) (n − 1)/2); A.g = alpha_bayes.g + (n/2) > B.c = lamma_bayes.c + (var(x) (n − 1)/2); A.c = alpha_bayes.c + (n/2) > B.e = lamma_bayes.e + (var(x) (n − 1)/2); A.e = alpha_bayes.e + (n/2) > Bayes.g = B.g/(A.g − 1); Bayes.c = B.c/(A.c − 1); Bayes.e =B.e/(A.e − 1) > SE_Bayes.g = sqrt((B.g ^ 2)/(((A.g − 1)^2)∗(A.g − 2))) > SE_Bayes.c = sqrt((B.c ^ 2)/(((A.c − 1)^2)∗(A.c − 2))) > SE_Bayes.e = sqrt((B.e ^ 2)/(((A.e − 1) ^ 2)∗(A.e − 2))) > LCL90_Bayes.g = Bayes.g − (qnorm90_percen SE_Bayes.g) > LCL90_Bayes.c = Bayes.c − (qnorm90_percen SE_Bayes.c) > LCL90_Bayes.e = Bayes.e − (qnorm90_percen SE_Bayes.e) > UCL90_Bayes.g = Bayes.g + (qnorm90_percen SE_Bayes.g) > UCL90_Bayes.c = Bayes.c + (qnorm90_percen SE_Bayes.c) > UCL90_Bayes.e = Bayes.e + (qnorm90_percen SE_Bayes.e)
2.3. Bayesian Approach with Informative Prior
Evaluation of the suitable value on the prior distribution is an important problem because of the difficulty of estimating parameters via the posterior distribution. The solution problem of available prior information has been developed using a sampling algorithm, such as this Markov Chain Monte Carlo (MCMC) method or called the informative prior. This algorithm of the MCMC method is to draw a random sample from the posterior distribution without the conveniently estimated prior distribution. The process is employed by sampling parameters from the Markov Chain method for estimating parameters on prior distribution by the Gibbs sampling algorithm [10]. Then, the posterior distribution uses a Markov Chain and Gibbs sampling to approximate parameters from the MCMC method. We carry out the rjags package, which provides an interface from R [22] to rjags library to generate a sequence of dependent samples from the posterior distribution. Sampling process from the MCMC method is as follows:(1)Draw from the normal distribution with parameter mean and variance .(2)The parameter is generated from the normal distribution and the is generated from the inverse gamma distribution with parameters and .(3)Set initial value from an exponential distribution and from the gamma distribution with constant values.(4)Generate from the posterior distribution based on the inverse gamma distribution with parameters and for , where is the number of Gibbs sampling algorithm.(5)Calculate the mean and standard deviation from the posterior distribution function.
For each chain, the first 2000 iterations were discarded, and the last 5000 iterations were used to obtain the posterior distribution of the parameter.
Thus, the MCMC estimator is .
The confidence interval is created by computing the mean and the variance from the MCMC process as
According to the central limit theorem with , we haveand .
Therefore,Then,
We say that the confidence interval at of by MCMC at is
The “rjags” package in the software R is used to estimate the MCMC estimator and confidence interval following (the example of 90% confidence interval): > library(rjags) > dataset = list(x = x, n = n) > inits = list(mu = 0.01, tau = 0.01) > jagmod < −jags.model(“model_normal.txt,” data = dataset, inits = inits,n.chains = 1, n.adapt = 2000) > update(jagmod, n.iter = 20, progress.bar = “text”) > posterior = coda.samples(jagmod, c(“mu,” “sigma”),n.iter = 5000, progress.bar = “text,” thin = 2) > post = as.data.frame(as.matrix(posterior)) > var.mcmc = (post$sigma) ^ 2 > mcmc.est = mean(var.mcmc) > SE_MCMC = sd(var.mcmc) > LCL90_mcmc.est = mcmc.est − (qnorm90_percen SE_MCMC) > UCL90_mcmc.est = mcmc.est + (qnorm90_percenv SE_MCMC)
2.4. Bayesian Approach with Noninformative Prior
The Bayesian theorem is dependent on the likelihood function and the prior distribution, which is used to calculate the posterior distribution. If the prior distribution defines the prior information, it will approximate the estimator on the posterior distribution. If we have no information on prior distribution, we call it a noninformative prior. Agresti and Coull [19] studied the approximation of the interval estimation of binomial proportions based on inverting the Wald large-sample normal test. Then, we used the Wald confidence intervals to construct the confidence interval of variance. The Fisher information in a single observation is needed as
From (4) and (24), the Fisher information is defined as
The asymptotic Wald confidence interval based on the maximum likelihood estimator tends to the central limit theorem asand .
We infer that in large sample,
The Wald confidence interval at of at is
The motivation of noninformative prior is that the fisher information indicates the amount of information by the observation about the parameter. The code for the Bayesian approach with noninformative prior based on Wald method in R programming language is as follows (the example of 90% confidence interval). > LCL90_Wald = MLE − (qnorm90_percen sqrt(2/n) MLE) > UCL90_Wald = MLE + (qnorm90_percen sqrt(2/n) MLE).
3. Simulation Study
The details of simulated data and the results are stated in this section. The random variable is generated from a normal distribution with the R program by setting the true parameter as mean at 2 and variance at 2, 6, and 12 as shown in Figure 2.
The sample sizes () are studied at 10 and 30 for small sample sizes, 50 and 100 for moderate sample sizes, and 500 and 1000 for large sample sizes. The estimated confidence intervals are obtained from the maximum likelihood method and Bayesian approach at the confidence interval level 90%, 95%, and 99%. Suppose the estimated confidence intervals cover the true parameter or population variance as 2, 6, and 12. We will count the number and compute the proportion denoted by the coverage probability (CP). The coverage probability is compared with the fixed confidence interval that we define the significance level at 0.05 ( = 1.96). If the fixed confidence interval range can cover the coverage probability, we will perform these methods and consider the minimum of the average width. The average width of a confidence interval is evaluated by computing the average of the difference values between the upper limit and lower limit. Hence, the coverage probability covers the range of the fixed confidence interval by computing
In this case, we define = 0.9, 0.95, and 0.99 that followed the level of confidence interval 90%, 95%, and 99%. The range of the fixed confidence interval is shown in Table 1.
The R program generates data and estimates confidence intervals at 10,000 replications in each situation. The estimating confidence intervals of variance by Maximum Likelihood (ML), Bayesian approach with available prior distribution (gamma, Chi-squared, and exponential distributions), informative prior distribution (MCMC method), and noninformative prior distribution (Wald confidence interval) in Tables 2–4. The first column shows the true variance parameter, and the second column demonstrates the several sample sizes. The coverage probability (CP) and average width (AW) are presented in the following twelve columns for these methods. The minimizing AW values illustrate the performance of these methods in the form of underline AW values. However, some AW values are shown blank because the coverage probability is not in the range of the fixed confidence interval from Table 1. By observing the CP and AW, the results appear as follows.
Simulation results in Table 2 show that the maximum likelihood (ML) method has a reasonable coverage probability for small sample sizes. Nevertheless, the Bayesian approach with prior distribution in gamma, chi-squared, and exponential distributions presents the most minimum AW, especially with moderate sample sizes. For large sample sizes, the Wald confidence interval has a good performance. Furthermore, the AW is shown to decrease as sample sizes increase. The CP of all methods is represented in Figure 3.
(a)
(b)
(c)
From Figure 3, the CP values of ML and Wald present the exact value of all variances for each sample size and play in the range of the fixed confidence interval. The CP of Wald tends to increase when sample sizes increase. However, the MCMC method decreases when sample sizes increase. All methods fall in the range following the increasing sample sizes. When the variance increases in Figure 4, the CP is shown the slightly different in large sample sizes.
(a)
(b)
(c)
(d)
(e)
(f)
In Table 3, the ML and Bayesian approach results with available prior distribution are similar to those in Table 2. However, the ML method shows the minimum AW in large variance at n = 300. The trends of CP of all methods are presented in Figure 5.
(a)
(b)
(c)
In Figure 5, the CP values of the Bayesian approach of gamma, Chi-squared, exponential distributions, and Wald trend to increase when the sample sizes also increase. Furthermore, when the variance increases in Figure 6, there is no affectation on the CP.
(a)
(b)
(c)
(d)
(e)
(f)
From Table 4, it is shown that the AW of ML has the minimum at small and moderate sample sizes. When the sample sizes are larger, the Wald has a good performance. The CP of all methods is presented in Figure 7.
(a)
(b)
(c)
Figure 7 shows that the CP values of ML are in the range of the fixed confidence interval. For the large sample sizes and variances, the CP values of all methods are in the range of the fixed confidence interval shown in Figure 8.
(a)
(b)
(c)
(d)
(e)
(f)
4. Application in Real Data
The real data are collected from the airborne particulate matter 2.5 (PM 2.5: mg/m3) from the air monitoring quality station of Bansomdejchaopraya Rajabhat University, Bangkok, Thailand, from April 1, 2019, to 2022. These data were obtained from Thailand’s Pollution Control Department (https://www.aqmthai.com) and used 10–1000 records for computing the confidence interval of variance. The distributions of PM 2.5 were presented in Figure 9, which showed the right skewness for the large sample sizes. Thus, the Shapiro–Wilk test was used to test the normal distribution of such data. From the p-values, it was evident that the sample sizes (10, 50, and 100) confirmed normal distribution except for the large sample sizes (300, 500, and 1000). Moreover, normal Q-Q plots were constructed to show the normal distributions from the six sample sizes (Figure 10), which verified that the small sample sizes followed the normal distribution. Table 5 reported the interval width of 90%, 95%, and 99% confidence interval for the airborne particulate matter 2.5 (PM 2.5: mg/m3).
The results show that the 10, 50, and 100 sample sizes of the maximum likelihood (ML) method had the shortest interval width, which corresponds with the simulation results. For the large sample sizes (500 and 1000), the Wald method attended the shortest interval width, similar to the simulation results. Furthermore, the MCMC method shows the shortest of 300 sample sizes. Therefore, it is a good choice for constructing the confidence interval when the data set supports a nonnormal form.
5. Conclusion
In this research, we have concentrated on estimating the confidence interval of variance via normal distribution by using maximum likelihood and the Bayesian approach. The coverage probability and average width are the criteria for the efficient method. Through a simulation study, the maximum likelihood method performs a reasonable method in small sample sizes following the Chi-squared distribution that creates the confidence interval (see [14]). The Bayesian approach depends on the gamma, Chi-squared, and exponential prior distribution, making a suitable performance method in moderate sample sizes. The Wald confidence is a reasonable working method for the large sample sizes in all cases. In particular, the CP of the Wald interval is such low values for the small sample sizes, but the large sample sizes converge into the range. Surprisingly, the MCMC method is weak even though the estimator obtains from the sampling algorithm. However, approximate results benefit normal distribution because of the inherent evolution of exact distribution. Form real data, the airborne particulate matter 2.5 is collected to estimate the confidence interval. It is clear from the results that the maximum likelihood and Wald methods are the reasonable working methods in small and large sample sizes, same as the simulation data set.
Data Availability
The data used to support the findings of this study are available from the author on request.
Conflicts of Interest
The author declares that there are no conflicts of interest regarding the publication of this paper.