Abstract
This study focuses on deriving coefficients of a simple linear regression model and a quadratic regression model using fractional calculus. The work has proven that there is a smooth connection between fractional operators and classical operators. Moreover, it has also been shown that the least squares method is classically used to obtain coefficients of linear and quadratic models that are viewed as special cases of the more general fractional derivative approach which is proposed.
1. Introduction
Initial concepts of fractional calculus (FC) can be traced back to seventeenth century when Isaac Newton, Leibniz, and L’ Hospital discussed preliminary ideas that shed light to future developments on FC. A considerable time elapsed before mathematicians returned to discuss the idea of FC. In 1819, Lacroix [1] mentioned the derivative of arbitrary order. Euler and Fourier also mentioned the derivatives of an arbitrary order. The first applications are made by Abel [2] in 1823. Ross [3] provided a historical track on the fundamental theory of FC.
The new research trend in FC since last century focuses on the investigation of its application in real life problems. In this regard, researchers have produced thousands of articles from various branches of sciences in which they showed how some problems defined in the classical calculus could be converted into FC problems. Another class of researchers in the fields consists of those who not only defined fractional calculus approach of solving problems but who also managed to show in their works that the FC might be more efficient than its classical counterpart in solving problems. In either case, it is common for researchers to back up their FC finding with similar results from classical calculus.
Fractional differential equations are a part of FC in which a substantial amount of work has been undertaken aiming to prove the power of the method over the classical differential equations. Works carried out along these lines with applications in Biology [4, 5], Physics [6], and Finance [7] are just a few to mention. Some more recent work on FC worth to mention are found, for instance, in [8], where the authors investigated bending of the beam using fractional differential equations. Sumelka et al. [9] reformulated the classical Euler–Bernoulli beam theory using fractional calculus. Stempin and Sumelka [10] studied the bending analysis of nanobeams aiming to improve the space-fractional Euler–Bernoulli beam (s-FEBB) theory. Sidhardh et al. [11]. presented their findings on their studies of the analytical and finite element formulation of a geometrically nonlinear and fractional-order nonlocal model of a Euler–Bernoulli beam. The size-dependent bending behavior of nanobeams is studied by Oskouie et al. [12] employing the Euler–Bernoulli beam theory, where the nonlocal effects are obtained via fractional calculus.
Solving methods of fractional differential equations are not necessarily like classical differential equations. Therefore, researchers prefer to define a fractional model to a problem and compare it with an existing classical model of the same problem. Such approach has drawn our attention, and the investigation of the existence of a smooth transition between FC and classical calculus is undertaken in this study.
In this work, we have chosen to investigate the derivation of the coefficients of simple linear and quadratic regression using FC. It is has become evident that the solution of the fractional approach offers a more general solution. Hence, the classical solution becomes a special case of the fractional solution. The aim of the work is to lay a foundation for the application of fractional calculus in statistics. That is to investigate to which extend fractional calculus can be used directly for estimating parameters of a statistical model. Hence, we mainly focus on proving that the fractional model is well defined for solving the problem.
2. Deriving the Coefficients of Linear and Quadratic Regressions
Classical approaches of linear and quadratic regressions are discussed in this section. Indeed, these are two models of data fitting with many applications in various branches of sciences. In general, these models are used to fit data collected from experiments in which there is a predictor, also known as the independent variable X and a dependent or response variable Y. Dataset to be used should be made of pairs , where n is the sample size. A simple linear model is defined bywhere is the slope and is the y intercept. Equation (1) is written provided that there is a deterministic relationship between the independent variable X and the response variable Y. However, in practice, such relationship does not exists as experimentally collected data are random in nature and contain random errors. A least squares technique is used in minimizing the error when fitting a linear model to available data. Statistically, the simple linear model is a best fit to the dataset. Then, (1) becomeswhere the variable represents the random errors in the process. Moreover, it is assumed that and . Introducing estimated coefficients of and denoting them by and , respectively, the fitted model becomes
Recalling the pairs of experimental data , and denoting those fitted by the model , it follows that the random error can be expressed as
Coefficients and computed using the least squared error method are those that best fit the model given by (1). Given the sum of squared error (SSE) between the observed and the fitted data points is denoted byIt is shown in [13] that the simple optimization of (5) helps to find the optimal value of and as
With a similar approach, the optimal coefficients for a quadratic model are computed. Given an experimental dataset , where its scatter plot exhibits an upward or downward concavity which is an indication that a quadratic model is appropriate. The deterministic quadratic model is defined aswith , otherwise, the model becomes linear.
Since in practice, experimental datasets are to be fitted, and a random error term is added. Hence, (7) becomes
The variable represents the randomness in the process, with and . Letting to be the estimates of , respectively, the fitted model becomes
Recalling the pairs of experimental data , and corresponding fitted model pairs , the random error can be expressed as
Coefficients , , and computed using the least squares error method are those that best fit the model given by (7). The sum of squared errors (SSE) between the observed and the fitted data points is denoted by
Optimal values of the coefficients , , and are easily derived from the following system of 3 equations obtained from (11).
Results obtained in the current section are based on Newtonian calculus. In the next section, analogue results are built using FC.
3. Deriving the Coefficients of Fractional Linear and Quadratic Regressions
This section presents similar results to those obtained in the previous section based on FC. We prefer to define and use FC tools whenever they are needed. We consider the linear model problem defined by (1) following the SSE setting (5). The aim is to use fractional partial fraction derivative to compute optimal coefficients of the fitted model.
Definition 1 (see [3]). The Riemann–Liouville fractional integral of order for a function is defined asProvided that the right-hand side of the integral is point-wise defined on , and is the gamma function.
Definition 2 (see [3]). The Caputo derivative of order for a function is defined aswhere is the integer part of q.
Definition 3. Given a function of two variables f, the fractional partial derivative of order q with respect to one variable is defined as .
Lemma 1. We consider q such that , with . Moreover, let ; the fractional integral is a well-defined number.
Proof of Lemma 1. It is done through integration by parts. Indeed, setting is a good hint to go through to the solution.
Minimization of the SSE defined by (5) is performed in order to obtain the best coefficient for and using fractional derivative. The results are given by some theorems and their proofs are as follows:
Theorem 1. We consider the simple linear model defined by (1), and the best coefficients of the fitted model (5) using fractional derivative are given by and , where q represents the fractional order of the derivative.
Proof of Theorem 1. Fractional derivative is used. In fact, the minimization of the SSE, (5), using the partial fractional derivative is written as follows:implying that . The other coefficient is derived similarly. Indeed, let the partial derivative be computed with respect to . Then,Last equation leads to the following , in which is plugged in to obtain a compact form B1 as follows:The last equation leads to . This ends the proof of Theorem 1.
Given the SSE of the quadratic model defined by (11), FC tools can be used to compute coefficients , , and of the fitted model. Theorem and proof of this assertion are given below.
Theorem 2. Given the quadratic model (7), the Least square-based coefficients of the fitted model (9) computed using fractional derivative that would best fit the model are as follows:
Proof of Theorem 2. The proof is done by the minimization of the SSE, (11) using fractional derivative. To ease computational procedure, the SSE is expanded as follows:Applying the partial fractional derivative to the expanded SSE, equation (19) leads toIt follows from (20) thatApplying similar methods to those given by (19) and (20), the remaining coefficients are given as follows:This ends the proof of Theorem 2.
It is clear that the estimated coefficients of the quadratic model (see equations (21)–(23)) are interconnected, making it difficult to compute any of them independently. In this regard, equations (21)–(23) are arranged to form a system of equations, (24) to express and compute the coefficients simultaneously.Equation (24) is written in a matrix form asGoodness of fit model measures how explainable is the target data through the model. As coefficient of determination, is a popular goodness-of-fit metric used in regression analysis. It is computed using the sum of squared regression (SSR) error also known as explainable error and the total sum of squared deviation of the data from its mean.
Definition 4. We consider a dataset in format, where is the predictor and the response variable. Let be the fitted response variable through a linear regression model. The coefficient of determination of the model is computed as follows:In practice, with a value of 0 means that the model does not explain the target variable, whereas a value of 1 means that the model is perfect, thus explaining the entire variation in the target dataset.
The square root of the is the absolute value of a new metric called the correlation coefficient and is denoted or . In practice, instead of using (26) to compute , one can just compute the square of the correlation coefficient. Computed with a value of 0 means expressing no correlation between and ; whereas values of -1 and 1 means perfect negative and perfect positive correlation, respectively.
Let be the slope of the linear regression model defined by (3). Let , be the standard deviations of and , respectively. There exists a relationship between and defined bySince and , it follows from (27) that and play similar role in determining whether the linear relationship between and is positive or negative. Besides their common relationship, determines the steepness of the regression line. Unlike and that are restricted to the closed intervals [0, 1] and [−1, 1], respectively, is unitless and its magnitude solely depend on the range of the datasets and .
4. Experiment and Comparison
This section aims to simulate the formulae established in previous sections. Two datasets are considered.
The first dataset which was retrieved from the Kaggle database [14] which consists of two variables, the predictor (X-variable) is the number of years of working experience of employees, whereas the response variable Y is their yearly salaries in USD. The scatter plot of the dataset shows a linear trend (see Figure 1(a)). Next to Figure 1(a) is the Figure 1(b), which represents three different models aiming to fit the dataset. The blue line is the linear regression taken in the classical sense, whereas the grey and brown lines represent fractional models taken, respectively, for q = 0.98 and q = 1.01. These lines were taken to illustrate the behavior of the proposed model. A global trend of the error rate of the classical approach and the fractional approach is shown in Figure 1(c). In fact, all possible values of the fraction in the interval [0.9–1.3] were considered using a step h = 0.001 between two values. This led to a total of 400 possible values. Each value was used in building the fractional model, and the fitting error rate was computed using the formula found in [15]. It was observed that when the fractional order of derivative coincides with the classical first derivative, both methods produce the same error rate. This is evident that the proposed method is well defined and that there exists a smooth transition from the fractional to the classical approach in this case.
(a)
(b)
(c)
When the fractional order of derivative coincides with the classical approach, the correlation coefficient of the model is and the coefficient of determination is ; standard deviations are and . The slope of the model is . These information leads to the conclusion that more than 95% of variation in salary depends on the number of years of experience. The relationship is positive, which means salary increases with years of experience.
The second dataset which was retrieved from the United Nation database [16] consists of two variables; the predictor (X-variable) is the indexes of years between 1970 and 1995, and the response variable Y is the Bosnia and Herzegovina population within those years. The scatter plot of the dataset shows somewhat a quadratic trend (see Figure 2(a)). Beside is Figure 2(b), which represents three different models fitted to the dataset. The blue curve is the classical quadratic regression, whereas the grey and brown curves represent fractional models taken, respectively, for q = 0.98 and q = 1.001. These curves were meant to illustrate the behavior of the proposed model. A global trend of the error rate of the classical approach and the fractional approach is shown in Figure 2(c). In fact, all possible values of the fraction in the interval [0.9–1.1] were considered using a step h = 0.001. This resulted in a total of 200 possible values of q. Each q value was used in building the fractional model, and the error rate of the model was computed using the formula found in [15]. It is observed that when the fractional order of derivative coincides with the classical first derivative, both methods produce the same error rate. This is evident that the proposed method is well defined, and similar to the linear case, a smooth transition from the fractional to the classical case is observable.
(a)
(b)
(c)
There are many prospective applications of the proposed method. In particular, the fractional derivative approach proposed in this work appears as a powerful alternative to estimate the fuzzy coefficients in a fuzzy linear model. Several authors have investigated coefficient computation of the linear model with a crisp predictor but fuzzy response [17–19]. In a book chapter on fuzzy linear regression [17], authors proposed a Monte Carlo approach to build the confidence interval of fuzzy coefficients and their estimations. The obtained coefficients were then written as a triangular fuzzy number. More interesting fact is that the Monte Carlo simulation was time consuming as it required about 30 min to complete the process. Based on experimental studies, we believe that two fractional derivatives with fractions from either side of 1 would be efficient enough to build such fuzzy coefficients. This is not in the scope of the current work, but the said approach will be investigated in forthcoming work.
5. Conclusion and Future Work
In this work, we investigated computation of the coefficients of simple linear regression and quadratic regression using fractional derivative as a result expressions for the regression coefficients of the linear and quadratic models which is established under the fractional approach. Their solutions investigated and applied to two different datasets. Moreover, a smooth transition was observed between the classical model and the fractional model as both models coincide when the fractional order of derivative q = 1. Another interesting fact that deserves in depth investigation in future work is to determine if the proposed fractional method can be a powerful alternative tool to Monte Carlo and other approaches used in estimating fuzzy coefficients in fuzzy regression analysis. The results of this work lay a foundation of further studies to investigate other statistical methods using FC approach.
For the future work, researchers can investigate a modified estimation for the regression coefficients by using a different type of fractional derivative such as Caputo, Psi-Caputo, and Hadamard fractional derivatives and trying to compare the accuracy of each estimator via the statistical methods.
Data Availability
Two datasets were used in the experimental section of this article. The first dataset “simple linear regression for salary data” is freely available at this URL: https://www.kaggle.com/search?q=simple+linear+regression. The second dataset “Bosnian population” is freely accessible at the following URL: https://population.un.org/wpp/Download/Standard/Population/.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Authors’ Contributions
Each of the authors, M.A. and Y.Y.Y, Y.T., and K.A. contributed to each part of this work equally and read and approved the final version of the manuscript.
Acknowledgments
This work was supported through the Annual Funding track by the Deanship of Scientific Research, Vice Presidency for Graduate Studies and Scientific Research, King Faisal University, Saudi Arabia (Project no. AN000648). The authors, therefore, acknowledge technical and financial support of DSR at KFU.