Abstract

This paper proposes a forecasting methodology that investigates a set of different sparse structures for the vector autoregression (VAR) model using the Ivanov-based least absolute shrinkage and selection operator (LASSO) framework. The variant auxiliary problem principle method is used to solve the various Ivanov-based LASSO-VAR variants, which is supported by parallel computing with simple closed-form iteration and linear convergence rate. A test case with ten crude oil spot prices is used to demonstrate the improvement in forecasting skills gained from exploring sparse structures. The proposed method outperformed the conventional vector autoregressive model.

1. Introduction

Crude oil, as one of the world’s largest traded commodities and most valuable energy resources, plays a vital role in the global economy. It is well known for its wide price fluctuations, which have a direct impact on the economy. The rise in oil prices may cause inflation and eventually affect the economies of oil importers, while the fall in oil prices may cause economic recession and political instability in the economies of oil exporters. Furthermore, even a minor fluctuation in the price of oil can result in significant economic losses and social consequences. The energy crisis and the constantly fluctuating price of petroleum have drawn considerable attention of researchers [13]. Crude oil is a globally influential commodity because it is the major source of primary energy. Crude oil prices are a reflection of market expectations for future macroeconomic variables. In today’s turbulent world, some shocks, such as the COVID-19 pandemic and the Ukraine war, are having a cascading effect on the world economy. In this environment, even a rumor of a possible crude oil production cut could result in a significant oil price hike. Crude oil forecasting, as suggested by many literature, has become an important topic in terms of both theoretical and practical implications [46]. However, forecasting crude oil prices is an extremely tough and challenging task in the prediction literature. On the one hand, the price of crude oil is fundamentally determined by supply and demand [7]. On the other hand, unlike other commodities, the price of crude oil is determined by exogenous factors, such as extreme events, global economic conditions, speculative expectations, political instabilities, as well as technological trends [811]. Because of the aforementioned factors, forecasting crude oil prices is one of the most important but challenging tasks, attracting the attention of an abundance of prediction literature.

According to the existing literature, different types of predicting methods have been proposed to forecast crude oil price. By using parameter evaluation methods, the existing approaches can be classified into three groups: (1) traditional, statistical, and econometric models, (2) artificial intelligence (AI) techniques, and (3) hybrid models.

The traditional statistical and econometric models used in crude oil forecasting studies include linear regression, co-integration analysis, autoregressive integrated moving average (ARIMA), generalized autoregressive conditional heteroscedasticity (GARCH) family models, naive random walk, gray model, vector autoregression (VAR), and error correction (ECM) models [8, 1113]. Morana [12] employed a semiparametric GARCH methodology to forecast the oil price over short-term horizons. Hou and Suardi [13] used nonparametric GARCH models to estimate and forecast crude oil price return volatility. Ye et al. [14] incorporated low- and high-inventory variables in a single equation model to forecast short-term crude oil prices. Mirmirani and Li [15] used a VAR-based method to make ex-post forecast of U.S. oil price movement. Lanza et al. [16] proposed a comprehensive analysis of crude oil and product price dynamics using co-integration and ECM models. However, traditional statistical and econometric methods are based on linear assumptions and have good prediction ability when the price series are linear or nearly linear. As demonstrated by the existing literature [4], the prediction performance of traditional statistical and econometric approaches might be very poor because there is a significant deal of nonlinearity and irregularity in crude oil price series.

Because of the limitations of traditional, statistical, and econometric techniques, the price forecasting literature has proposed a bunch of nonlinear and AI models, including support vector machines (SVMs) and artificial neural networks (ANNs). Movagharnejad et al. [17] developed a neural network model to investigate the price variations of various commercial oils in the Persian Gulf region. Chiroma et al. [18] proposed an evolutionary neural network model, which was based on a generic algorithm and neural network, to forecast the West Texas Intermediate (WTI) crude oil price. Abdullah and Zeng [19] investigated a machine learning approach for crude oil price prediction with an artificial neural network-quantitative (ANN-Q) model. Abramson and Finizza [20] used belief networks, which are knowledge-based models, to predict crude oil prices. Shambora and Rossiter [21] used an ANN model with moving average crossover inputs to forecast future crude oil prices. Xie et al. [22] forecasted crude oil prices using SVM-based methods and compared their performance with the ARIMA and BPNN models.

Numerous studies find that AI models often have better forecasting ability than traditional statistical and econometric models in price forecasting [4, 23, 24]. However, AI models also have their own shortcomings and limitations. For instance, ANN is sensitive to parameter selection [4]. To overcome the limitations of single traditional AI tools, more and more hybrid methods, particularly decomposition-based hybrid models, have been applied to the forecasting of crude oil prices [11, 2527].

Although these decomposition-based methods outperform in forecasting and analysis, these hybrid techniques have some limitations in price forecasting. For instance, some hybrid models with fixed basis design are sensitive to parameter settings in denoising [28]. In addition, the forecasting accuracy of hybrid models tends to be constrained by the underlying techniques, and some heterogeneous hybrid models are computationally intensive [29].

Many literature used VAR model to predict the prices and returns of commodities [3035]. The multivariate VAR model, as one of the most widely used econometric techniques, has been used in numerous empirical studies. However, the VAR model has two main problems: (1) the number of time series in a VAR model is limited because the number of parameters to be estimated is quadratic with the number of time series contained [34]; (2) the VAR estimation procedure does not consider fat-tailed errors, so extreme observations in the volatility series are ignored [36].

To address the concerns raised above, we propose a new method that combines the VAR- and Ivanov-based least absolute shrinkage and selection operator (LASSO) framework to forecast multiple crude oil product prices. In addition, we design a variant auxiliary problem principle (VAPP) algorithm for solving Ivanov-based LASSO-VAR (I-LV) problems that can be implemented in parallel based on their characteristics. The work proposed in this paper is closely related to the standard VAR and provides the following original contributions: first, we investigate a set of different sparse structures for the VAR framework using the Ivanov regularization based LASSO framework. Second, this paper applies VAPP to fit the different VAR Ivanov-based LASSO variants. Finally, we present a scalable forecasting method that is based on parallel computing, a fast convergence optimization algorithm, and matrix calculations.

Many new studies on crude oil price prediction have recently been published. Jiang et al. [37] combined a decomposition-ensemble approach with sentiment analysis to forecast crude oil prices. Because crude oil futures price data is nonlinear and nonstationary, Sun et al. [38] adopted the idea of “divide and conquer” to develop a new crude oil futures price combination forecasting method based on decomposition and reconstruction integration technology. Starting with the market economic model, a novel dynamic time-delay gray model for energy price forecasting is selected based on the differential information of the differential equation and difference equation, as well as the data reduction principle [39]. Wu et al. [40] developed a hybrid framework in which the Hampel identifier is employed to identify and correct outliers, while the complete ensemble empirical mode decomposition removes noise through decomposition and reconstruction of data. To improve the forecasting accuracy and stability, they proposed a modified multiobjective water cycle algorithm. Based on the forecasting research of multisource information and decomposition-ensemble. Guo et al. [41] proposed a multiperspective crude oil price forecasting model under a new decomposition-ensemble framework.

The remainder of this paper proceeds as follows: Section 2 presents the methodology for forecasting crude oil prices. Section 3 describes the application of the VAPP method to fit the VAR model in its different Ivanov-based LASSO variants. Section 4 presents the forecasting results and evaluates the performance of the proposed methodology. Finally, Section 5 presents the conclusion.

2. Ivanov-Based LASSO-VAR Forecasting Methodology

Let represent both vector and matrix norms. Let denote a k-dimensional vector time series. represents a coefficient matrix related to the lag l. In order to get a compact matrix notation, let Y =  define the response matrix, the matrix of explanatory (or predictors) variables in which , and p is the order of vector autoregressive process. is the matrix of coefficients.

To simplify the notation, we consider . Then, there are two standard LASSO-VAR formulations to combine a regularizer (such as ) and a data-fidelity term as follows:(i)Tikhonov regularization (referred to as T-LV)where is a scalar regularization (or penalty) parameter controlling the amount of shrinkage.(ii)Ivanov regularization (referred to as I-LV)

Similarly, is a scalar regularization parameter.

Although these two problems are equivalent (under mild conditions), I-LV formulations may be more convenient in practice because the corresponding parameter is easier to adjust. However, the I-LV problems cannot be efficiently dealt with. Therefore, in this paper, we will develop an efficient algorithm for solving I-LV problems.

Different regularization penalties can be used in the I-LV model to reduce the effective dimension of the problem, and different sparse patterns can be detected based on the inherent structure of VAR. The I-LV framework does not assume that all predictors contribute to the model; instead, it extracts the most significant predictors. The efficient use of appropriate penalties will lead to more accurate estimation and prediction strategies.

Table 1 briefly describes the following I-LV structures that promote sparsity: standard I-LV (I-sLV), lag-group I-LV (I-lLV), lag-sparse-group I-LV (I-lsLV), own/other-group I-LV (I-ooLV), and causality-group (I-cLV). The different penalties applied to them result in different types of sparsity, depending on the selection target that manages them. More information on these structures can be found in studies conducted by Cavalcante et al. [42].

3. I-LV Fitting by VAPP

The Ivanov-based LASSO-VAR problem can be expressed as the following Nonlinear Convex Cone Programming (NCCP):where the decision variable is B, objective function , represents different regularization terms, which are listed in Table 1, C is a convex cone. Zhao and Zhu [43] introduced a flexible first-order primal-dual algorithm called VAPP for solving NCCP problems. For the VAPP type algorithms to be proposed, their main subproblems at each iteration have closed-form solutions. We review three operators that will help us express these closed-form solutions of VAPP conveniently.

The first minimization problem is as follows:where and . It has a closed-form solution, which is given by the soft-shrinkage operator defined as follows:where sign (⋅) is the sign function.

The second minimization problem is as follows:where and . It has a closed-form solution which is given by the following equation:

The third minimization problem is as follows:where , and . It has a closed-form solution which is given by Chartrand and Wohlberg [44].

Now we apply the VAPP to solve the above five kinds of I-LV models, and simple closed-form iterations are given. The L1 penalty works as a sparsity-inducing term over individual entries of the coefficient matrix B, and I-sLV problem can be written as follows:and the primal-dual iterative scheme of VAPP is as follows:where and . The details about the selection of parameters and are shown in Zhao and Zhu [43]. Then, it follows from (5) that the closed-form solution of the B-subproblem is given by the following equation:

The I-lLV model considers the coefficients grouped by time lags and looks for time lags that improve forecast accuracy.

The primal-dual B and update solution can be obtained by the following equation:where . Then, the closed-form solution of B-subproblem is given by using (7).

However, it may be too restrictive for crude oil spot price forecasting because all the coefficients of some lags are not considered or sometimes inefficient by including the entire lag if only few coefficients are significant. Therefore, the I-lsLV model adds lag sparsity to the I-lLV.

Similar to the above procedure, the primal-dual iteration of VAPP is as follows:where and . By using (9), we can get B-subproblem closed-form solution as follows:

Many crude oil spot price predictions are influenced more by their own past observations than by past observations of other spot prices. In the I-ooLV, the coefficients are grouped by the diagonal entries and by off-diagonal entries.

Its primal-dual iteration is as follows:where and . Moreover, the closed-form solution of B-subproblem is as follows:

The I-cLV model groups the coefficients according to their corresponding spot prices in order to learn a causal inference from the data.

The primal-dual iteration is as follows:where , and B-subproblem closed-form solution is as follows:

4. Practical Implementation and Results

In this section, we first describe the data set used in our paper, followed by the experimental setup for predicting crude oil prices. Finally, we discuss in detail the forecasting results.

4.1. Data Description and Experimental Setup

The VAPP algorithm is applied to the proposed I-LV variants in order to predict crude oil spot price values for horizons up to five-steps-ahead. The crude oil spot price data deployed in this paper are from the U.S. Energy Information Administration. The spot price dataset include US WTI crude oil, European Brent crude oil (Brent), New York Harbor regular conventional gasoline (NYCG), U.S. Gulf Coast regular conventional gasoline (USCG), New York Harbor No. 2 heating oil (NYHO), New York Harbor ultra-low-sulfur No. 2 diesel fuel (NYDF), U.S. Gulf Coast ultra-low-sulfur No. 2 diesel fuel (USDF), Los Angeles ultra-low-sulfur No. 2 diesel fuel (LADF), U.S. Gulf Coast kerosene-type jet fuel (USJF), and Mont Belvieu Texas propane (MB). The sample time ranges from January 2, 2018 to December 31, 2019. We use the samples from 2018 and 2019 as training and testing data, respectively, in our studies.

Table 2 lists time spans and observations of the training and testing sample. The observations in the training and testing samples are 246 and 249, respectively, because commodity prices are not reported on weekends or holidays and we discard observations with missed prices.

Table 3 presents descriptive statistics of different commodity prices. Two lags are utilized in each of the models. The evaluation of the I-LV structures performance is accessed using the root mean squared error (RMSE) and mean absolute error (MAE) calculated for each t + h; h = 1; 2;…; 5, lead time with the following expressions:where represents the forecasting made at time instant t and is the observed crude oil spot price value.

4.2. Forecasting Results and Discussion

The performances of the I-LV models are compared by computing the improvement over the VAR model (using least square estimate) in terms of RMSE and MAE. Tables 4 and 5 show the average MAE and average RMSE across all spots for different I-LV structures.

Tables 4 and 5 show that the I-LV models exhibit more significant forecasting accuracy than classical VAR, with the exception of one-day-ahead prediction of I-lLV. In addition, I-cLV is obviously better than other methods. It is possible to observe that the performance of VAR drops rapidly with the lead time, however, I-LV structures are relatively stable. Figure 1 compares I-cLV with the VAR model, representing the improvement over the VAR model for each commodity for the first lead time. The results show that, for the first lead time, the VAR only outperforms I-cLV for one commodity, namely “MB.” Apart from that commodity, the improvement over VAR ranges between 5.29% and 87.39% for MAE and 5.01% and 87.47% for RMSE.

Through the analysis of the correlation between each commodity from Table 6, we find that the correlation between “MB” and other commodities is not significant, which is the reason why the prediction performance of I-cLV is poor. Finally, Figure 2 for “USCG” shows a visualization of the real crude oil price and the forecast crude oil price output provided by the I-cLV model for the first lead time during a one-year period.

In order to understand the joint dynamic behavior of this group of commodities, the sparsity patterns (i.e., coefficients’ matrix) obtained by the I-LV structures and VAR for the first lead time are depicted in Figure 3. The blue dots represent coefficients that are nonzero entries. Figure 3 shows that classical VAR and I-lsLV methods give rise to 100% nonzero entries. Immediately afterwards, the best performance I-cLV gives rise to 42% sparsity. It can be observed that almost all figures agree that the diagonal coefficients of the first lag are nonzero entries, indicating that a variable’s own first lag is more likely to improve the forecast than the other entries. Therefore, the I-cLV sparse structure can be useful in determining which commodities can promote the forecasts for other commodities and obtain the best prediction.

Table 6 shows that the spot price of each crude oil is closely related not only to the price of that crude oil, but also to the prices of other crude oils. However, we note that not all crude oil prices are closely related to the prices of other types of crude oil, such as MB’s. Only the I-cL model considers these factors at the same time, while the other five models do not. Therefore, the I-cL model is superior to the other five models in terms of prediction accuracy.

5. Conclusion

This paper describes a forecasting technique that combines VAR and several variants of the Ivanov-based LASSO framework to fully explore data from crude oil time series distributed across different spots. The proposed methodology investigates competing sparse structures for the VAR coefficients matrix and employs the VAPP optimization framework to ensure fast convergence and parallel computation. For a real case study with ten crude oil spots, all the different sparse structures of the I-LV model show better performances than the VAR models. The I-cLV structure turns out to be the best choice for forecasting crude oil prices.

Data Availability

The data used to support the findings of this study are included within the article.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

This research is supported by National Natural Science Foundation of China (72073098; 71871140; 72173087), National Social Science Fund of China (grant number: 21ZDA094), Fundamental Research Funds for the Central Universities (grant number: 2021110834), Shanghai Philosophy and Social Science Planning Project (2020BJB012), and The Humanities and Social Sciences Program of the PRC Ministry of Education (20YJA790054).