Abstract
The presented article is methodological in nature and is devoted to the analysis of observation series of financial asset quotation changes in capital markets. The most important feature of these processes is their instability, which manifests itself in high sensitivity to seemingly minor disturbing factors. This phenomenon is well-studied in the theory of nonlinear dynamical systems and is described by models of deterministic chaos. However, for the processes considered in the article, the dynamic instability of the immersion environment is exacerbated by stochastic uncertainty caused by random fluctuations in the pricing process. As a result, describing observation series of quotations of financial assets is difficult because it involves stochastic chaos. This article analyzes and classifies chaotic series of observations to help model and forecast related processes.
1. Introduction
Currently, there are no management strategies that unambiguously guarantee winning in speculative operations on capital markets. Trading, by its nature, is a venture. It is difficult to imagine what the entire world trade would look like without management risks. The very concept of risk directly follows from the need for management in conditions of uncertainty.
Cybertrading is an important trend in modern finance trading, where the majority of decisions is being made by artificial intelligence. This requires the development of increasingly complex mathematical models for automatic detection of market situations. These models use algorithms such as machine learning, deep learning, natural language processing, and sentiment analysis to detect patterns and trends in the data and make decisions accordingly. They must be able to handle large amounts of data in real-time and be robust enough to handle changes in the market [1].
Within the framework of the efficient market hypothesis (EMH), it is assumed that investors behave rationally, that is, they tend to make management decisions based on generally accepted probabilistic ideas about the expected changes in the state of a particular financial instrument. However, the analysis of the application of this hypothesis in practice showed its inconsistency [2]. The reason why traditional methods of analyzing the market state are not effective is because of the level and nature of the uncertainty that lies in the way of an effective forecast. This uncertainty is much higher and more complex than the limitations within which traditional methods are implemented.
In fact, the main problem of managing speculative trading operations for active investments has been and remains the uncertainty of the dynamics of the processes taking place in the markets. It should be noted that any management in general, and trading operations in particular, is proactive, i.e., based on the forecast of the development of the market situation. Indeed, the result of a management decision always lies in the future relative to the moment of its execution. This is especially clearly seen in the example of speculative operations in the process of trading on electronic capital markets. The main task of a trader has always been making a prediction of the change in quotations based on the available financial instruments. Furthermore, we will call such instruments as working, unlike other financial assets used as auxiliary in the process of analyzing and forecasting the development of the trading situation.
In the case when a trader manages to construct an effective forecast, the problem of managing the working instrument becomes trivial. In other words, building an effective forecast is the necessary and sufficient condition for constructing a winning management strategy. An effective forecast will be understood as a quantitative or qualitative prediction of the cost of a working instrument, which can be a basis for a winning management strategy. It is essential to keep in mind that, when assessing the effectiveness of a predictive model, you should not focus on its accuracy or reliability, but on the final outcome [3].
An essential feature of trading operations is that quotations of market assets are not physical quantities, instead, they are the result of a mental group assessment of the market’s state. The time sequence of such estimates forms an inertialess stochastic process, the system component of which is an implementation of dynamic chaos [4–7]. The presence of a chaotic component in the dynamics of quotations significantly reduces the possibility of building an effective forecast and, as a result, leads to low effectiveness of the generated control decisions.
The natural requirement to improve the quality of management of speculative operations leads to the need for a deeper understanding of the features and structure of chaotic processes. To this end, we will consider the genesis of uncertainty caused by unstable immersion environments as a list of problems that arise for an expert analyst when solving control problems in conditions of stochastic chaos.
2. Methods
2.1. Dynamic Uncertainty
The presence of uncertainty in the dynamics of pricing is the main problem for the analysis and forecasting of the trading situation for effective management of speculative operations. The initial data for the analysis and forecasting of dynamics are retrospective implementations of a nonstationary random process describing the dynamics of quotations of market assets and the environment of their immersion. However, the object of speculative research in asset management is not the series of observations themselves, but their system component formed by sequential filtering of random fluctuations. It is the system component of the series of observations that serves as the basis for further forecasting and the formation of proactive management decisions.
Within the framework of the traditional probabilistic paradigm, the system component is interpreted as an unknown deterministic process. The main source of uncertainty is the incompleteness of knowledge of the relative number of factors affecting the dynamics of the observed process. As a result, observation series are interpreted as random processes by the end user. Then, the system component can be identified using variants of sequential estimation [8–11].
The mathematical model used in the extrapolation is sequentially selected and corrected for adaptive versions of filtering algorithms.
Data analysis based on Bayesian estimation is the development of the statistical approach to identify the system component of observation series [12–14]. In this case, the system component of a series of observations is considered as a conditional mean of the observed random process. Within this assumption, computational schemes such as the Kalman filter [15] or other generalized variants of this algorithm (for example, filters with finite memory) are used for its formation [16–20].
As the practice of technical analysis of market assets has shown, the abovementioned traditional approaches to forecast observation series of pricing processes have proved ineffective. The reason for this is the high instability of the immersion environment, described by various models of chaotic dynamics [21, 22].
The problem of chaotic dynamics in unstable immersion environments was described in the mid 20th century in the fundamental works of Lorentz [23], Haken [24], Nicolis and Prigozhin [25] and others, who laid the foundations of a new synergetic paradigm describing the instability of solutions in the dynamics of open nonlinear systems. According to the general theory of nonlinear differential equations, their solution may contain so-called bifurcation points in which the dynamical system is parametrically unstable. At these points, even an extremely insignificant perturbation can radically change the dynamics of the integral curve obtained as a result of solving a system of nonlinear differential equations.
From an economic point of view, bifurcation points represent moments in time when even a small external disturbance can catastrophically change the dynamics of the development of the value of an asset or even the entire market. In [21, 22], the chaotic nature of the system component of the series of observations of market asset quotations is substantiated. A simple visualization of the dynamics of quotations illustrates all the characteristic features of a chaotic process: an oscillatory nonperiodic trajectory, presence of local trends and anomalous observations, heteroscedasticity, etc. As an illustrative example of such a series of observations, Figure 1 shows the graphs of the process of changing the quotations of the EURUSD, GBPUSD, and USDJPY currency pairs on the Forex market over a time interval of 250 days.
The most reliable mathematical criterion of randomness is the maximal Lyapunov exponent, which evaluates the average measure of divergence of the system’s trajectories. Among other approaches to quantitative evaluation of time series randomness, we can note rescaled range analysis (RS). It allows one to calculate the Hurst exponent, a measure of the randomness of a series [26]. The values of the Hurst exponent close to 0.5 indicate stochastic series. The Hurst exponent being close to 1 suggests that the time series under consideration is generated by some chaotic system. For the series, a fragment of which is shown in Figure 1, the Hurst exponent fluctuates in the range 0.68–0.72 (window width L = 200).
According to [27], the most promising segmentation method is considered to be the Hölder condition, which is a local measure of the regularity of a function. The Hölder exponent exceeding 1 indicates a smooth function. If the exponent is less than one, then the function is singular in the given point. For the series in Figure 1, the estimates of the Hölder exponent range from 0.2 to 1.1 with an average value of 0.61 (window width 200–400).
Of particular interest is the Hinich test [28–30], based on the spectrum of the 3rd order, more precisely, on the assessment of bicoherence. In Figure 2, the P_gauss statistic can be interpreted as the weight of the possibility that this segment can be described by the Gaussian model. Relatively small (less than 1.5–2) values of the del_R statistics indicate a series of data that could be described by a linear model.
Figure 2 shows the segments of the values of the Hinich test calculated on sliding windows (with overlap). The dynamics of these statistics for the entire available data show that the series are non-Gaussian almost throughout their entire length, but significant parts of them can be described by linear models.
2.2. Statistical Uncertainty
As already mentioned, the current state of both an individual asset and the entire market is influenced by many factors of various nature. The number of significant economic, political, psychological, sociological, and such factors can reach hundreds and thousands. At the same time, these factors themselves are implementations of random processes, the influence of which can vary significantly over time.
The attempts to use the apparatus of probabilistic modeling and statistical data analysis led to models of stochastic dynamics of quotations based on the hypothesis of IID (independence and identity of data distribution) [31]. This hypothesis is based on a number of assumptions about the nature of the processes taking place in the capital markets, to justify the normal distribution of changes in quotations with a mean that is continuously differentiable and bounded variance. In the simplest case, this scheme results in a Markov random walk, assuming that current observations are sufficient to implement a trading forecast.
The problem of statistical uncertainty lies in the significant inadequacy of the simplified probabilistic data model described above for the results of descriptive statistical analysis of real market processes [21, 22, 32]. The probabilistic characteristics of the series of observations do not meet the set of limitations that provide applicability or effectiveness of traditional statistical algorithms for data analysis and forecasting.
The most significant deviation from the simplified data model is the nonfulfillment of the hypothesis on the stationarity of the residuals of observation series. Residual in this case refers to the difference between the observations and the system component of the observed process. In particular, sequences of such observations are heteroscedastic [33, 34].
Statistical uncertainty is a well-known problem and has been studied in the literature on robust statistics [35–41]. However, in this case, the uncertainty about the probabilistic data structure is superimposed on the uncertainty caused by the systemic instability of deterministic chaos. As a result, the processes characterizing the dynamics of capital markets should be classified as stochastic chaos, which has the highest degree of complexity in terms of forecasting and management tasks.
2.3. Multidimensionality and Multiple Connectivity
The problem of multidimensionality is obvious: the state of the market, as noted above, is affected by a large number of interrelated external and internal factors of very different nature. The more factors are considered when constructing a model of the market, the more reliable the forecast will be, the better the trading decisions will be. However, considering each factor involves identifying its level of significance (which may vary in the forecast interval), structural and parametric identification of the dynamics of the evolution of the factor itself, and the function of its influence on the market condition. In other words, comprehensive monitoring of the entire immersion environment of the working instrument is necessary, and this means the need to identify and track dozens and hundreds of significant and interrelated factors of influence.
It is obvious that the complete solution of such a problem would rely on the need to take into account thousands of latent factors of influence and is beyond even supercomputers. Therefore, in practice, such solutions are limited to taking into account the influence of the most significant factors, and the total effect of all others is described as a stationary random component that meets the central limit theorem, i.e., subordinate to the Gaussian distribution.
As noted above, such a simplification can be accepted with a certain level of trust (more precisely, distrust), keeping in mind the presence of local trends and anomalous observations, possible heteroscedasticity of series of observations, etc. However, dialectically speaking, it is said that every dark cloud has a silver lining. By having multiple connectivity and stable correlations over large intervals of observations, we are able to use statistical data analysis to gain fragments of local stability. Examples of such solutions are given in [42, 43].
2.4. Source Data Completeness
The availability of the results of monitoring the state of the market and the trading environment is necessary, but far from sufficient condition for effective decisions. The fact is that the monitoring results may either not contain the information (even indirect) necessary to prepare a qualitative prognostic assessment of the state of the market, or they may not be displayed correctly enough. The reasons for this discrepancy can be roughly divided into two parts:(1)Incompleteness or insufficient reliability of the monitoring process(2)The human factor.
In essence, the problem of incompleteness of data is the main cause of the above problems of dynamic and statistical uncertainty. However, even if there was a Laplace demon capable of taking into account all the factors influencing the pricing process of trading assets, it would still not be possible to obtain a guaranteed prediction result due to the chaotic nature of the processes. As mentioned above, for unstable immersion environments, even minor disturbing effects can lead to an extremely inadequate reaction from the process.
The presented problems are associated, first of all, with the features of the data generated by the systems that monitor the state of assets and their immersion environments. Along with them, there is a group of problems caused by the peculiarities of management tasks in conditions of market chaos and the specifics of the human factor. Let us describe some of them.
2.5. Multicriteriality
The quality of a trading decision is usually based on the analysis of two or more quality criteria. In the simplest case, such criteria are contradictory requirements for maximizing profit and minimizing risk. As a rule, such optimization problems do not have a joint solution.
The use of scalarizing metrics can help simplify multicriterial problems by reducing them to the optimization of a single scalar parameter. An alternative option is to optimize one generalized parameter, and all other characteristics are translated into a system of constraints. For example, for the above example, this means the task of maximizing profits while maintaining the risk level above the set one.
Solving the problem of multicriteriality on the basis of a multiexpert system [44] seems promising. In this case, various software experts obtain decisions using various optimization or other criteria, and then coordinate them using mathematical methods of conflict resolution [45, 46].
2.6. Heterogeneous Data Integration and Analysis
It is quite clear that the analysis of well-structured numerical data differs significantly from the processing technologies of textual and other poorly structured information. Currently, there exist various methods of analyzing heterogeneous data; however, their joint processing still constitutes a rather complex scientific problem.
One of possible solutions, as mentioned above, is the use of methods of multiexpert analysis. In this case, independent or weakly dependent software experts produce independent solutions, and at the second level, metaexpertise is carried out, which searches for a compromise taking into account the weights of individual experts.
2.7. Market Psychology
As already mentioned above, the classical model of an efficient market is based on the hypothesis of a rational investor operating in accordance with general ideas about “common sense” and priorities for the stability of trading operations [1, 2]. Unfortunately, neither the individual psychology of an individual investor, nor the social psychology of individual groups of investors and stock market players in many situations fit into these models.
The specifics of social market psychology are particularly acute under the influence of the news stream. Manipulating the perception of news leads to an unjustified abrupt change in the price of an asset or even a market segment, which significantly complicates the possibility of forming an effective forecast for such situations.
2.8. Human Factor and Data Analysis Interpretation
The final decisions in the field of speculative data analysis, as a rule, belong to a person. The specifics of human thinking in quantitative analysis allow us to consider an extremely small number of factors of influence (at best, 2-3 interrelated factors). Human consciousness is, in principle, limited to a three-dimensional representation of reality, while the requirements for the correct accounting of disturbing influences involve operating with state spaces with a dimension of tens and hundreds of units. Thus, the task, in principle, is out of the quantitative control of an expert, analyst, or investor. The natural way out of this situation is to aggregate data to dimensions that can be visualized. However, any aggregation, as well as any data compression, inevitably leads to partial loss of information.
An equally important component of the human factor problem is the interpretation of data and the results of their processing. This question is complex, it is contained, explicitly or implicitly, in a number of the above problems related to the incompleteness of the initial data, issues of multicriteria, limited perception, specifics of market psychology, etc.
It appears that this issue does not have an unambiguous solution within traditional data analysis and requires the latest information technology. These include artificial intelligence technologies, including the tasks of constructing knowledge bases, methods of analyzing tasks for individual, and group decision-making schemes. The most radical approach to this problem is the exclusion of the human from the control loop. In this case, the decision is made by a trading robot, or a network of software experts combined into a single multiexpert system.
2.9. Asset Management in the Conditions of Dynamic Chaos
The problems of forecasting and management listed above in the conditions of stochastic chaos significantly complicate the process of developing effective management solutions. Conventional technologies for increasing the stability of forecasting and management processes often become ineffective when faced with an unstable market environment of immersion. For example, adaptation methods often turn out to be infeasible due to the inertia and discontinuity of the processes taking place in the markets. The feedback chain simply does not have time to react to sudden changes in the dynamics of quotations. On the other hand, increasing the sensitivity of the feedback loop inevitably leads to pathological reactions to random fluctuations of the observed process, and, therefore, to statistical type II errors (“false alarms”) in the decision-making circuit.
Another traditional approach to increasing the stability of management, based on the robustification of estimation and forecasting algorithms, also leads to an increase in type II errors. Note that a posteriori analysis of retrospective data often allows us to determine such a fine line between sensitivity and roughness (robustness) of the decision-making algorithm, for which even weak control strategies turn out to be advantageous [47, 48]. However, this facet is not stable. Even small deviations from the optimal values of the parameters of the management model quickly lead to a loss in the subsequent observation interval.
Thus, a review of the state of the general problem of forecasting and control in conditions of stochastic chaos shows that the solution of this problem requires the development and application of new unconventional methods and algorithms for data analysis, significantly different from the known statistical and extrapolation computational schemes.
It is extremely important to have an idea of typical variations in the dynamics of quotations and their classification when developing forecasting and management algorithms. Such information is necessary to build mathematical models of observation series that contain changes in the state of the market. The further materials of this article are devoted to the methodological aspects of this task.
2.10. System Dynamics as Reactions to News
Let us make an important remark first. In itself, the nonstationarity of the observed process is not a catastrophic problem for predicting the state of dynamical systems. For example, the differentiable dynamics of the mean, in accordance with the Weierstrass approximation theorem [49], can always be described by some polynomial model. The approximation residuals can behave “quite decently” and with a certain level of approximation be described by the same stationary Gaussian model. The problem is that this approximation is made over the implementation of an already implemented process. Further development of the dynamics of the mean for statistical chaos in the near future turns out to be completely inadequate in relation to the newly constructed polynomial or another model. As a result, this approach, successfully used for many inertial physical and technical processes, turns out to be unsuitable even for short-term speculative asset management tasks in a market environment. However, the decision itself is of the simplest logical nature: it is enough to answer the question whether the quotes will grow, decrease, or progress within insignificant limits for a certain period of time.
Note that the same reason limits the possibility of using adaptive control. For most physical, biological, or technical processes, one can count on the inertia of processes with real mass or energy potential. This makes it possible to adjust the system model periodically or continuously, adjusting its parameters or even its structure. In information systems, associated with virtual representations of the market community about the value of assets, inertia is practically absent. Any quote under the influence of the news stream can almost instantly change its value many times. Hence, the phenomenon of the “joker” arises, which significantly limits the possibilities of adaptation to obtain an effective extrapolation forecast by means of traditional technical analysis.
However, the news stream is not a time continuum. News that significantly affects the dynamics of quotations form a nonstationary time sequence. Theoretically, in the intervals between the news, the dynamics of the asset should maintain some trend with an additive stochastic component superimposed on it. However, the process of a priori identification of such intervals, which has been named segmentation, is extremely complex. This is due to the presence of many latent factors that “interfere” with each other in a completely unpredictable way, with the appearance of “joker” news and other properties of market chaos.
In the simplest form, the news stream can be divided into three types of news:(1)Insignificant news that can change the dynamics of the observed process abruptly over a short period of time, after which the previous trend is restored. The trend itself might not change, but its scatter does. For example, the standard deviation is increased relative to some average dynamics.(2)Influential news that can significantly change the trend or its variability for a sufficiently long period of time in a few days or weeks.(3)Important (strategic) news that radically reverse the long-term trend and determine its overall dynamics for weeks or months.
For example, in Figure 3, consider the plot of the impact of news on the dynamics of EURUSD for 30 days. The arrow with the number 3 indicates the moment of the important news release, which determines the general trend of the quote change over a time interval of 30 days. The arrows with the number 2 indicate the moments of the release of news that can reverse the trend for several days, but not change its strategic direction of development. And finally, the arrows with the number 1 indicate the moments of news that can only slightly change the trend, but, nevertheless, they are still noticeable against the background of typical random fluctuations of the quote.
Analysis of such data indicates that if not the process of changing quotes itself, then at least its statistical structure can be maintained for a sufficiently significant time interval. For any identified structure, it is almost always possible to build an effective asset management strategy. In other words, if it is possible to quickly determine the general trend of the observed nonstationary process, then it is possible to use a pre-prepared strategy option that provides effective asset management for this dynamic structure.
This directly suggests the task considered in this article: the study of the possibility of timely identification of the current structure of chaotic dynamics. It is obvious that this task is essential for timely detection of significant changes in the structure of the dynamics of an observed process under the influence of a nonstationary news stream.
2.11. Fragmentation and Classification of Chaotic Dynamic Processes
As an example, let us consider a 30-day-long observation segment of EURUSD quotes shown in Figure 4. This plot showcases several sections of the typical structures of chaotic dynamics:(1)Areas of abrupt growth or decline. The process proceeds at a very high speed, with a large dynamic range and a small scatter relatively to the approximating linear trend.(2)Areas of rapid growth or decline of quotations followed by significant corrections in the form of significant sawtooth fluctuations in the opposite direction. The process has a large dynamic range and a large scatter relatively to the approximating linear trend.(3)Areas with a horizontal linear trend (the so-called “flat” or “sideways trend”). Such sections may differ significantly in the magnitude of the scatter and the frequency of crossing the line of the approximating mean.(4)Areas of rapid growth or decline of quotations with insignificant corrections which become additive fluctuations. The process has a large dynamic range and a small scatter relatively to the approximating linear trend.(5)Areas of relatively slow growth or decline. Similar to paragraphs 1, 2, and 4, they can also be divided according to the values of the scatter parameters and the nature of the correction process.
All things considered, given the very imprecise definitions of terms such as “fast,” “slow,” “large,” and “substantial”, we can propose a classification of typical structures of chaotic dynamics, presented in Figure 5.
Note that the definition of the above terms can be made in terms of fuzzy logic or based on a set of quantitative constraints made using plots of retrospective data at large observation intervals. However, the specific values of the parameters of these constraints can vary significantly depending on the dynamic properties of a particular asset (“instrument”). This is due to the different volatility and variance characteristics of different market instruments.
Identification of the structure requires the selection of a set of indicators which determines the type of structure of the observed process on the selected sliding window.
Analysis shows that the change of structure type is preceded by the change of the Hurst exponent. This is given by the series segmentation on the basis of this indicator. Similarly, a decrease in Hölder’s exponent is associated with abrupt jumps. Laminar segments with slow growth or decline are identified by the Hinich test value falling below 1.5. Segments with pronounced nonlinearity inevitably demonstrate non-Gaussianity, which is often used observing, for example, the dynamics of sample kurtosis [26–30].
The choice of the type of structure, as well as any other statistical conclusion, necessarily gives rise to type I and type II errors. In the first case, the indicator does not recognize the established structure, and in the second it makes an incorrect decision about the origin and type of the established structure.
In contrast to the classical scheme of statistical identification, due to the above reasons, it is impossible to construct a distribution function of the sample indicator, and, therefore, it is impossible to correctly assess the level of confidence in decision. The only option for constructing a dynamic structure recognition system is an empirical approach that uses heuristic schemes for constructing decision-making rules followed by numerical testing on large datasets of retrospective data. The size of the specified dataset should be large enough so that the indicator can be repeatedly tested on various structures of chaotic dynamics and transition processes.
2.12. Stochastic Chaos Model: Decomposition and Analysis
In the traditional theory of control of dynamical systems [50, 51], the simplest model of direct observations is given by the additive relation as follows:where is a deterministic unknown process reflecting the true dynamics and is some noise component determined by the errors of observations.
In this case, the process is considered as a system component used in the process of forecasting the observed process and developing control decisions. It is assumed that the system component can be identified mathematically, for example, via regression analysis methods [52–55].
In the more general case, based on the Bayesian–Kalman paradigm [12–14], the trajectory of motion is described by a random process as follows:where is the transition matrix and is the random component, the so-called noise of the system. As in the previous case, the noise of the system is usually modeled by a stationary Gaussian process. The task of filtering in this case is to isolate the system process , which is a conditional average of the observed random process .
It is obvious that such an interpretation of the components of the observation model is unsuitable for representing quotation series. In the conditions of market chaos, there are practically no observation errors, the entire stochastic component is completely determined by the noise of the system. The observed process fully (up to the selected rounding digit in the digitization process) corresponds to the true dynamics of quotations. At the same time, the noise component of the system cannot be mechanically discarded by smoothing, because its values are meaningfully inseparable from a number of quotes. Therefore, the modified two-component additive representation of considered process (1) in this class of applied problems requires a qualitatively different interpretation.
In the considered problem, the system component is understood as a smoothed quasiregular process used to determine a trend, forecast, or directly in the process of decision-making, and is a random component of the observation model formed by residuals , where is a smoothed curve formed by one of the sequential filtering algorithms [15–20].
Note that with such a definition, there is a fundamental uncertainty associated with the very concept of a system component. In essence, the division of the observed series into a system process and a noise component in this case turns out to be ambiguous and requires additional definition related to such subjective factors as the choice of the filtering algorithm and the criterion of its effectiveness. The process of identifying the system component is decisively influenced by strategic preferences of participants in trading operations [3], which is an exogenous factor.
In essence, profit or loss in trading operations, one way or another, is associated with the choice of a trading strategy that makes management decisions based on ideas about subsequent changes in the values of quotes of trading assets. It is customary to distinguish between short-term strategies focused on trading operations during the day session, medium-term (from several days to several months) and long-term (annual or multiyear) strategies that are close to the investment management. It is obvious that the dynamic trends that determine management decisions will be perceived differently. This means that the tasks of identifying the system component of the series of observations will be different. In particular, intraday fluctuations are insignificant for long-term trading operations, but they completely determine the effectiveness of short-term daily speculations.
Thus, the division of the general dynamics of quotations into systemic and random is conditional, determined by external factors and requires clarification when approaching each specific task. It follows that the random component, which is noise in relation to the selected system component, may be completely nonrandom from the point of view of traditional statistical criteria of randomness and independence of observation series.
In other words, the stochastic component that needs to be filtered out to identify the system component of the dynamics of quotations that is fundamentally important for trading, by its statistical nature should not be and often is not “noise” in the traditional sense of the probabilistic-statistical paradigm. In particular, the random component may not be a stationary and normal process and, as a rule, contains its quasisystem component determined by conditional mean.
3. Results
3.1. Statistical Analysis of the Structure of Chaotic Process Observation Series
The formalized division of observation series into system and random components can be carried out by sequentially complicating the model of the system component until the residuals turn into a stationary noise process. With this approach, it turns out to be expedient to present the initial process in the form of an additive three-component observation model, which includes a system component used in the process of making trading decisions, a quasisystem component with higher-order variation dynamics, and a purely random component forming a stationary time series. However, the implementation of such a concept in practice encounters specific difficulties characteristic of the process of price dynamics of trading assets. In this case, sequential computational schemes of dynamic filtering are most often used [10, 12, 15–20].
Let us consider filtering random error in the process of sequential fitting of a series of observations on a one-day observation interval. To this end, we will consistently increase the degree of the approximating polynomial simulating the system component to the level until the statistical criteria of agreement confirm the random nature of the corresponding discrepancies between the model and real observations. Figure 6 shows the plots of changes in quotations of the DJ index and its polynomial approximations (1st, 2nd, and 3rd degrees) for two one-day trading sessions, as well as Figure 7 shows the corresponding approximation residuals. Data used have the step of 30 s and numbers of counts are used as an argument at X-axis.
(a)
(b)
(a)
(b)
The independence and stationarity of the series of approximation residuals is tested via known statistical criteria [52, 56]. To test the hypothesis about the stochastic independence of an observation series, we use the median series criterion [56]. For the selected example, at (the number of minute counts during the operation of the New York Stock Exchange), the critical values are, respectively, equal to .
The corresponding values of the decision statistics and the decision to accept the hypothesis of the independence of regression error series for three trading sessions on the New York Stock Exchange are shown in Table 1. The value 1 in the column means that the hypothesis does not contradict the data and 0 denotes that the hypothesis is rejected.
It can be seen from the above data that the hypothesis of the independence of the considered observation series is not rejected in most cases. It is quite sufficient to use approximating polynomials of order 2-3. A slight decrease in the quality of estimation at is explained by the poor conditionality of the matrix of normal equations at high orders of the approximating polynomial.
It should be noted that this criterion cannot check the stationarity of the considered processes. So, for example, if the amplitude of the first half of the time series shown in Figure 3 is artificially increased, and the second half is left unchanged, then the parameters and will not change, the hypothesis about independence will not be rejected, and the condition of stationarity of the process will obviously not hold.
Checking the independence and stationarity of evaluation residuals is not an end in itself. The main task is to identify the system component that is significant for the process of making trading decisions. The instability of the residuals due to variations in scatter parameters is not significant. The main role here belongs to variations of the mean, which form trends with mixtures of aperiodic and oscillatory processes.
The criterion of ascending and descending series can be used [56] to assess the independence of such series of observations. To verify this test computationally, we use the same data for three trading sessions on the New York Stock Exchange as in Table 1. However, we will not use approximations by polynomials of degree , because they lead to the matrix of normal equations degenerating. The results are shown in Table 2. For the given data, the critical values are, respectively, and .
Thus, in estimating the parameters of a regression model based on a complete sample of observations, the previously made assumption about the presence of a third, quasisystemic component in the dynamics of the considered process proved to be infeasible. The random component remaining after the estimation is a sequence of independent samples. Of course, this does not mean that this component forms a stationary process (either in a restricted or broad sense of this definition). In particular, from the above plots in Figure 6, one can see the presence of a system process that is oscillatory, but nonperiodic.
The usage of traditional criteria for independence verification does not encompass this process, leading to the introduction of chaotic dynamics into the initial process. As a result, predicting the development of situations in currency, stock, and other markets becomes ineffective.
However, before studying this issue by analytical or numerical methods, it is necessary to bring the process of assessing the state of the stock market closer to practice. The approach discussed above is only theoretical. This is due to the fact that the estimation process was carried out based on joint processing of data observed during the entire trading session. In practice, the decision is formed sequentially, as data becomes available using a sample of increasing volume, or based on a sliding observation window. In this regard, we will reanalyze the residuals of regression estimates taking into account the last remark. To do this, we will use a sequential regression estimation scheme based on a sample of increasing volume for various sizes of the sliding observation window used to calculate the transfer coefficient of the LSM (least squares method) filter.
Figure 8 shows the graphs of sequential LSM evaluation for DJ index quotes for three working days (hereafter the time intervals corresponding to the time when the trading platform was not working are “cut out”) and one working day.
(a)
(b)
It is not difficult to see that an attempt to use all the accumulated information to estimate current values significantly increases the inertia of the estimation process and leads to a significant shift. For a sliding observation window, the quality of data recovery, as can be seen from Figure 9(a) (sliding window of minute counts) and Figure 9(b) ( counts) is much higher. However, in both cases, the main problem of qualitative restoration of the system component remains the delay in approximation relative to the observed process, which ultimately leads to a fundamental decrease in effectiveness. As expected, the growth of the window increases the degree of smoothing and at the same time leads to an increase of delay.
(a)
(b)
For the case of estimating by a sliding sample of observations with the size of minute counts, we will construct one-day sequences of errors of regression estimates and check them for independence using the criteria given above. Plots of these errors with sliding observation windows of the size, respectively, and minute counts are shown in Figure 10.
It can be seen that in the above circumstances, simulating real sequential processing, and not the idealistic scheme of joint data processing described at the beginning of this subsection, there is a clearly expressed quasisystem component, the existence of which has been suggested. This component becomes more pronounced as the sliding observation window grows and the delay caused by it in forming an assessment of the current state of the stock market indicator.
To strictly confirm this observation, we will test the statistical hypothesis about the independence of the selected series of observations using the criterion of ascending and descending series [56]. The critical values of this criterion for this series are equal to and . The numerical values of the decision statistics for the observation window of are and , for the observation window , they are and . Thus, in accordance with the chosen criterion and the level of confidence , the hypothesis of the independence of the observation series is rejected, which confirms the presence of a regular component in the series of residuals.
To reconfirm this conclusion, we used the median series criterion [56]. The critical values in this case are, respectively, and . The values of the decision statistics for the same confidence level for the observation window are and , for the observation window they are and . Thus, according to the median criterion, the hypothesis of independence is also rejected in both cases.
The fundamental point of the obtained conclusion is that the quasisystem component virtually does not contain a pronounced linear trend, as it is oscillatory and nonperiodic. Dynamics of this kind are extremely difficult to adjust with traditional computational schemes of joint data processing. The concept of adaptation also does not allow for effective implementation due to the fact that nonperiodic fluctuations are associated with a delay in evaluation. If we add to them the time required to identify changes in the dynamics of the controlled process, then the constructed adaptation loop will adapt to the already outdated dynamics and, in fact, only worsen the accuracy of the estimation. The most constructive way out of this situation is an attempt to use robust evaluation schemes that are resistant to the influence of the identified quasisystem component.
In conclusion of this section, we will consider the possibility of reducing the level of the regular component in residuals series by increasing the degree of the approximating polynomial. As an example of numerical analysis, we will consider the same parameters as in the previous case. The only difference is that a different trading session was chosen. The order of the approximating polynomial was chosen to be 2 (Figure 11(a)) and 3 (Figure 11(b)), respectively.
(a)
(b)
The independence test showed that the values of the decision statistics of the criterion of ascending and descending series in the first case are equal to and , and in the second case, and . The decision statistics for an approximating polynomial of order 2 and the median series criterion in a window of are, respectively, and , and for the observation window , they are and . Choosing the confidence level , the threshold values of the decision statistics are, respectively, and , and, consequently, the hypothesis of stochastic independence is rejected. For the observation window, we have the values of decision statistics equal to and , and at those critical values, the hypothesis of stochastic independence is also rejected. Similar results are obtained when using an approximating polynomial of order 3.
Note that in addition to these criteria, the Abbe criterion can be used to verify independence [53, 56]. However, its use presupposes a preliminary test of whether the errors are Gaussian. Under these conditions, this test leads to a negative result, which violates the condition of applicability of this criterion. In this case, the normality analysis used a criterion based on the evaluation of the values of asymmetry and kurtosis, and the normality condition was not fulfilled due to large kurtosis.
3.2. Three-Component Model of Quotation Observation Series
From the above studies, it can be concluded that the initial process can be represented as a three-component modelwhich includes:(i)A system component used for making trading decisions and consisting of a complex smoothed nonlinear process with pronounced trends and fluctuations(ii)A quasisystem noise component , which is an unshifted oscillatory nonperiodic (chaotic) process(iii)A stationary random process with its distribution, as will be shown below, tending to the Gaussian law.
The graphically described representation can be illustrated by the curves shown in Figures 12–14.
It is important to point out once again the subjectivity of the process of identifying the system component, which is based on the trader’s general considerations about the degree of smoothness of the initial series in the interests of ensuring the process of making effective trading decisions.
It is obvious that this parameter will be largely determined by the subjectively chosen trading strategy. Next, in Figure 13, the quasisystem component of the model (3) is given against the background of residuals between DJ quotes and the selected system component. To determine it, one can reuse the LSM approximation or some simpler and faster way to estimate the conditional average.
In particular, in this case, an exponential smoothing algorithm [11] with a transfer coefficient was used. This process is also a variant of chaotic dynamics, i.e., an oscillatory nonperiodic process, but it no longer contains pronounced trends and is well centered.
Figure 14 shows the third purely random component of model (3), formed by the difference between the residuals of the system component and the values of the quasisystem component. It is not difficult to see that this process is close to stationary Gaussian noise. The criteria given in [14] were used to verify this fact.
It should be noted that in practice, the normality criterion may not be fulfilled, since the smoothing used to identify the quasiregular model leads, as already noted above, to a noticeable kurtosis. Nevertheless, an approximate description of the distribution of the random component of the Gaussian curve may be acceptable for many applied tasks.
4. Discussion and Conclusion
The studies presented in this article are devoted to an attempt to understand and formalize the structure of observation series of chaotic processes. The most complex type of chaotic movement generated by the inertia-free and unstable information submersion environment is considered. The most striking example of such an environment is the modern electronic capital markets: stock, currency, and commodity markets.
The fundamental difference of the processes taking place in these markets is their inertia-free nature. Any known chaotic processes associated with material and energy processes, such as turbulent gas and hydrodynamic flows, do not allow instantaneous abrupt changes in dynamics due to their physical nature, namely inertia. It is known [4] that the dynamics of nonlinear open systems with a dimension greater than three described by systems of nonlinear differential equations, can contain bifurcation points in which the solution, i.e., the integral curve of the evolution of the state of the system, is parametrically unstable. This means that even extremely minor external disturbances can lead to radical changes in the dynamics of the observed process. This phenomenon, which has received the name of deterministic chaos, is fully feasible for the task of monitoring the pricing process in capital markets. For physical processes, a perturbation at bifurcation points means a relatively smooth (without discontinuity of the derivative) transition to another poorly predictable trajectory of evolution. There are in many cases abrupt changes in the series of observations, which significantly complicates computational procedures for identifying and predicting such processes for intangible information processes that reflect the current perception of market participants about the value of an asset.
This phenomenon in itself creates new problems for predicting the dynamics of the observed process. However, it is not difficult to see that in addition to dynamic uncertainty, data that reflects changes in market asset quotations contain statistical uncertainty integrating the influence of a large number of different latent factors of influence from the immersion environment. As a result, the best interpretative model is the additive two-parameter Wold model (1), in which the smoothed system component is determined by the dynamic chaos model, and the random component is determined by a stochastic process integrating the influence of all hidden factors.
Traditional observation models used in data analysis tasks are based on the assumption of stationarity and Gaussianity of the random component of the observation model. However, as the analysis of the probabilistic nature of the series of observations of quotations of market assets shown in the article, these hypotheses are not fulfilled. This is due to the fact that the immersion market environment is characterized by periodic amplifications of various factors of influence, which violates the Lindenberg condition of the central limit theorem [57, 58], and, as a consequence, leads to a violation of Gaussian conditions. Using well-known statistical criteria, it is shown that other important characteristics of the random component of the data model, such as stationarity and independence of the series of observations, are also not fulfilled.
It follows from this that as an interpretive model of the processes considered in this article, it is advisable to use a three-component scheme (3), in which a quasisystem component formed by a nonstationary random process is added. This model provides a visual representation of the structure of the observed processes. However, the question of the possibility of optimal or quasioptimal forecasting or proactive control algorithms based on it remains open.
Nevertheless, the materials obtained in the article allow us to draw a number of useful conclusions for practice:(1)The presence of a nonstationary random component clearly indicates that any traditional methods of statistical analysis will be suboptimal, and in some cases infeasible. This does not mean there is complete infeasibility. So, LSM in any case will provide an estimate that minimizes the sum of squared deviations. However, the resulting shift will significantly reduce the effectiveness of proactive management.(2)The presence of inertia-free discontinuous nonperiodic changes significantly complicates the possibility of applying any forms of adaptation of forecasting algorithms and proactive management. It is obvious that the inertia of the feedback loop ultimately forbids monitoring sequences of abrupt changes in observation series. Increasing the sensitivity of the feedback loop can lead to even more negative consequences associated with the system response of the control loop to random fluctuations generated by the third component of model (3).(3)It appears that robust estimation [35–41] is a more promising approach to increasing the stability of proactive management algorithms in conditions of stochastic chaos. This class of techniques proposes the development of algorithms with reduced sensitivity to variations in input data. In particular, it can be assumed that the probabilistic structure of the nonstationary process of quotations of market assets is described by a set of distributions from a given limited class of feasible distribution functions. The law which maximizes Fisher information is selected as the most unfavorable distribution for a given class. In this case, using the minimax approach, the forecasting algorithm is formed as the best solution (for example, according to the maximum likelihood criterion) for the previously selected “least favorable” distribution. It is obvious that the effectiveness of such an approach for stochastic chaos conditions is extremely difficult to assess analytically and numerical methods are the main tool for investigating this issue.(4)The revealed difficulties in the data structure allow us to conclude that it is advisable to solve the task of managing assets circulating in the capital markets indirectly, dividing it into two stages. The first stage is formalized quantitative analysis of the state of the entire market segment, including the managed asset, as well as other factors of the immersion environment that affect the state of this asset and whose quantitative characteristics can be monitored. The first stage should be carried out continuously, giving proactive indications of the need to change the management strategy. At the second stage, based on a sequential analysis of the data structure, a selection or automated generation of a predictive management strategy is carried out, used for a fairly limited time interval.(5)The structure of processes can be identified using various indicators (including the parameters of trends and channels). At the same time, the question of the effectiveness of individual indicators for different areas of the observed processes remains open. A formalized analysis of the quality of such indicators in conditions of nonstationary dynamics is extremely difficult, and numerical examples do not guarantee their effectiveness in all the variety of situations characteristic of statistical chaos. This leads to the conclusion about the expediency of using a multiexpert approach, the concept of which is given in [44]. An expert is primarily understood as software data analysis tools used in the construction of trading robots. Further development of this approach will be focused on hybrid algorithms based on the principles of artificial and hybrid intelligence. The conclusions set out in paragraphs 1–5 are preliminary and require numerical verification. They will be the subject of our further research.
Data Availability
The source of data is Finam.ru (https://www.finam.ru/, accessed on February 28, 2023).
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
The authors are grateful to participants at the Center for Econometrics and Business Analytics of Saint-Petersburg State University (https://ceba-lab.org, CEBA) seminar series for helpful comments and suggestions. The research was supported by the Center for Econometrics and Business Analytics and the Endowment Fund of St Petersburg University.