Abstract
Science and technology innovation (STI) policy is a strategic principle to guide the whole cause of STI. The study on STI policy and its effect is particularly important. Most of the existing studies on the effect of STI policy focus on the effect of a single policy, and the studies on the effect of policy combination and its differences need to be further enriched and improved. This study proposes a method combining system simulation experiment and analysis of variance (ANOVA) to study the differences of combination effects of STI policies. The results show that there are significant effect differences in the combination of STI policies as a whole, but when it comes to different combinations of STI policies, not all policy combinations have significant differences. This study not only points out whether there are significant differences in a certain effect among which combinations of STI policies but also points out whether there are significant differences in all effects among which combinations of STI policies at the same time. This study has theoretical and practical significance for realizing scientific policy-making and sustainable development.
1. Introduction
STI is the inexhaustible driving force for the development of modern countries and the fundamental driving force for realizing sustainable economic development. At present, the improvement of STI ability and the high-quality development of STI have become the theme of the times. Nowadays, countries all over the world attach great importance to their own STI. Effective innovation and high-level patented technology have become important indicators to measure the level of national economic development and comprehensive national strength. In order to accelerate the pace of STI, countries all over the world have formulated STI policies to promote the development of innovation activities and the transformation of achievements. STI policy is the basic action criterion stipulated by a country to realize the STI task in a certain historical period. It is the strategic principle to determine the development direction of STI and guide the whole STI [1]. STI policy can ensure the effective implementation of various innovation activities and the rational allocation of innovation resources. Its reasonable formulation will help to improve innovation performance, so as to promote the construction of national innovation system. As a key factor to strengthen the national scientific and technological strategic force and drive the development of STI, STI policy has attracted extensive attention in the academic circles and has become an important research topic [2].
After studying the existing literature, it is found that the research on policy effects (including STI policy effects) related to this research is mainly reflected in the following two aspects:(1)What methods are used to study policy effects? The existing research methods on policy effects (including STI policy effects) are divided into two categories: qualitative research methods and quantitative research methods. Most of them analyze the impact of policies from a qualitative perspective. For example, Surianto et al. studied why policy efforts to reduce disaster risk often fail to improve future disaster relief [3]; Krivosheev et al. studied the effectiveness of EU policy in Walloon, Belgium, and the economic development of the region compared with Flanders, another Belgian region [4]. Based on the current situation of industrial transfer in Guangdong Province, Shi et al. discussed the role of policies in promoting industrial transfer and existing problems, and put forward suggestions on the innovation of regional policies in Guangdong Province [5]; Yu [6] and Lan et al. [7] studied the impact of policies on regional economic development and believed that the government should formulate regional economic policies in line with its own regional economic development direction according to the economic development of each region, so as to scientifically and reasonably allocate the resources of each region; Lin studied the impact of EU policies on Portugal’s national STI capacity [8]. In addition, a few of them analyze the impact of policies from a quantitative perspective by establishing models. Wang et al. used system dynamics to build a mechanism model of the impact of STI policies on regional innovation capability and studied the impact mechanism of different policies on regional innovation capability [9]; Avdiushchenko and Zając proposed a set of possible indicators to evaluate the progress of EU countries in realizing circular economy at the regional level [10]; O’Brien and Burrows used vertical and mixed methods to analyze the effectiveness of regional policies for large-scale layoffs [11]; Gülal and Ayaita took the introduction of minimum wage in Germany as the standard experiment and used the double difference model to analyze the impact of minimum wage on well-being [12].(2)Research on the effect of single policy: most of the existing research on the effect of STI policy is to study the effect of a single STI policy. Cheng et al. found that government research and development (R&D) investment and human investment have a positive effect on STI, while enterprise financing policy has a significant negative effect on STI [13]; Deng and Long [14] and Yu [15] pointed out that R&D investment has a positive impact on STI; Fan [16] and Wang and Deng [17] believed that intellectual property protection helps promote STI; Zhuang et al. emphasized that the environmental policy of STI has a significant impact on STI [18]; Álvarez-Ayuso et al. studied the impact of tax credit on R&D [19]; Raffaello and Paolo studied the innovation effect of R&D subsidy policy in northern Italy [20]; Mukherjee et al. studied the relationship between taxation and innovation and pointed out that taxation hinders innovation [21]; Cappelen et al. believed that tax incentives have little effect on the marketization of new products and patented products, but they are conducive to the innovation of product production process and the formation of new products [22]; Guerzoni and Raiteri took enterprises in EU Member States as samples, and considered that government procurement not only stimulates enterprises’ R&D investment, but also stimulates innovation output [23]; Aschhoff and Sofka empirically tested the effect of government procurement on innovation output by using the survey data of German enterprises and the results show that government procurement can promote enterprise innovation [24]; Song and Zhang studied the relationship between government procurement policies and implementation rules to promote independent innovation, combed the practices in typical countries, and pointed out that government procurement is an important policy tool to support the innovation [25]; Hu et al. used provincial panel data to conduct an empirical test on whether China’s government procurement has produced the expected technological innovation effect and the results show that China’s government procurement not only does not promote innovation, but hinders innovation [26]. At present, there are also a few studies on the combination effect of STI policies. Kalcheva et al. used the triple difference method to study the impact of the combination of supply policy and demand policy on innovation [27]; Dou et al. found that the combination of STI policies has a significant impact on enterprise technological innovation [28]; Guo et al. believed that the combination effect of various STI policies is better than a single type of policies. When the policies are issued, we should pay attention to the complementarity and systematicness of the policies and avoid focusing too much on a certain link [29].
To sum up, the existing research on the effect of STI policy is mainly the effect of a single policy, and the research on the combination effect of STI policy needs to be further enriched and improved. In fact, the effect of STI policy is often the result of the joint force of multiple policies. At the same time, the existing research on the combination effect of STI policy needs to pay more attention to the research on the difference of policy combination effect. The detailed analysis of the difference of policy combination effect is helpful for researchers and policy makers to have a deeper understanding of the effect of policy combination, and it is also helpful for scientific policy implementation. In view of this, this study will use the ANOVA method to study the differences of the combination effects of STI policies.
The above is the first part of this study, that is, the introduction. This study will be carried out according to the following structure: firstly, the research methods used are introduced, that is, the one-way ANOVA method and multiple comparison method are introduced; secondly, the differences of policy portfolio effects are analyzed in detail by using one-way ANOVA and multiple comparison method; the last part is the summary of this study.
2. Relevant Theoretical Basis
Because STI has the characteristics of externality and risk of public goods, it will hinder the enthusiasm of the main body of STI, and lead to the fact that the supply quantity in the market is often difficult to reach the social optimal level. This provides a theoretical basis for the government to stimulate the innovation power of STI subjects through policies [30]. Market failure theory holds that in the case of market failure, it is difficult to achieve Pareto optimal state only by market domination. At this time, it is necessary to combine the market with the government. R&D and innovation activities have the characteristics of externality and risk of public goods to a certain extent, which will lead to the failure of STI market, and provide a theoretical basis for the government to intervene in the R&D activities of STI subjects. Endogenous growth theory is based on Schumpeter's innovation theory, takes technological progress as an endogenous variable, and believes that human capital and knowledge accumulation bring the increasing marginal rate of return, so as to ensure economic growth and social progress. These endogenous factors that bring about the economic growth will be affected by government policies and are sensitive to policies. Romer [31] pointed out that we should vigorously promote various policies to promote innovation, such as giving direct subsidies to knowledge output, paying attention to education, strengthening intellectual property protection, etc. Through these policy orientations, we should send signals to STI subjects and stimulate their R&D enthusiasm.
3. Research Method: One-Way ANOVA and Multiple Comparison
ANOVA is used to test the significance of the difference between the mean of two or more samples. Its basic idea is as follows: by analyzing and decomposing the fluctuation of experimental data, and then comparing the possible systematic fluctuation and random fluctuation between each group of experimental data under a certain influencing factor, it is inferred whether there is a significant difference between the overall mean values. If there is a significant difference, it indicates that the influence of this factor is significant; otherwise, it is not significant [32, 33]. When the influence of only one factor on the experimental results is considered, it is called one-way ANOVA; when two factors affect the experimental results, it is called two-way ANOVA; by analogy, when more than two factors affect the experimental results, it is called multiway ANOVA.
The multiple comparison method in the parameter method is to test which overall mean values are equal and which overall mean values are different through pairwise comparison between the overall mean values. There are many methods for multiple comparison. In this study, the least significant difference (LSD) method proposed by Fisher is used. It is a simple deformation of t-test. It makes full use of sample information in the calculation of standard error and uniformly estimates a more robust standard error for the mean of all groups [34].
3.1. Related Concepts and Basic Assumptions of One-Way ANOVA
The one-way ANOVA method involves three terms: factor, level, and observation. Their meanings are as follows:(1)Factor, also known as condition, refers to the object to be tested in the ANOVA(2)Level, also known as treatment, refers to different values corresponding to factors(3)Observation, refers to the experimental data obtained at each level of the factor
The one-way ANOVA method has the following basic assumptions:(1)The assumption of normality is that each population obeys the normal distribution. For each level of factors, the observed values are simple random samples from the normal population.(2)The homogeneity of variance is assumed, that is, the variance of each population should be equal. For each group of observation data, they are extracted from the normal distribution of the same variance.(3)Independence assumption, that is, the observations are independent of each other.
3.2. Basic Process of One-Way ANOVA
When the one-way ANOVA method is used for research analysis, the basic process shown in Figure 1 can be adopted.
Firstly, the null hypothesis and alternative hypothesis are put forward. If the factor to be tested has k levels, and the corresponding mean value of each level is ui, where i = 1, 2, …, k, then the null hypothesis and alternative hypothesis are as follows:
H0: u1 = u2 = … = uk (independent variables have no significant impact on dependent variables);
H1: u1, u2, …, uk are not all equal (independent variables have a significant impact on dependent variables).
The null hypothesis (H0) means that the factor to be tested has no impact on the experimental results, while the alternative hypothesis (H1) means that the factor to be tested has an impact on the experimental results.
Secondly, the test statistics are constructed and calculated. In order to test whether the null hypothesis (H0) is true, it is necessary to construct appropriate test statistics first. Specifically, it is necessary to construct three sums of squares of error, which are sum of squares of total error (SST), sum of squares of factor error (SSA) and sum of squares of random error (SSE). Let xij represent the j-th observation value of the i-th level, then the calculation formulas of these three sums of squares of error arewhere represents the mean value of the factor at Level i and ni is the number of the i-th overall experimental data, i = 1, 2, …, k; represents the mean value of all observations, n = n1+n2++nk.
These three sums of squares of error satisfy the following identity relationship.
The sum of squares of error divided by their corresponding degrees of freedom is called mean square error. The degree of freedom of SST is n − 1, where n is the number of all observed values. The degree of freedom of SSA is k − 1, and its corresponding mean square error is usually recorded as MSA, where k is the number of factor levels. The degree of freedom of SSE is n − k, and its corresponding mean square error is usually recorded as MSE. Thus, the value of F = MSA/MSE can be obtained, and it obeys the F (k − 1, n − k) distribution.
Finally, make statistical decisions. According to the given significance level α, check the F distribution table to determine Fα (k − 1, n − k). If F > Fα (or P < α), then reject the null hypothesis, indicating that the factor has a significant impact on the experimental results. Otherwise, accept the null hypothesis and consider that the factor has no significant impact on the experimental results.
3.3. Basic Flow of LSD Method
When the one-way ANOVA method is used for research analysis, the basic process shown in Figure 1 can be adopted.
After one-way ANOVA, if you want to know which population means are equal and which population means are different, you need to make multiple comparisons. In this study, LSD method is used for multiple comparison, and its basic steps are similar to one-way ANOVA, as follows:
Firstly, the null hypothesis and alternative hypothesis are put forward.
H0: ui = uj;
H1: ui, uj are not equal where 1 ≤ i < j ≤ k.
Secondly, the test statistics are constructed and calculated. The test statistic of LSD method is t statistic, and its calculation formula is
t statistic obeys the t distribution with n − k degrees of freedom.
Finally, make statistical decisions. According to the given significance level α, check the t distribution table to determine tα/2 (n − k). If |t| > tα/2(n − k) (or P < α), then reject the null hypothesis, indicating that there is a difference between the two population mean values. Otherwise, accept the null hypothesis and consider that there is no difference between the two population mean values.
3.4. Welch’s ANOVA and Games–Howell Test
It is worth noting that in this study, when the variances are unequal, we will use Welch’s ANOVA method to replace the aforementioned one-way ANOVA method for corresponding analysis, because Welch statistic is better than F statistic when the variance is unequal [35]. At the same time, we will use the Games–Howell test method to replace the above LSD method for corresponding analysis. For a detailed introduction to the Games–Howell test method, please refer to reference [36].
4. Analysis on the Differences of Combination Effects
This study is based on our previous research achievement “Analysis on the combination effect of science and technology innovation policy: from the perspective of system simulation” (hereinafter referred to as Achievement A), which proposes a system simulation method to analyze the combination effect of STI policy. This study uses the system simulation model in Achievement A to carry out simulation experiments and obtain simulation experimental data, so as to analyze the differences of policy combination effects. The following is a brief introduction to the system simulation model in Achievement A.
Rothwell and Zegveld [37] believe that the policy effect originates from different levels. Therefore, the STI policy can be divided into three categories, one is supply policy, and the other two are demand policy and environmental policy. In this study, our goal is to analyze the impact of different level STI policies. Therefore, this classification method will be adopted. Considering the quantification of policies and the availability of data, the STI supply policy in Achievement A only considers the capital investment policy and talent training policy, the STI demand policy only considers the intellectual property policy and STI service policy, and the STI environment policy only considers the enterprise financing policy and enterprise tax policy. The Achievement A establishes the conceptual model as shown in Figure 2, that is, the causality diagram. The system simulation model corresponding to the conceptual model, that is, the stock flow diagram, is shown in Figure 3.
The above is a brief introduction to the system simulation model in Achievement A. For the detailed introduction of Achievement A, please refer to my previous research Achievement A.
It is worth noting that Achievement A has conducted an empirical study using the relevant data of STI in Guangdong Province from 2010 to 2019. Its system simulation model does not consider the uncertain factors, but this study will take the uncertain factors into account. Specifically, this study will add random distribution to the expression of relevant variables in Achievement A, as shown in Table 1. For example, for the annual GDP, the expression in this study is: annual GDP = the expression of annual GDP in Achievement A + RANDOM NORMAL (−5460.12, 5291.63, 0, 3622.37). Among them, RANDOM NORMAL(min, max, mean, stdev) is the normal distribution form in the system dynamics simulation software VENSIM, which needs to input four parameters.
It can be seen from Table 1 that all the random distributions in the expressions of Variable 1 to 9 are a normal distribution, because the expressions corresponding to these variables in Achievement A are obtained by linear regression analysis method, which requires that the error term obey a normal distribution and the mean value is 0. In Achievement A, the expressions of Variable 10 to 13 are obtained by weighted regression analysis. This study assumes that the error terms of variables 10 to 13 obey a normal distribution. The parameters of the random distribution shown in Table 1 are obtained by analyzing the error term.
This study will make a difference analysis on the combination effects of the above three types of policies in SPSS statistical analysis software. The effects here mainly refer to the economic effect and STI effect. The per capita GDP and patent application acceptance are used as the effect indicators respectively. In this study, each type of policy takes three levels. Specifically, the three levels of supply policy are benchmark level, relative benchmark increase of 5% and relative benchmark decrease of 5%. The three levels of demand policy are similar to those of supply policy. The three levels of environmental policy are the benchmark level, the relative benchmark increase of 1% and the relative benchmark decrease of 1%. The benchmark levels for each policy are detailed in Achievement A. Since there are three types of policies, and each type of policy has three levels, this study will design 33 = 27 simulation experiments, as shown in Table 2.
4.1. Descriptive Statistics
After the preliminary analysis of the data (that is, the model output data with the simulation time of 2025) obtained from the above simulation experiments, some descriptive statistical analysis results are obtained as shown in Table 3. It is not difficult to find from Table 3 that the number of experiments in each experiment is the same, 50 times. From Table 3, we can also see the mean value and standard deviation of per capita GDP (PCGDP) and annual patent application acceptance (APAA) of each simulation experiment.
4.2. Test of Normality and Test of Homogeneity of Variance
Because the one-way ANOVA method requires that the population should meet the normality and homogeneity of variance, it is necessary to test the normality and homogeneity of variance before analyzing the differences of policy combination effects. It can be seen from Table 1 that the normality requirements are met. From Table 4, it can be seen that the significance (P value) of per capita GDP is greater than 0.05. Thus, the homogeneity of variance is met at this time. For the annual patent application acceptance, the significance (P value) is less than 0.05. Hence, the homogeneity of variance is not met at this time.
4.3. Difference Analysis of Economic Effect
For the per capita GDP, since each population meets the requirements of normality and homogeneity of variance, we use the abovementioned one-way ANOVA method for analysis. As can be seen from Table 5, the significance (P value) is less than 0.05, which indicates that there are significant differences in the economic effect (i.e., per capita GDP) of various combinations of STI policies. It is worth noting that this significant difference refers to the significant difference of the whole. At this time, we do not know which combinations have significant differences, and which combinations have no significant differences. In order to answer this question, it is necessary to make multiple comparisons so that we can clearly understand which combinations have significant differences. It can be seen from Table 6 that when 0.05 is taken as the significance level, there is a significant difference between the economic effect of the policy combination corresponding to Experiment 1 and the economic effect of other policy combinations, except that it is not significantly different from the economic effect of the policy combination corresponding to Experiment 7, 10, 13, 19, and 25. In addition to the policy combination corresponding to Experiment 1, similar difference analysis can also be carried out for other policy combinations in combination with Table 6, which will not be repeated here.
4.4. Difference Analysis of STI Effect
For the annual patent application acceptance, although each population meets the requirements of normality, they do not meet the requirements of homogeneity of variance. Therefore, the abovementioned one-way ANOVA method is not applicable here. At this time, we can use the abovementioned Welch’s ANOVA method for analysis. As can be seen from Table 7, the significance (P value) is less than 0.05, which shows that the STI effect (i.e., annual patent application acceptance) of various combinations of STI policies are significantly different as a whole. In order to further understand which population mean values are equal and which population mean values are different, it is necessary to make multiple comparisons using the aforementioned Games–Howell test method. It is not difficult to find from Table 8 that, at the significance level of 0.05, there is no significant difference between the STI effect of the policy combination corresponding to Experiment 1 and the STI effect of the policy combination corresponding to Experiment 4, 7, 10, 13, 16, 19, 22, and 25. However, there are significant differences with other policy combinations in the effect of STI. In addition to the policy combination corresponding to Experiment 1, similar difference analysis can also be carried out for other policy combinations in combination with Table 8, which will not be repeated here.
4.5. Comprehensive Analysis of Differences
The above analyzes the differences of the economic effect and STI effect of each combination of STI policies. Next, we take policy Combination 1 as an example and analyze these two effects of each combination of STI policies at the same time. By sorting out Tables 6 and 8, we can get Table 9. It can be seen from Table 9 that the economic effect and STI effect of policy Combination 1 are significantly different from those of policy Combination 2, 3, 5, 6, 8, 9, 11, 12, 14, 15, 17, 18, 20, 21, 23, 24, 26 and 27, but not from those of policy Combination 7, 10, 13, 19 and 25. In addition, there are significant differences in economic effect between policy Combination 1 and policy Combination 4, 16 and 22, but there is no significant difference in STI effect.
5. Conclusions
5.1. Research Summary
STI has become an important driving force for national and regional economic and social development. As a tool for the government to guide and promote the promotion of scientific and technological competitiveness, STI policy has a significant impact on STI. Most of the existing studies focus on the effect of a single policy, and the studies on the effect of policy combination and its differences need to be further enriched and improved. Under this background, this study proposes a method combining system simulation experiment and ANOVA to study the differences of the combination effects of STI policies. The research is summarized as follows:(1)On the whole, there are significant differences in both the economic effect of STI policy combination and the STI effect of STI policy combination.(2)Specific to different combinations of STI policies, not all combinations of STI policies have significant differences in policy effects.(3)There are significant differences in the economic effect of some STI policy combinations, but the differences in the STI effect are not significant. There are significant differences in STI effect among some STI policy combinations, but the differences in economic effect are not significant.(4)This study not only points out whether there are significant differences in economic effect or STI effect among STI policy combinations, but also points out whether there are significant differences in economic effect and STI effect among STI policy combinations at the same time.
5.2. Policy Implications
This study gives us the following enlightenment:
Firstly, the empirical results show that there are significant differences in policy effects among some STI policy combinations, while there are no significant differences in policy effects among some STI policy combinations. Therefore, policymakers should fully consider the differences in policy effects among policy combinations when formulating policies, so as to avoid making useless efforts in formulating policies. In other words, policymakers should avoid formulating policies that have no significant difference in policy effects from the original policies.
Secondly, the empirical results show that the economic effect and STI effect of STI policy combinations are not always synchronous, that is, sometimes there are significant differences in one effect but not in the other. Therefore, when making policies, policy makers sometimes have to make a trade-off among a variety of policy effects.
5.3. Contribution and Prospect
The contributions of this study are as follows:(1)The existing research mainly focuses on the effect of a single policy, and the research on the combination effect of STI policy, especially the research on the difference of policy combination effect, needs to be further enriched and improved. This research enriches and expands the relevant literature research, makes up for the shortcomings of previous research, and provides an effective supplement for the existing research on STI policy.(2)This study proposes a method combining system simulation experiment and ANOVA to study the differences of the combination effects of STI policies. This method based on the combination of system simulation and data analysis and mining enhances the modeling and solving ability of practical problems, provides a reference framework for similar research, and reflects the characteristics of this study in research methods.
This study provides enlightenment for policy makers to fully understand the effect of STI policy and make appropriate adjustment. At the same time, it enriches and expands the relevant literature research, which has certain theoretical and practical significance. However, this study also has some limitations. For example, only six kinds of STI policies are considered, and other STI policies are not deeply studied. In future research, more STI policies can be included in the research.
Data Availability
The data used to support the findings of this study are included within the article.
Conflicts of Interest
The authors declare that there is no conflicts of interest regarding the publication of this paper.
Acknowledgments
This work was supported by the China Postdoctoral Science Foundation (grant no. 2021M693775).