Abstract
Feature explanation plays an increasingly essential role in the e-commerce search platform. Most of the existing studies focus on modeling the user’s interests to estimate the click-through rate (CTR). A good e-commerce system not only needs precise ranking to inspire users’ shopping desire but also needs feature explanation to meet the demands of shop owners. The e-commerce traffic health of merchants is very important. How to effectively achieve shop owners’ multiple goals still remains as an open problem. In industrial search systems, merchants’ key demands mainly include two aspects. On the one hand, merchants want to know rule analysis of online traffic distribution, so as to help them understand the logistics of online traffic. On the other hand, they need relevant online traffic participation tools, which instruct them to participate. To address these issues, we propose a factor marginal effect analysis approach (FMEA) based on game theory, which can compute the contribution of one-dimensional features to the enhancement of online traffic. First, we use machine learning to model the business target. Then, we improve the SHAP value algorithm, which can provide clear business insights. Finally, we calculate the marginal effect of each feature on the business outcome. In this way, we provide a traffic analysis guidance method and address merchants’ participation challenges. In fact, the FMEA has been deployed in a real-world Large-Internet-Company’s App search systems and successfully serves online e-commerce service to over hundreds of millions of consumers. Our approach can guide operational decisions effectively and bring +10.05% revenue for the flow index, +7.54% for the user feedback index, and +2.46% for the service index.
1. Introduction
Feature explanation has attracted increasing attention in both industry and academic communities. It can not only enhance users’ trust in the e-commerce search system but also help them make better and faster decisions. Murdoch et al. [1] discuss the definition of interpretability and how to select and evaluate the machine-learning models for interpreting. Guidotti et al. [2] summarize and classify the methods for interpreting black-box models in machine learning. Kohavi et al. [3] share how to develop experiment platforms to make it harder for experimenters to be misled, which is used to explain the intuition through solid statistical reasoning. Scott [4] introduces the core idea of the SHAP algorithm in game theory and how to apply it in feature interpretation. Shrikumar et al. [5] study how to calculate feature importance from SHAP perspective in game theory. Bai et al. [6] introduce attentional mechanism that applies in explainability. Chang et al. [7] studied the interpretability of generalized linear models. Explainable machine learning techniques [8] are employed to quantify the contributions of the impacting factors to the time efficiency, thereby identifying the fundamental causes. In [9–11], the interpretability of neural networks is studied. Qi et al. [12] introduce the interpretability of adversarial training. Ge et al. [13] propose an explainable fairness framework, which can discover critical features and calculate an explainability score for fairness. LiEGe (listwise explanation generator) [14] studies the problem of content-based explanation of search results in the two newly defined settings: novelty and comprehensive explanation generation. Rossi et al. [15] propose an explainability framework for embedding-based link prediction, which can be applied to any embedding-based LP model. In [16], session-based recommendation with interpretability, which is guided by meta-path and self-attention mechanism, is studied. Reference [13] is an interpretable research based on recommendation fairness. Nizri et al. [17] propose an automatic method that generates intuitive explanations for a SHAP-based payoff allocation, which provides customized explanations for SHAP values. Balog and Radlinski [18] propose evaluating explanations for item recommendations, which presents an analysis of intentions behind explanations. Tsukuda and Goto [19] propose an explainable recommendation method for repeat consumption, which designs nine explanation styles and validates the persuasiveness of these styles. A feature refinement network [20] is proposed, which learns context-aware feature representations at bit level to explain features in different contexts. Kunkel et al. [21] analyze both direct and indirect effects among constructs of major interest for RS research, which includes explanation quality, recommendation quality, social presence, and trustworthiness.
Despite effectiveness, these explainable recommender models still suffer from some limitations. It is difficult for these existing methods to provide quantitative deterministic guidance, such as how much one-dimensional feature can be improved and how much recommendation traffic can be improved. Further analysis is needed in conjunction with the data. Inspired by these limitations, we proposed a novel explainable feature quant algorithm based on game theory.
To summarize, our major contributions are listed as follows:(1)To the best of our knowledge, this is the first work that demonstrates the effectiveness of applying the explainable feature marginal contribution approach, which uses features of the shop owner’s concern for search system factor quantization.(2)The FMEA can compute how much one-dimensional feature contributes to the enhancement of online traffic, which is used to effectively guide merchant operations.(3)We design a second-order interpolation operator, which is used to transform the game theory SHAP value into the product cross probability. In this way, it can learn business interpretation better.(4)The case study can help the operator explain online bad cases and give reasonable diagnostic suggestions.(5)We verified the effectiveness of the algorithm in Large-Internet-Company data, which brings a revenue increase of +10.05% for the flow index, +7.54% for the user feedback, and +2.46% for the service index.(6)Our solution fits product rankings with explainable features, which map traffic scores to multiple dimensions. In this way, merchants can easily understand the position of a product in the industry and diagnose their problems. The purpose is to provide business operation suggestions, model the impact of operation intensity on factor changes, and guide traffic improvement.
The remainder of this paper is organized as follows. Section 2 provides a synopsis of related works. The preliminaries of our work are provided in Section 3. The framework concept and the proposed factor marginal effect analysis (FMEA) approach are introduced in Section 4. The dataset and experimental result analysis are thoroughly examined in Section 5. Finally, Section 6 contains an overview of the previous sections as well as some discussions in future work.
2. Related Work
In the era of big data, feature explanation [22] plays an important role in increasing product sales and assisting human decision-making. Interpreted modeling clearly tells businesses which factors have more influence on search traffic, so as to provide business diagnosis and guidance. Studies have shown that providing appropriate explanations [23] could improve user acceptance of the recommended items, as well as benefit user experience in various other aspects, including system transparency [24], user trust, effectiveness, efficiency, and satisfaction.
2.1. Feature Importance
Feature importance is a key tool for interpreting the constructed models and analyzing the relationship between features and labels. We define feature importance [25, 26] as any quantitative assignment of influence to the features used by machine-learning models. On the one hand, feature importance techniques attribute importance to a feature in relation to the model or its predictions. On the other hand, feature importance techniques produce explanations related to the business. Shapley value [4, 5, 27] is the weighted sum of a feature’s contribution to the total prediction over all possible feature combinations. This method approximates the Shapley value of each layer in the deep neural model and then calculates the contribution of features. Permutation feature importance [28] measures the significance by rearranging the features randomly on the dataset and then evaluating the rate of change in loss. TAYLOR expansion [29] is used to obtain the smooth derivative of the nonlinear loss function. The relevant method calculates the variance of the neuron weight change during the training process, which is employed to measure the feature importance.
2.2. Feature Explanation
The idea of interpretable machine learning [30] is to consider both the prediction accuracy and model interpretability and try to find the best balance between them. Based on different scenarios, the interpretability can be divided into intrinsic interpretability and ex post interpretability. An attentive recurrent neural network (Ante-RNN) [31] for the dynamic explainable recommendation is proposed, which can provide multimodel explanations according to the user dynamic features.
2.2.1. Intrinsic Interpretability
Intrinsic interpretability means that the structure of the model is relatively simple and the user can clearly see the internal structure of the model. The model has an interpretable effect at design time. Traditional statistical models (such as linear regression [32], logistic regression [33], and decision trees [34]) have strong interpretability. However, these models are less accurate. An explainable medical recommender system uses graph concepts to provide an interpretable approach [35] to medical data, which is based on community detection algorithms. Explainable boosted linear regression (EBLR) [36] for time series forecasting is proposed, which is an iterative method that starts with a base model, and explains the model’s errors through regression trees.
2.2.2. Ex Post Interpretability
Ex-post explainable [37] methods can better enhance the interpretability of a model and extract valuable information after training. The popular deep learning has complex internal structures. It is difficult to observe the changes of data neuron by neuron, and research on explainable machine learning has generated an enthusiastic response in both academia and industry. A novel method to explain black-box models [38] is proposed, which employs numeric association rules to explain and interpret multistep time series forecasting models. A new deep learning architecture xDNN [39] is proposed, which combines reasoning and learning in a synergy and explains its efficiency in terms of time and computational resources.
3. Preliminaries
In this section, we first formally define the problem and then provide several key notions relevant to our proposed solution. The motivation of this research is also provided.
3.1. Problem Definition
To improve efficiency, the existing ranking model in e-commerce search systems is complex, leading to a lack of understanding of traffic distribution strategies. The interpretable model can help us gain a deeper understanding of traffic distribution strategies and provide guidance and suggestions. Our solution enhances the explainability of business factors, with the goal of covering all factors. The explainable approach makes the subsequent estimation of operational effects more comprehensive and accurate. Compared with feature importance, the biggest advantage of SHAP value is that it can reflect the influence of each feature in each sample and also exhibit a positive or negative impact. Therefore, we choose to design our solution based on the SHAP value. For each sample, the model produces a predicted value, and the SHAP value is assigned to each feature of the sample. The target formula is as follows: represents the predicted value for the i-th sample, describes the baseline of the model, shows the contribution of the j-th feature of the i-th sample to the predicted value, and k represents the number of features. Based on the calculated SHAP values, we verify the distribution of interpretable factors. Our definition of interpretability in this study is as follows: if product A ranks higher than product B, there must be at least one factor whose SHAP value in product A is greater than that in product B.
3.2. Key Notions
Here, we discuss the necessary notions of our framework.
Click-through rate (CTR): in the e-commerce field, search ranking models estimate the click-through rate of users and then combine other business considerations to determine product ranking. Our interpretability method can be used to explain the marginal contribution of different factors to the target variable.
SHAP: Shapley value is mainly used to solve the allocation equilibrium problem in cooperative game theory [40]. This study focuses on the e-commerce search system, and first uses actual features to model the product ranking, and then introduces SHAP to explain the model. The SHAP value can not only explain the importance of features but can also quantify and estimate the positive or negative impact on label variables. Compared with other feature explaiable methods, information gain can intuitively reflect the importance of each feature for the model’s predictive value, but cannot quantify the positive and negative relationship between features and the final results. Therefore, our interpretable solution chooses the SHAP value.
3.3. Research Motivation
The existing ranking model of the e-commerce search system has a complex structure and a large capacity, resulting in a lack of understanding of traffic distribution strategies. At the same time, businesses do not understand the rules of search ordering and lack an operational grasp in marketing linkage. To better solve these problems, we propose an explainable scheme, which uses the factors that have an impact on the target, and then analyzes the relationship between the traffic distribution factor and the commodity ranking. Compared with other baselines, the advantage of our approach is that it can reflect the influence of the features in each sample and also show the positive and negative influence.
4. Our Method
This section introduces our proposed framework, named explainable factor marginal effect analysis (FMEA) approach based on game theory. The FMEA framework consists of two stages that are applied recursively: predict model training and feature explanation. The first stage utilizes well-known machine learning or deep learning models. The second stage generates features’ marginal effect based on game theory. Our explainable model architecture is shown in Figure 1.
4.1. Search System Feature Explanation Scenario
The Large-Internet-Company’s search system feature explanation aims at stimulating the commodities traffic that reaches the linkage threshold after shop owners paid marketing. Feature explanation brings two main effects. First, more active advertising brings more revenue to Large-Internet-Company. Second, it helps shop owners manage the search and recommendation traffic better and brings a positive impact for their sales. To promote shop owner’s awareness of linkage effect and facilitate the business departments to control the delivery, we represent the impact of marketing linkage as an index that can be intuitively understood by the business. Based on game theory, the features’ marginal contribution is mined and the deterministic influence of marketing linkage factors is quantified.
The merchant does not know about the search rules for their goods. They lack the interpretative and instructive search operation tools, which often leads to malpractices such as fraudulent billing. Our interpretable approach uses the form of factor marginal contribution evaluation to help merchant clearly understand the overall level of their products in search and understand how to operate to improve their product rankings. In this scenario, the diagnosis process for businesses is shown in Figure 2.
4.2. Explainability of Feature Importance
The ex post explanation has attracted more and more attention in the industry. For better business interpretation, we map the SHAP value to real business probabilities as shown in Figure 3.
4.2.1. Feature Importance Based on the Game Theory
The feature set of training data is , and the calculation formula of the marginal contribution of feature is as follows:
Formula (2) is an expected value, which represents the variation of prediction results between the participating model and the nonparticipating model under different feature combinations. represents the complete set of features, and represents the feature subset excluding . The value of S has various conditions, which correspond to different feature combinations. The formula is derived as follows: Step 1: extract from the feature set with a probability of . Step 2: select a subset from the remaining feature subset with a probability as follows: Step 3: multiply the probabilities of step 1 and step 2. The product is the probability of each combination of features.
We propose a second-order interpolation operator as follows:
Specifically, the interpolation operator transformation is as follows:
The process of approximate Shapley estimation for a single feature importance is shown in Algorithm 1.
|
4.2.2. Feature Importance Based on Causal Inference
First, regression solves the problem of optimal linear prediction. Let be a parameter vector:
Linear regression finds the parameter that minimizes the mean square error, and the linear solution is given by the following equation:
We can estimate the beta by using the following formula:
In data analysis, we want to estimate the causal effect of the variable T on the outcome of y. Therefore, we use regression with this variable to estimate the effect. Even if we add other variables to the model, they are usually auxiliary variables and do not affect the calculation of this causal effect. For a regression variable T, the parameters associated with it are given by the following formula:where is randomly assigned and is the average causal effect. We have multiple regressors, which can extend the following formula to fit. We are really interested in estimating the parameters associated with , and the other variables are auxiliary variables.where can be obtained by the following formula:where is the residual of all other covariates regression on . This means multiple regression coefficients are bivariate coefficients of the same regressors after considering the effects of other variables in the model. is the bivariate coefficient of T after all the other variables have been used for the prediction. Extending to the multivariable case, we see how regression provides a marginal explanation for the intervention factors after excluding other influences. The estimated value of the intervention factor coefficient can be interpreted as how the outcome changes with the intervention, holding all other included variables constant.
On the basis of factor explanation, we try to introduce the method of causality analysis, which calculates the feature importance by factor regression coefficient. The results of feature importance are in agreement with the explainable theory, which further strengthens the confidence of our proposed method.
4.2.3. Feature Importance Merge
We combine the feature importance of game theory and the causal inference, and the integrated feature importance is shown in Figure 4(a). To reduce the blank space in Figure 4(a), we divided the features into two groups. Features with similar value ranges were placed in the same group. We then drew two subplots using different coordinate scales, which are shown in Figure 4(b).
(a)
(b)
(c)
Samples and features of the recommended business continue to grow as the iterations and the number of model parameters also increase, but interpretability becomes important and difficult. In the actual scenario, especially in business operation diagnosis, it is necessary to give the business a certain degree of explanation, to help the business growth. At the same time, through intuitive explanation, it can improve the model effect and iteration efficiency.
4.3. Explanation of the Feature Marginal Effect
We improved the game theory model to calculate the marginal effects of different features, which are shown in Figures 5 and 6.
In Figure 5, the horizontal axis is the SHAP value, which represents the feature marginal contribution. The vertical axis represents different features: the more red the color, the larger the feature value; the more blue the color, the smaller the feature value. Figure 6 is the result of mapping the SHAP value via the business operator. The horizontal axis represents the probability of ranking improvement, which has more business significance compared to the SHAP value. The vertical axis represents different features: the more red the color, the larger the feature value; the more blue the color, the smaller the feature value.
From Figures 5 and 6, the data performance of the main interpretable factors is as follows: queryFlowscore has a positive effect; the higher the queryFlowscore, the greater the rank probability upgrading. competeScore has a negative effect; the lower the competeScore, the greater the rank probability upgrading. When the queryTopRatio is at a low or high value, it is easier to achieve a higher rank probability upgrading. The marketing output of the product has a positive effect, the higher the marketing output, the greater the probability of achieving rank upgrading. The conversion rate (CVR) performance of the product has a positive effect; the better the CVR performance, the greater the probability of achieving rank upgrading. The click-through rate (CTR) performance of the product has a positive effect; the better the CTR performance, the greater the probability of achieving rank upgrading. The gross merchandise volume (GMV) performance of the product has a positive relationship; the better the GMV performance, the greater the probability of achieving rank upgrading. The better the order line performance, the greater the probability of achieving rank upgrading.
4.4. Online Deployment
With the search ranking iterates, samples and features increasing, interpretability becomes more difficult. However, in the business, especially in the diagnosis of merchant operation for business, certain interpretability should be provided to merchants to help them grow. Besides, the explainability can also improve the customer experience of E-commerce platforms. In this article, we discuss the effect of interpretability for business. The application of our explainability approach is shown in Figure 7.
The online board includes priority id, diagnostic analysis, and quantified guidance suggestions, which are shown in Table 1.
5. Experiments
5.1. Datasets
Our data come from the real scene of a Large-Internet-Company marketing linkage project. There are 31 features in total, including commodity features and marketing features. label is designed to increase the ranking of goods to 1 and decrease the ranking to 0. The model training data volume distribution is shown in Table 2. The main features used for training are shown in Table 3. We describe the details of the features from the following four dimensions: feature name, feature definition, feature influence mode, and feature business meaning. (1) Feature name: queryCompeteScore. Feature definition: the competition degree of the main search terms placed on the product. Feature influence mode: negative effect; that is, the lower the competition degree of search terms, the greater the probability of obtaining cross-ranking promotion. Feature business meaning: the degree of competition represents the proportion of products marketed under the word, that is, whether the product marketing is in a more competitive marketing environment. The product is more affected by the marketing linkage, the query words of which have high degree competition. It is easier to show marketing effects under search terms with low competition. (2) Feature name: srOrderline7davg. Feature definition: the order line performance of the product in the competitive product set. Feature influence mode: positive relationship; that is, the better the order line performance, the greater the probability of obtaining cross-ranking promotion. Feature business meaning: compared with competing products, goods with more order lines are easier to obtain marketing linkage, and the positive relationship shown is consistent with cognition. (3) Feature name: srCvr7davg. Feature definition: cvr denotes the conversion rate of the product in the competing products. Feature influence mode: positive effect; that is, the better the cvr performance, the greater the probability of obtaining cross-ranking promotion. Feature business meaning: the cvr of a product is closely related to the search core index, and compared with competing products, the product with good cvr is easier to obtain marketing linkage. (4) Feature name: srCtr7davg. Feature definition: cTR denotes the click-through rate performance of the product in the competitive product set. Feature influence mode: positive effect; that is, the better the cTR is, the greater the probability of obtaining cross-ranking promotion. Features business meaning: the cTR of a product is closely related to the search core indicators. Compared with competing sets, the product with good CTR performance is easier to obtain marketing linkage. (5) Feature name: queryTopRatio. Feature definition: the exposure ratio of the product on the head search term. Feature influence mode: when the feature value is at a low value or an extremely high value, it is easier to obtain a higher probability of crossover lift. Feature business meaning: goods need to have a main source of flow, the head or tail query words of which are easier to get marketing linkage flow. (6) Feature name: srGmv7davg. Feature definition: GMV stands for the gross merchandise volume of the product in the collection of competing sets. Feature influence mode: positive relationship; that is, the better the GMV is, the greater the probability of achieving cross-ranking promotion. Feature business meaning: compared with the competing sets, the products with high GMV are easier to obtain marketing linkage. (7) Feature name: output. Feature definition: marketing output of the product. Feature influence mode: positive effect; that is, the higher the output is, the greater the probability of obtaining cross-ranking promotion. Feature business meaning: the output of marketing represents the inspection results of commodity quality through advertising, and it is easier to obtain marketing linkage for products with a good transformation effect. (8) Feature name: queryFlowScore. Feature definition: the liquidity of the main search terms of the product. Feature influence mode: positive effect; that is, the higher the fluidity of search terms for product placement, the greater the probability of obtaining cross-ranking promotion. Feature business meaning: liquidity indicates the degree of commodity rotation under the word, that is, whether there are more strong commodities under the word. Search terms with high liquidity have large space for increasing exposure and easily to get linkage improvement.
5.2. Evaluation Matrix
5.2.1. Offline Evaluation
We use AUC as offline evaluation, and use as online evaluation. The AUC (area under ROC curve) shows the ranking ability of the model; the higher the AUC, the better the model performance. It is defined as follows:where is the set of positive examples, is the set of negative examples, is the value of model prediction, and is the indicator function. In the Large-Internet-Company dataset, the AUC of the baseline is 0.6918, the AUC of the DNN explainability is 0.7025, and the AUC of our method is 0.7134.
5.2.2. Online Evaluation
Our online evaluation includes flow performance score, user feedback score, and service performance score. The details are shown in Table 4.
5.3. Baselines
(i)Data analysis [41]: in industry, computing feature importance via data analysis decision-making can help e-commerce provide more efficient and accurate information services(ii)XGBoost [42]: tree boosting is a widely used machine learning method, which can compute feature importance via information gain of split nodes(iii)Permutation [43]: permutation feature importance can be combined with any regressors and classifiers, which is widely used in deep neural networks.The disadvantage of data analysis is that it requires a lot of manual statistics, the analysis perspective depends on business experience, and manual experience is relatively limited. The disadvantage of XGBoost and Permutation is that only feature importance can be calculated, but the marginal effect of each feature change on the target variable cannot be quantified. Our proposed method solves the pain points of the abovementioned three bases and can automatically calculate the marginal effect of each feature.
5.4. Experimental Setup
We implement all the models using Python 3.6.7 on GPU Tesla P40. To be fair, we divide the datasets into train data with 80%, valid data with 10%, and test data with 10%, and all of these models share the same train-valid-test datasets. We repeat each experiment 10 times and take the average value as the evaluation index. In all the experiments, we also use the same input features of items and users, as well as the other training hyperparameters. All comparison experiments use the sigmoid activation function and Adam optimizer, and the learning rate is 0.01. Besides this, the batch size is 1024, the epoch is 20, and regularity coefficient is used.
5.5. Explainable Effect Analysis
Based on the whole category of Large-Internet-Company items, we analyze the linkage effect among conversion rate, query competition, and ranking uplift probability, which are shown in Tables 5 and 6.
Item conversion rate is closely related to the core index of search. Compared with similar products, the item with good performance of cvr is easier to obtain marketing linkage. As can be seen from Table 5, the better the conversion rate performance, the greater the probability of achieving uplift-grade promotion.
Competitive score represents the marketing linkage effect under the query words, which means competition degree of the product. The search terms with high competition, the products are greatly affected by the linkage. Less competitive search terms are easier to reflect linkage effects. As can be seen from Table 6, the lower the competition of search terms for product placement, the greater the probability of achieving cross-uplift promotion.
5.6. Case Study
Our online business application scenario is to establish an interpretable and diagnosable traffic tool for merchants, providing operational leverage for merchants. The bad case performance can be explained by using the factor marginal effect analysis method (FMEA). In the “personal care” category, we found that the marketing input of “product id = 100027183286” exceeded 77% of the products in the same category, but the increase in ranking was only 8%. The details are shown in Table 7.
In Table 7, compete denotes search word competition, output means item marketing output, and fluidity is search word competition. Therefore, our suggestion is to select search terms with low competition and good fluidity, so as to increase the linkage effect of the product in the search field.
For the bad case in Figure 8, our proposed model is utilized to explain why it is difficult for the product to achieve rank upgrading. From the data analysis, we can see that four factors (competeScore, srCtr7davg, queryTopRatio, and product marketing output) have a significant impact on whether the product can achieve rank upgrading. The competeScore factor has a significant negative effect (−0.11) on the probability of rank upgrading for this product, and queryCompeteScore is higher than the overall situation of the category it belongs to, indicating that there are more competing products for the search term exposure of this product, making it difficult to achieve rank upgrading. The srCtr7davg factor has a significant negative effect (−0.08) on the probability of rank upgrading for this product, and the CTR is only higher than that of 13.7% similar products, lower than the overall situation of the category it belongs to, indicating that the conversion rate of the product is poor, making it difficult to achieve a ranking improvement in the search field. The queryTopRatio factor has a significant negative effect (−0.06) on the probability of cross-rank upgrading for the product, and the queryTopRatio is higher than the overall situation of the category it belongs to, indicating that the product traffic mainly comes from the head search terms, making it difficult to achieve rank upgrading. The product marketing output factor has a significant positive effect (+0.05) on the probability of rank upgrading for the product, and the marketing output is higher than the overall situation of the category, indicating that the product has more marketing investment on and off the platform, thereby increasing the probability of rank upgrading. Therefore, it is suggested to operate the search terms of this marketing product and choose low-competitive, waist-to-tail search terms for advertising, so as to increase the linkage effect of the marketing product in the search field.
5.7. Data Analysis of the Regression Test
In terms of the model’s usability, we conducted validation from two perspectives: model accuracy and trend consistency to ensure that the model can not only predict the probability of rank upgrading of products accurately but also obtain insightful guidance to improve the product’s rank probability. Figure 9 shows the validation of model accuracy, from which it can be seen that the prediction error of the model under each feature to the sample’s actual performance is around 1%.
Over a relatively long time period, the indicators are monitored and the collections of products with increased indicators and the collections of products with decreased indicators are compared. Figure 10 shows that the cross-rank upgrading probability of the product collection has the same trend of change as the prediction.
6. Conclusion
We propose a factor marginal effect analysis approach (FMEA), which is used to quantize the correlation between online traffic and features. Specifically, based on game theory, the FMEA can compute how much one-dimensional feature brings about the enhancement of online traffic. Besides, we design the high-order operator to quantify the feature contribution. Extensive experiments are conducted and the results demonstrate the effectiveness of our method. In fact, the FMEA has been deployed in Large-Internet-Company search systems and successfully serving over hundreds of millions of consumers for online e-commerce service. This is the first work to integrate game theory interpretability into machine learning to evaluate feature contribution, and more related studies will be further explored.
This study focuses on improving the machine learning explanation by considering the factor marginal effect analysis, which provides a novel explainable intelligent method. However, the limitation of our proposed method is that the precision of explainable causal intervention effects needs to be further addressed. Explainability may inspire the development of novel training methods and evaluation metrics that guarantee the trustworthiness and consistency of even the most complicated models.
In the future, our work can be further studied from both traffic diagnosis and factor prediction of return on investment (ROI). Based on the improvement of game theory, traffic diagnosis can introduce a SHAP scheme with sample weighting to quantitatively estimate the contribution of each feature gap to the target variable. Through the introduction of a causal inference algorithm, it can predict the ROI effect of business action space and provide suggestions.
Data Availability
The data were used under license for the current study, and so are not publicly available. However, the data are available from the corresponding author upon reasonable request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Authors’ Contributions
Yingshuai Wang conceptualised the study idea, assisted with software and methodology, performed project administration, wrote the original draft, and reviewed and edited the manuscript. Sachurengui carried out investigation and reviewed and edited the manuscript. Dezheng Zhang conceptualised the study idea, supervised the study, performed project administration, and reviewed and edited the manuscript. Aziguli Wulamu collected the resources, curated the data, and reviewed and edited the manuscript. Hashen Bao performed project administration and reviewed and edited the manuscript.
Acknowledgments
This work was supported by the Science and Technology Innovation 2030-New Generation Artificial Intelligence Major Project (Grant no. 2020AAA0108703) and the project of Collaborative Innovation Center of Mongolian Medicine in Inner Mongolia Autonomous Region (Grant no. MYYXT202001).