Abstract
Many precise personalized learning recommendations in massive open online courses (MOOCs) have emerged in the intelligence education field. Up to now, most researches simply put the dual learner-resources relations into consideration and are short of studies looking deep into its intrinsic social relation, thus rarely introducing the influential factors such as social trust, which means to apply the mutual trust relation between learners in the precise personalized learning recommendation. Therefore, we propose a personalized learning recommendation method based on learners’ trust and conduct a quantitative analysis on two aspects: social trust and influence, so as to realize a precise personalized learning recommendation service. First, we establish a new module on social trust scale which integrates the interactive information and preference degree to reveal the implicit trust relation between learners in social networks and construct social trust networks. Next, we adopt improved structural hole (ISH) algorithm by integrating the topological structure of social trust network with learners’ interactive information and identify the most influential learners cluster by the ISH algorithm. For the final stage, we predict the score of target learners based on explicit and implicit feedback information and realize the personalized learning recommendation for new learners. Since the score is predicted, we compare MAE and RMSE in two real-world datasets which are Canvas Network and Wanke website, respectively. The result of experiment validates the accuracy and effectiveness of our recommendation model.
1. Introduction
Nowadays, massive open online courses, or MOOCs, are attracting widespread interest as an alternative education model. Many MOOCs platforms such as Coursera, edX, and Udacity have been built and provide low cost opportunities for anyone to access a massive number of courses from the worldwide top universities [1]. With the 2.0 action projects for education informatization being carried out quickly and the construction of MOOC platforms, the supply of excellent courses resources will increase year by year, which will inevitably lead to resources overloading. How to help learners quickly locate the target interesting course is also a question that needs to be considered in the process of building an intelligent MOOC platform. Due to the low threshold for registration, open resources, and asynchronous and unconstrained presentation in MOOCs, it can be predicted that the number of courses covered on the website will explode as the size of the online course learning website expands and becomes more influential throughout the world. Information overload has become a crucial challenge that users are overwhelmed, and to help learners locate their interested course or appropriate learning resources, learning partners and domain experts are becoming an unprecedentedly important task, attracting attention from both research and application domains [2, 3]. How to let the user find the most suitable course in massive course information will become one of the most important problems that need to be solved to promote the user experience. In MOOC platforms, recommender systems learn users’ learning needs and preferences and direct them towards possible resources of interest [4]. With the recent pandemics, the subscription to MOOC platforms has increased by 25–30%, which makes the research on recommender systems in these platforms more and more relevant. Therefore, how to provide learners with targeted learning resources and improve the learning efficiency of online courses is a major challenge for online education. The diversified courses in MOOCs platforms demand an effective way of precise personalized learning course recommendation for their learners.
From the national education policy level, both the Preparation for Future Learning: Reshaping the Role of Technology in Education issued by US National Education Technology Plan 2016 and Educational Informatization 2.0 Action Plan proposed by China Ministry of Education emphasize education should rely on big data and artificial intelligence technology to innovate and improve educational data system and thus promote personalized learning and educational governance. Appropriate courses can be recommended to learners by embedding personalized recommendation technology into MOOC platforms. On the one hand, it can save a lot of time for the learners to locate the target courses and provide personalized learning services. On the other hand, it can greatly improve the utilization rate of the courses resources. By embedding personalized course recommendation technology, the platform can provide more intelligent service, thus realizing the innovative supply of educational resources services.
These years scholars have attempted to apply the techniques of recommendation systems to solve the problems of educational recommendation [5], whose main goal is to identify learners’ interested items or possible interested items in the future and provide them with the corresponding resources, peer and expert recommendation services, through analyzing learners’ historical interests and preferences, which is, therefore, called personalized learning recommendation system [6]. According to Social Relation Theory (SRT) [7], users with strong social relation always share similar preference and mutual influence in some aspects, which benefits to construct the personalized recommendation system. With the popularity of social platforms, trust information is very common in most social media; it can describe the relationship between users, which means that users have common interests in some aspects [8]. Therefore, research on personalized recommendation method based on social trust has become a hot issue and attracted extensive attention from both the academic and educational fields.
Social trust relation, which reveals one’s comprehensive evaluation of another user’s behavior and ability, often is regarded as a decision support tool to deconstruct relevant reliable information sources, especially seek advice from those sources, and then make the final decision [7]. However, the uncertainty of trust relationship becomes the biggest challenge to the trust prediction due to the different trust relationship between different users and the difference of trust relationship intensity in different domains. Besides, influence, another important aspect influencing user’s behaviors in social networks, extends the description of social trust from another dimension. Influence plays an increasingly prominent role in behavior influence. For example, early politicians used their influence to win elections, and businessmen used influence to sell goods. Microblog celebrities used their influence to lead the opinions, and their influence can be witnessed in many hot topics and unexpected events. Therefore, this paper conducts a quantitative analysis on two dimensions of social trust measurement: trust relationship and influence, to calculate the trust degree of trust relation between individual users and cluster influence and then improve the efficiency and accuracy of personalized learning recommendation.
This paper treats students in MOOC learning as users in electronic commerce, the test questions as items, and the scores on the test questions as rating to the items. Moreover, since most researches on personalized learning recommendation systems simply put the dual learner-resources relations into consideration and often ignore the impacts of social trust on precise recommendation, this paper proposes a precise personalized learning resource recommendation method based on social trust, constructs a “learner-resources-social trust” triangle recommendation model, which depicts the social trust relation between individual learners and between learners clusters through quantitative analysis of social trust and influence dimensions, and thus provides precise recommendation for personalized learning recourses and improves learner’s self-inquiry learning ability. The architecture of our recommendation algorithm is shown in Figure 1, which shows using social network to dig the trust users and influential users and then apply it to recommending. The main contribution of this research is as follows:(1)A new model of social trust measurement is proposed and social trust networks are constructed, which integrates learners’ interactive information and preference degree, to excavate the implicit trust relationship among learners in social network.(2)Based on the topological structure of social trust networks and the interactive information among learners, the improved structural hole algorithm is applied to identify the most influential cluster of learners in the networks.(3)Perform quantitative analysis on social trust and influence degree, together with implicit and explicit learner’s feedbacks, to predict learner’s test scores.(4)Comparison experiments in two real-world datasets, Canvas Network and Wanke website, validate the accuracy and effectiveness of our recommendation model.
The remainder of this paper is organized as follows. Section 2 gives an overview of the related work. Section 3 demonstrates details of our proposed model. Section 4 illustrates experiment datasets and evaluation metrics. Section 5 shows experiment analysis. Section 6 concludes the paper.
2. Related Work
In brief, there are three kinds of efforts involved in our work: personalized learning recommendation, trust-aware recommendation, and social influence analysis. We review previous works on these topics in this section.
Present researches on personalized learning recommendation system mainly focus on three different recommendation objects, which are resource, peer, and learning routes. Relevant researches reveal that personalized learning recommendation system could exert certain positive influence on learning interest and efficiency, which could drive a more in-depth application of online learning. Bao et al. [2] proposed the basic elements and the model in social recommendation system from four perspectives, such as learners’ features, the characteristics of recommendation objects, social relation, and the context. Based on the element model, Bao designed a social recommendation system model and method based on interested topics. Thus, proper education resources could be recommended in accordance with learners’ actual demands in different contexts, and meanwhile by recommending resources to proper learners, the model could well realize the connection between Learner-to-Resource and Resources-to-Learner and offer more precise education service based on multidimensional correlation analysis of global data. Jiang and Zhao [9] illustrated how to apply the AprioriAll algorithm to dig up learning route among the same learning group cluster and conducted precise personal push in accordance with learners’ characteristics, such as learning style and knowledge background. Eugenijus [10] applied artificial intelligence technology, such as ant colony optimization, to realize the push of static and dynamic personalized learning primitives in line with learning style. Its experiment reveals that research outcomes applied in e-learning could cut short the learning time and improve learning efficiency and quality. Seng et al. [11] proposed a powerful Item-based Collaborative Memory Network (ICMN) for ICF, which is based on the architecture of End-to-End Memory Networks. Ding and Zhang [12] came up with the idea to integrate learners’ social network information with traditional collaborative filtering, so as to calculate the trust degree between new learners and friends. Through their friends’ rating data to learning resources, the new learners’ rating could be predicated. This way could fill the missing data of new learners in the learner-resources rating matrix and realize the personalized recommendation to new learners.
The research on social recommendation system can be traced back to Referral Web System (RWS) proposed by Kautz et al. [13] in 1997, which integrates social networks on the traditional collaborative filtering model to provide users with more precise and efficient recommendation results. This proves that the integration of social relationships provides reliable data support for the recommendation system. While improving the recommendation accuracy, it also provides new ideas for solving the cold-start problem. The matrix decomposition recommendation model has the characteristics of high recommendation accuracy, good scalability, and high flexibility. For this reason, researchers have proposed a series of matrix decomposition models that integrate social information. Trust-based social recommendation system is the most common social recommendation system. The basic method is to use social trust as an indicator to measure the social relationship between users and to calculate the trust value between users by calculating the mutual trust relationship between users. Generally, two users with higher trust values have more similarities. A large number of researchers have conducted research on recommendation systems based on user trust. Yang et al. [14] proposed TrustMF, a social recommendation model based on the relationship between trust and being trusted. According to the directivity of trust relationship, TrustMF maps each user into two different K-dimensional feature vectors, which are called trustor feature vector and trusted feature vector, respectively. Jamali and Ester [15] introduced the method of trust transition into the recommendation algorithm (SocialMF), spreading the trust relationship by constraining the similarity of the average preferences of users and their friends, thereby obtaining more precise results. SoReg is a social regularization recommendation model proposed by Ma et al. [16] on WSDM in 2011. The idea of this model is similar to that of SocialMF; that is, it is assumed that the user feature vector should be similar to the friend’s feature vector. For this reason, it is proposed to use social information to regularize the user feature vector, so as to use the preference information of friends to influence the final prediction score of the user. In order to deal with the sparsity of rating and trust relationship, Guo et al. [17] introduced social information on the basis of the SVD++ model [18] and proposed a trust-based matrix decomposition model TrustSVD that takes into account both scores and trust information, taking into account the explicit and implicit effects of score and trust information when predicting the score of unknown items. Zhang et al. [19, 20] proposed a novel trust value measurement model that combines user interaction information, preferences, and trust which is brought up to explore the implicit trust relationship between users in social networks and reconstruct the social trust network. Wu et al. [21] proposed TrustEV and took the view of multitask learning to unite collaborative filtering for recommendation and network embedding for user trust. Yu et al. [22] introduced an adaptive trust-aware recommendation model based on a new trust measurement developed using a user-item bipartite network. Ardissono and Mauro [23] extended trust-based recommender systems with additional evidence about trust, based on public anonymous information, and proposed the Multifaceted Trust Model (MTM) to define trust among users in a compositional way, possibly including or excluding the types of information it contains. Bao et al. [24] decomposed trust information into four dimensions to measure: Goodwill, Integrity, Ability, and Predictability. The above algorithms study the trust intensity between users from the perspective of scoring or trust data and have made a significant improvement compared with the algorithm which regards the influence of each friend on the user as the same.
Social influence refers to the case when individuals change their behaviors under the influence of others. Various fields including viral marketing, online advertising, and personalized recommendation have expressed their interests in understanding diffusion. Domingos and Richardson [25] investigated social influence in the customer network. They proposed a model to identify customer’s influence between each other in the customer network and built a probabilistic model to mine the spread of influence for viral marketing. Aris et al. [26] exploited a statistical analysis method to identify and measure whether social influence is a source of correlation between the actions of individuals with social ties. These works prove that analyzing social influence can help us to understand peoples’ social behaviors, provide theoretical support for making public decisions and influencing public opinion, and promote exchanges and dissemination of various activities [27]. In light of the significant increasing of social networks in both scale and volume, how to measure a user’s social influence to other users becomes increasingly important.
The key node measurement methods based on node attributes and network location can be divided into two types: network local attributes and network global attributes. The node important ranking index based on the network local attributes considers node information and the neighbor node information, which is simple calculation with low time complexity. However, the network global attributes consider network global information, the index of which is high accuracy and time complexity. The global calculation of evaluating the nodes importance methods usually includes betweenness centrality (BC), closeness centrality (CC), and so on. BC measures the times a node acts as a bridge along the shortest path of two other nodes [28]. It requires that information should spread through the shortest way, but sometimes the information does not spread through the shortest way in most real networks. CC is defined as the average distance from one vertex to the other vertices in a network [28], which is used to measure the ability of nodes in the network to influence other nodes through the network. The closer the node is, the more important the node is in the center of the network. However, these algorithms’ time complexity is very high as they require knowing the whole network structure in advance and also need traversing the graph. Local evaluation of nodes importance measures includes the degree, Structural Holes, etc.; the easiest one is based on the degree of a node. Albert et al. [29] used the Degree Centrality (DC) to find the most influential nodes in the social network. They pointed out that, in heterogeneous scale-free networks, nodes with high degree (also called Hub nodes) have a higher influence. However, the DC measurement method only considers the number of neighbors but ignores the topological relationship between the neighbors and does not consider the location of the node in the network. Therefore, it does not reflect the interaction between neighboring nodes, resulting in inaccurate results. In order to consider the topological relationship between neighbor nodes in the evaluation index, Chen et al. [30] proposed using the two parameters of clustering coefficient and degree to jointly evaluate the propagation ability of the node. Seng et al. [31] put forward a hybrid top-N recommendation algorithm that combines mutual trust and influence. Burt [32] proposed another effective method using local information to evaluate the nodes importance, which is to find the “structural holes” in complex networks. Yu et al. [33] proposed an algorithm that can identify important nodes in a complex network, since this algorithm only considers the neighbor nodes and the nearest neighbor node of target node regardless of the overall structure of the network, the nodes’ importance can be calculated based on the local information of the complex network, and it is verified on the ARPA network that the algorithm is better than DC, BC, and CC in terms of node importance measurement. Moreover, Rezvani et al. [34] devised two fast yet scalable algorithms for the top-k structural hole spanner problem, by developing innovative filtering techniques that can filter out unlikely solutions as early as possible. The invented techniques are built up on fast estimations on the upper and lower bounds on the cost of an optimal solution and an observation of that the Articulation Points (AP) in a real social network usually are the structural hole spanners of the network. Inspired by the above research, we analyze how structural holes influence the procedure of information diffusion and study a novel problem of mining structural hole spanners in social networks; moreover, we propose an ISH algorithm which makes it possible to efficiently excavate key nodes in directed graphs.
3. The Proposed Model
This section introduces the overall framework of the ITSVD and details the three key stages of the method in five steps. The overall framework of the ITSVD is illustrated in Figure 2. To facilitate discussion, some symbols are introduced with specific definition as shown in Table 1. It functions as follows: first construct a “learner-resource” rating matrix based on learners’ learning record or practice tests, then apply the explicit and implicit feedback information together with preference degree to measure the trust relation between learners, and then discover the learners’ clusters sharing mutual trust with the target learners; after that, identify the learners’ cluster with the biggest influence in the trust network through improved structural hole algorithm, which points that one with higher trust degree has greater influence over others, and finally predict the resource rating values of the target learners through the social trust recommendation model. Afterwards, precise personalized top-N recommendation could be offered to the target learners based on the predicted rating values, which means that the resources within the top-N rankings could be recommended to target learners.
The model mainly includes three stages and five steps. The three stages refer to the calculation of learners’ trust value (trust relation), identification of the influential learners’ clusters (influence), and model training. The three stages are implemented in five steps:
(1) Calculation of trust degree
Step 1. We firstly use the interaction information between learners to calculate their initialization of trust value. We put forward a hypothesis: while learners have a common ranking score on the same test , record is an interaction.
Step 2. Trust value could be changed along with interactions; thus the interactions on different tests could change the trust value as well. Moreover, users tend to trust more those who have successful interaction over their favorite tests. Therefore, we introduce preference degree to measure learners’ preference over different tests.
Step 3. At final step, assign different weight over tests in each interaction based on learners’ preference over test , then generate the final trust value, and the trust network is formed after filtering out the weak trust relationship.
(2) Identification of influential learners
Step 4. Based on the directed trust network achieved in Step 3, apply nodes and their neighboring topological structures to calculate the constraint value of nodes. Utilize the constraint values as an indicator to evaluate the importance of node, which is called “Influence.” Finally, screen out the cluster of the most influential learners.
(3) Model training
Step 5. Combining explicit and implicit feedback information, we introduce the trust relation between learners to reveal their influence over each other, so as to improve the accuracy of simulation test. To avoid overfitting, the loss function adopts the regularization strategy of TrustSVD [18] algorithm, that is, to adopt less punishment to learners and tests with higher scores, but to adopt more punishment to the cold-start learners (new learners) and new tests.
3.1. Calculation of Learners’ Trust Value (Step 1, Step 2, and Step 3)
Calculation of learners’ trust value is an important step in social recommendation process. Trust originates in the experience of the subjective individual [6]. The more the learner trusts learner , the more the interaction that will occur. In reality, trust degree between people gets influenced by interaction relation, as well as by the interaction result. Successful interaction could increase their trust degree and vice versa. Besides, the trust influence resulting from interaction varies along with the preference difference caused by learners’ different interest over tests.
Therefore, in this section, we propose a new measurement model of trust relation in consideration of learners’ interaction information and preference, so as to take comprehensive account of learners’ scoring and preference degree over different tests.
First, we make the following two hypotheses:(1)If two learners score the same value over one test, they are assumed to have one interaction.(2)If the score of test i achieved by learners and is higher or lower than average score of each learner, this interaction is considered to be a success, or otherwise that is a failure, where , are the evaluation of item by user and user , respectively, and are the average scoring of user and user , respectively.
Under the condition when learner u and learner share one iteration shown as , the trust measurement of learner u over learner is as below:Init(, ) refers to the initial trust value, calculated as
Threshold value indicates the minimum interaction times when two learners fully trust each other. points to the preference degree of learner u over test i: represents the cluster of learners who have already scored on test . The more the similarities the learner shares with others in the cluster, the more he prefers test . refers to the similarities between learner and learner . Users who are similar to user would like item , scored by user . To solve the problem that the number of common tests shared by learners is limited, we introduce the classic measurement of two variables’ relevance by Pearson Correlation Coefficient (PCC) formula to estimate the similarities, which is proposed to calculate two variables’ relevance by Karl Pearson in 1880s. By introducing the weight parameters of common tests’ number and adjusting the PCC between 0 and 1, the similarity formula is modified as
The different weights are assigned to different tests based on the learner’s preference in successful or unsuccessful interactions. Therefore, the final trust value is achieved by
3.2. Identification of Influential Learners (Step 4)
Traditional structure hole (SH) theory enjoys a wide application and serves well to identify the influential nodes in social network. SH is an effective method which only uses local information to identify key nodes of networks. SH is originally developed by Burt [32] who introduced this concept in an attempt to explain the origin of differences in social capital. Burt’s theory suggests that structural holes are gaps in information flows between individuals linked to the same ego but not linked to each other. It indicates that the people on either side of the hole have access to different flows of information. The calculation of SH is relatively complicated. Generally, there are two evaluation indices: the index given by Burt [32] himself and the betweenness centrality index. The former involves four aspects: Effective Size, Efficiency, Constraint, and Hierarchy, where the degree of constraint is the most important. The latter mainly refers to Freeman’s betweenness centrality [35] for the overall network and its promotion form, where the basic idea is that if a node is on the shortest path of many other node pairs, the node has a higher betweenness centrality, and it is more likely to occupy the structural hole location. The constraint refers to the node’s ability to use SH in its own network, setting the evaluation criteria as the node’s reliance value to other nodes. The formula is as follows:where refers to the nodes which have direct connection with node i in undirected graph. Since the trust network is directed network, to avoid confusion in later process, is used to replace . stands for the percentage of energy which node i spends to maintain the relationship with node j in its total energy.where <j,i> is the connection between node j and node i; if connected, it is 1; otherwise it is 0. is determined by the number of bridging node q between nodes i and j. The closed triangles are formed more and more due to the close ties between nodes i, j, q, which is not conducive to the widespread dissemination of information. In (10), C(i) can comprehensively evaluate the number of neighbors and the tightness of connections between nodes. Higher values of indicate that the in-degree (number of incoming nodes) of one node is smaller and has a high closed degree with these users. Such nodes are less likely to acquire new information. On the contrary, nodes with smaller constraint coefficients have greater influence on information spreading.
However, its limit lies in that it only measures the relation between a node and its close neighboring nodes but ignores the topological relationship between a node and its two-step neighboring nodes. Therefore, some important nodes could be neglected. As shown in Figure 3, from the one-step neighbor perspective of nodes A and E, nodes A and E have the same structure. According to the calculation formula of constraint degree proposed by Burt, the constraint degree of node A and node E is
However, it can be seen from Figure 3 that the trusted node C of node A and the neighbor node G of node E have different topological structures, and node G has better relationship than node C. Therefore, Burt’s constraint degree calculation method cannot effectively quantify the node differences in the digraph. It is necessary to improve the calculation method of network constraint degree to measure the importance of nodes in the network more accurately.
The Information Flow Theory (IFT) indicates that information spreading on Internet always reaches to the leading nodes first, and then from the leading nodes it reaches out to a broader network. From this view, if a node has connection with many leading nodes, the node has more possibilities to become a structure hole. On the other hand, when observing the influence of node E on node A and node G, respectively, it can be found that there is another trust node L in node G, which weakens the influence of node on and to some extent weakens the influence of node E on the trusted nodes H, i, and J of G. Thus, we propose two assumptions:(1)If the node j trusted by node i is a leading node or Hub node with many in-degree nodes, then node j could greatly enhance the influence of node i.(2)If the node j trusted by node i has multiple truster nodes, the influence of node i will be weakened.
An improved structural hole algorithm (ISH) is put forward by integrating the in-degree and out-degree influence of neighboring nodes on target nodes with the classic structural hole algorithm:
With the above formula, the constraint coefficient of all nodes could be calculated. Afterwards, the top-k% learners with the smallest constraint coefficient are marked as IU, as the learners cluster with global influence, where represents the nodes with direct connection with node i in the undirected graph; and stands for ; p(j, i) refers to energy ratio that node i spends in maintaining the relation with node j.
According to the calculation formula of constraint degree proposed by ISH, the constraint degree of node A and node E in Figure 3 is
It is found that node E has smaller constraint degree than node A, which means that node E has better ability of information transmission, i.e., influence. The improved formula has changed from considering only one-step topology neighbor to introducing two-step topology information, so as to more accurately excavate the structural hole nodes in the network.
3.3. Model Training
There is a problem that learners’ explicit information is difficultly obtained, which has data sparsity in social Internet. And except the relationship of one-to-one trust, the reliability of learners is also important which is influence in social Internet. The influence of different learners is great difference, where a few learners not only switch public opinion, but also have long term sustainability effect on other learners. Thus, trust learners and influential learners have effect on other learners in sense. The rationale behind SVD++ is to take into consideration user/item biases and the influence of rated items other than user/item specific vectors on rating prediction. The TrustSVD model is built on top of a state-of-the-art model known as SVD++, which combines implicit influence of trusted users and implicit influence of trusting users. The proposed algorithm based on SVD++ [17] and TrustSVD [18] introduces the trust relation between learners to reveal their influence over each other, so as to improve the accuracy of simulation test:where bu, bj are the bias of user u and item j; u is the global mean score; Iu is the cluster of items that u has rated; yi is the implicit feedback on item i; is the influence of all other items I on u’s rating prediction. Then let be the vector in the trust eigen matrix W and be the influence of user trusted by over u’s rating prediction on item j. For each influential learner , inner product refers to the impact of influential learners f over target test i.
To avoid overfitting, the loss function adopts the regularization strategy of TrustSVD algorithm, that is, to adopt less punishment to learners and tests with higher scores, but to adopt more punishment to the cold-start learners (new learners) and new tests; the loss function can be expressed as
Uj, Ui represent the cluster of learners who have scored, respectively, on items j and i, and represents the trustors’ cluster of learner . To reduce the complexity of the model, the same parameter for all the regularized terms is applied, except the regularized trust term .
To obtain the local optimal value of the objective function of equation (10), the following gradient descent for all learners and items is performed on bu, bj, pu, qj, yi, pf, and in the train dataset:
The pseudocode for the ITSVD is shown in Algorithm 1, where the input is regularization parameters and learning ratio . First, randomly initialize the eigen matrix and the bias vector (Line 1), then calculate the loss function, and update the variants with stochastic gradient descent method (Lines 10–16) until the function recesses (Line 2). Finally, return to the bias vector eigen matrix and the bias vector as output (Line 19).
4. Datasets and Evaluation Metrics
The emphasis of social recommendation is to collect learners’ social data in online learning platforms. In order to highlight the attributes of social relations and avoid bias, this paper selects data from two real-world online learning platforms: Canvas Network (http://www.canvas.net) and Wanke (http://www.wanke001.com/index).
Canvas Network is an open online course learning platform owned by Instructure, an American educational technology company. It offers open online courses, including massive open online courses (MOOC), which are free of charge to learners around the world. The chosen set is a public data set from January 2014 to September 2015 released by Canvas Network in March 2016. The data set includes 238 courses in 10 disciplines on Canvas Network platform, among which 130 courses are related to vocation and education, whereas relatively few courses are related to medical, mathematics, physics, and computer science. Majority of the courses last from 1 to 2 quarters, and only 26 courses expand to 3 quarters or longer. Most courses span from 35 to 65 days. The data include 325199 aggregate records, each representing individual learner’s behavior in a course. The variables can be generated from three sources: course management, interactions between learners and courses, and learners’ questionnaire. The structure of Canvas Network is based on the 2014 edX dataset. Due to deidentification process, Canvas removes two fields of learner’s gender and nationality, which is worth analyzing, but adds the field about learners’ questionnaire, compared to the 2014 edX data. Canvas offers 26 lists, including 4 types of information, regarding courses, learners’ basic information, learning intention, and learning behaviors. The specific attributes include course IDs, user IDs, registration, browse history, course discussion, completion, score, motivation, country, learner’s type, study time per week, education background, age, gender, registration time, course start time and end time, recent interaction, number of interactions with the course, interaction dates, number of viewed chapters, numbers of discussion posts, and course duration dates or dates with learner’s participation in the course. Our experiment conducts data preprocessing to the Canvas data and deletes the data with no score.
Wanke (http://www.wanke001.com), a commercial online learning platform launched in 2013, is an interactive flipped classroom teaching platform for Chinese universities, similar to MOOC. Wanke combines the online MOOC teaching with offline classroom interaction, brings online learning into evaluation mechanism, and realizes the one-window classroom service, including teacher-learner real-time interaction, course tests, score evaluation, and personalized learning experience, etc. To verify the validity, the C Programming Language course learning data from 2016 to 2018 and 161 learners from 4 classes enrolled in 2015, 2016, and 2017 are selected as experiment subjects. The attributes of Wanke include user name, name, student ID, major, knowledge point, study times, etc., among which study times represent the number of times a learner spends in completing the test. Meanwhile, the tests’ score of all 56 knowledge points in the C Language Programming course is chosen for the research. Considering that some tests have no score, the maximum study times are set at 10, and the scores that exceed the maximum study times default to 0. For rest data, take the ratio of learners’ study times to their maximum study times on a test as the learner’s score on the test. After the processing, the value range of score is [0, 1].
| ||||||||||||||||||||||||||||||||||||||||||||
Table 2 illustrates the specifications of two data sets, which finds out that Canvas Network data is very sparse, whereas Wanke data is less sparse. The choice of these two datasets benefits to observe how our model performs in different sparse conditions. Table 3 illustrates the attribute comparison of two data sets.
Ultimate goal of learning resources recommendation system is to fulfill the personalized learning demands. Learner’s satisfaction rate over the recommendation result is a key index to evaluate the system. However, the satisfaction rate is very subjective and it is hard to quantitatively measure how much they are satisfied. Thus, certain quantitative evaluation indexes are applied, such as precision rate, recall rate, coverage rate, F value, Root Mean Squared Error (RMSE), and Mean Absolute Error (MAE), etc. In this section, we adopt two widely applied precision indices: MAE and RMSE to evaluate the experiments:where N is the rating times in the experimental test. Lower MAE and RMSE represent better recommendation performance.
5. Experiment Analysis
In this section, a series of experiments are conducted in two real-world data sets: Canvas Network and Wanke001.com to compare the proposed methods with other mainstream recommendation methods.
5.1. Comparative Experimental Methods and Parameters Setting
ITSVD and other selected main recommendation methods are compared, excluding the impact of trust and influence:(1)UAvg, IAvg: baseline approach. Avg applies the rating average of each user to predict the unknown rating, whereas IAvg applies the rating average of each item to predict the unknown rating.(2)basicMF: the basic matrix decomposition algorithm.(3)SVD++ proposed by Koren [18], a recommendation algorithm based on latent (hidden) factor. It applies user’s historic evaluation data as implicit feedback and deducts the preference information of users and items which is evaluated in recommendation process.
The optimal parameters are selected for final comparison, and then its performance is verified when the eigenvector dimension is at 5 and 10.
The optimal parameters are set in the experiment as below:(1)Biased MF: set at Canvas Network data cluster; set at Wanke data cluster.(2)SVD++: let at Canvas Network data cluster; let at Wanke data cluster.(3)ITSVD: set at Canvas Network data cluster; set at Wanke data cluster.
5.2. Experiments and Results Analysis
This section conducts experiments and analyzes results from 3 perspectives:(1)Comparison analysis of personalized learning resources recommendation models. Compare the proposed model, trust-based ITSVD, with other recommendation models (such as UAvg, IAvg, basicMF, and SVD++) which neglect the trust and influence effects in account of the accuracy and efficiency in predicting the test scores of target learners.(2)Discuss the setting of k, quantity of influential learners to the global recommendation performance.(3)Simulation tests on influential nodes show the proposed model has better capacity in identification of influential learners.
5.2.1. Comparison Analysis of Personalized Learning Resources Recommendation Models
To keep the experiment unbiased, tests are carried out on Canvas Network and Wanke data sets, and the accuracy of the models is verified at the dimension of 5 and 10. Table 4 illustrates the results and analysis of all models’ performance at two data sets, where the optimal results are in bold, and the numbers with asterisk are the best results except the optimal results. The table reveals that the MAE and RMSE from ITSVD model are smaller than from other models, which means the personalized learning resources recommendation model based on social trust can significantly improve the global performance and quantity of resources recommendations.
Besides, one noticeable point of ITSVD is that its recommendation accuracy in Wanke data set is lower than in Canvas Network, which is due to the sufficient score data in Wanke. Under this case, other models could perform well; for example, the MAE and RMSE from AVeg reach, respectively, 0.078 and 0.112 and thus show no much differences between those models.
5.2.2. Analysis of Number of Influential Learners k
Parameter k controls the cluster number of influential learners. Select the most influential top-k % learners as the cluster of global influential learners. Since it is considered to be only a small number of influential learners in the social network, the experiment selects the top 10% users as subject. In order to testify the influence of parameter k on the recommendation performance, k value is modified from 0 to 10 in step 1. Figure 4 shows that when k = 0, excluding the effect of influential learners, the experiments in two data sets show the worst results, which suggests the cluster of influential learners will improve the recommendation performance. Meanwhile the descanting rate of error value in Wanke data set is slower than that in the Canvas Network, which is because the excessive trust relation among learners in Wanke results in the overlap of influential learners’ cluster with trust learners’ cluster and thus could not unveil well learners’ interest.
(a)
(b)
Another interesting finding lies in that the error value in the Canvas Network does not gradually decrease along with the increasing number of influential learners. The reason behind is the serious polarization of influence due to the sparse data, which indicates only few learners enjoy high influence, whereas the majority enjoy little or no influence. When exceeds certain threshold value, the low-influence learners will be included. These learners take no benefits in scoring predication but misguide target learners.
5.2.3. Analysis of Influential Learners’ Identification
Simulation tests on influential nodes are carried out under the classic SI model of epidemic diseases to effectively identify influential learners. The model effectively simulates the transmission of simulated information and virus.
Under the SI model, nodes in the network obtain two possible states at any time: susceptible (S) and infected (I).
The S node can at each time period transmit virus to neighboring nodes at a transmission probability of . If the node once is infected to I status, the process is irreversible.
Two experiments are designed to evaluate the dynamic performance of different influential nodes measurements in scale-free network model. Experiment 1 carries out the influence identification to top-10 learners, so as to compare the accuracy of algorithms in ranking the nodes’ transmission capacity. Experiment 2 aims to identify the influence of global learners, so as to comprehensively compare the identification capacity of different algorithms in low-influence and high-influence nodes’ identification and then improve the experiment’s reliability.
Experiment 1 sets top-10 learners ranked by different influence measurement models as initial transmission source, compare the changes of nodes’ status along with time t, and then apply as the evaluation index the ratio of infected nodes I during the transmission process. Set social network data in Canvas Network as sample, , the maximum time step, tmax = 50, and Si is sum of infected nodes after the time as the actual transmission influence. Meanwhile in order to get reliable results, experiments are conducted several times to get the mean value:where M is the repeated times of experiment on node i. As above mentioned, set M = 100, and get its mean value as the infected nodes at time t. Figure 5 compares the changes of ratio in Canvas Network of DC, SH, and ISH models and reveals the results from ISH model show stronger and quicker transmission capacity than that from SH and DC models, which verifies ISH model’s advantage at node’s influence ranking.
Experiment 2 conducts node’s influence analysis over global learners under the Simulation Investigation (SI) model but expands the subjects to the global learners. The design is as follows: Set real social network data at Canvas Network as sample; keep the relevant parameters setting of Experiment 1 unchanged. The initial transmission sources could be any learners in the network, and the transmission time is . Same as Experiment 1, conduct the experiments several times to get the mean value of relevant nodes’ transmission capacity, as illustrated in Figure 6. Due to limited space, here only display the top-200 real influential nodes of each algorithm. The segmented information of influence is clearly marked in purple (low-influence nodes) and red (high-influence nodes). The specific results are as follows:(1)Low-influence nodes (among the purple area): comparing SH and DC models, their results are quite similar; however SH model ranks the nodes more advanced than DC model, which suggests the DC model is better than SH model. Moreover, the low-influence node ratio is apparently lower in ISH than that of SH and DC models, and its low-influence nodes aggregate at the second half section.(2)High-influence nodes (among the red area): Analyze the top-200 nodes, and find out that the numbers of high-influence nodes out of DC, SH, and ISH models are close to each other with little difference, which suggests the principle is valid: the node with more in-degree is of more importance. The improved SH model is availed of utilizing the neighboring nodes’ degree information, which proves its advantage of high-influence nodes identification.(3)Mean value: Higher mean value indicates better identification performance. Thus among the three models, mean value of ISH is the highest, which shows its best identification capacity.
(a)
(b)
(c)
The above experiment indicates the global identification capacity of influence models. ISH model enjoys better performance in identifying low-influence and high-influence nodes over the other models, as well as its global identification performance.
To conclude, ITSVD model, the proposed precise personalized learning resources recommendation model based on trust and influence, shows better performance over its function, predication accuracy, and influential learners’ identification. The reason behind is that ITSVD model takes into consideration the mutual trust relation between learners, that is, learners with stronger trust relation enjoy more similarities, and impact of influential learners, that is, stronger trust indicates stronger influence capacity, as well as more impact over other learners. Thus, ITSVD could improve the accuracy and efficiency of recommendation system.
6. Conclusion
The future of online learning lies in massive online learning. Its priority depends on whether massive online learning could offer more precise personalized learning resources recommendation service. Along with the popular social networks, learner would more and more likely rely on social networks for learning and communication, which offers more clues to understand learners’ cognitive features, learning styles, and personal interest.
What we propose here is a precise personalized learning resources recommendation model based on trust and influence. Through looking into the inner social connection between learners, the paper unveils the implicit connection among learners, learning resources, and social relation. Experiments’ results on two real-world data sets, Canvas Network and Wanke, prove the accuracy and efficiency of ITSVD model’s application in education recommendation field. The paper opens new perspectives for precise personalized learning resources recommendation in massive online learning.
Future research will explore the quantitative analysis of social trust, multimode learning, and trust mechanism and include offline real interactive behavior data into trust analysis.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This work was supported by the Fundamental Public Welfare Research Program of Zhejiang Province (no. LGF22F020036), the Educational Science Planning Project of Zhejiang Province (no. GH2021562), and the National Experimental Teaching Demonstration Center of Computer Science in China.