Abstract
In the background of artificial intelligence (AI) era, deep neural networks (DNNs) also have a far-reaching impact on all walks of life. For the field of education, the integration of deep neural network technology and language teaching is more in-depth. Targeting at the problems of high costs and low efficiency of manual evaluation, this paper aims to study the evaluation of the English subjective questions by using deep neural networks. Firstly, the surface features of the English character information, including English antisense, negative information, semantic features, word frequency, sentences length, and words order, are combined; and the calculation method of the English sentence similarity based on multifeature fusion is established through the analytic hierarchy process (AHP). Secondly, the English text recognition is composed of long and short memory lost by aggregation cross entropy. The introduction of aggregation cross entropy can effectively improve the accuracy of text recognition. Finally, the evaluation model based on deep neural network is used to evaluate the English subjective questions. The experimental results, in terms of accuracy, recall rate, and effectiveness of the automatic recognition, show that the proposed method has high accuracy and can effectively improve the quality of the English teaching and realize a personalized teaching approach.
1. Introduction
The Republic of China holds many English tests every year, and through so many years of practice, the correction of English test papers has a relatively mature and comprehensive process. Among them, the correction of objective questions has been mechanized for many years, but the subjective questions are still corrected and marked manually. Manual correction has many disadvantages, such as the concentration of correction teachers or evaluators, the prevention of teachers’ violations, and the provision of marking venues. In addition, due to these issues, the evaluation process will consume a lot of human and material resources. At the same time, in order to ensure the fairness of the evaluation results of the English subjective questions, each test paper must be evaluated by several teachers at the same time, which will greatly increase the workload of the English teachers.
With the gradual advancement of modern educational English classroom design, the learning environment of the English teaching and learning is changing rapidly. These challenges promote the innovation and reform of the English learning methods. According to the English teaching, from the initial need to rely on the oral English teacher to learn English pronunciation, to the later use of tape recorder or phonograph to practice English listening, there are also modern multimedia teaching laboratories and corresponding technologies, which have played an important role in the process of English teaching. However, after in-depth research, it can be found that technology rarely has an impact on the English writing and teaching. The traditional evaluation systems have always been to solve the problem of pain points in English writing evaluation, accurate, timely, and comprehensive English evaluation feedback. However, classroom cooperation does not have efficient teaching means, and students’ writing ability cannot be effectively improved. Although the emergence of automatic scoring system has effectively solved the problem of timeliness, however, the current existing scoring systems generally have some problems, such as low scoring accuracy, certain loopholes in scoring rules, and single feedback effect, which limits the development of English subjective questions evaluation. Moreover, most of these systems are aimed at native English users, but they do not deal with the situation of nonnative English-speaking countries, in particular, where great efforts are required to encode the English grammar. Therefore, they are not suitable for Chinese users.
In this paper, we develop a scoring system for the evaluation of the English subjective questions based on the deep neural networks. First of all, we use the similarity calculation method of the English sentences based on multifeature fusion, and then the aggregation cross entropy approach is introduced to identify the English subjective questions. Finally, the subjective questions evaluation model based on deep neural network is adopted. Moreover, compared with other English subjective question evaluation methods, the results show that not only is the proposed method better than other methods in evaluation accuracy, but the recognition recall rate and recognition effectiveness rate are also better than other methods. This can effectively reduce teachers’ teaching tasks and improve the feedback effect. The innovations of this paper are as follows:(i)This paper aims to study the evaluation of English subjective questions by using deep neural network(ii)We firstly use the similarity calculation method of English sentences based on multifeature fusion(iii)The surface features of English character information are combined, and the calculation method of English sentence similarity based on multifeature fusion is established by using analytic hierarchy process(iv)The aggregation cross entropy approach is introduced to accurately identify the English subjective questions(v)English text recognition is composed of long and short memory lost by aggregation cross entropy
The remaining of this manuscript is structured as follows. Section 2 describes state-of-the-art related work. Key techniques of English subjective question evaluation are illustrated in Section 3. An evaluation model based on deep neural network is suggested in Section 4. Experimental evaluation and results are discussed in Section 5. Finally, Section 6 concludes the paper along with directions for further research.
2. Related Work
Foreign automatic evaluation systems for the English subjective questions have been developed for many years and have been very mature and applied to large-scale examinations. Most of these systems are aimed at native English users, but they do not deal with the situation of nonnative English-speaking countries. Therefore, they are not suitable for Chinese users. Domestic research in this area started relatively late, but some corresponding methods are also put forward. Liu et al. explained that, with the continuous development of artificial intelligence technology, the evaluation of English objective questions has been gradually handed over to the computer for processing [1]. However, in the process of scoring English subjective questions, it is easy to be affected by the subjective influence of the marking teacher, and the high-intensity marking work is easy to make mistakes. Aiming at this type of problems, an intelligent scoring method of English subjective questions based on deep learning is proposed. The intelligent analysis method based on big data can quickly compare students’ handwritten answers with correct answers, so as to realize the correct evaluation and automatic scoring of subjective questions. However, these methods have the problems of low scoring accuracies [1].
In order to realize the automation of English online examination, Chen and Liu suggested a model of English subjective questions evaluation based on the concept of sentence similarity [2]. Firstly, from the similarity of three types of text features such as (a) English keywords; (b) semantics; and (c) syntax, the possibility of English subjective questions evaluation is analyzed. Next, the data of answers are cleaned and classified, and the main steps of automatic evaluation are followed. These steps are as follows: (i) build a multifeature sentence recognition calculation model, (ii) take semantics as the core, (iii) make a comparative analysis of the English subjective questions based on the comprehensive similarity of multifeatures, (iv) theoretically improve the accuracy of the English subjective question evaluation, and (v) take the automatic evaluation system for English subjective questions as the prototype. Experimental outcomes show that the proposed calculation method of sentence acquaintance can evaluate the English subjective questions in the form of an intelligent evaluation system. Although the suggested model has good development prospects, however, on the whole, the accuracy is low [2].
Song and Wang proposed an English subjective questions scoring system based on word segmentation algorithm [3]. This method introduces word segmentation in detail and improves and studies the existing algorithms. The proposed method uses the combination of word segmentation technology and text similarity to automatically evaluate English subjective questions, from the recognition degree of text length and text word line, and then combined with influencing factors to form a comprehensive similarity. In the results using the main characteristics of the comprehensive examination subjects, three influencing factors were scientifically set to test the test paper [4]. The test paper questions were four subjective questions, and the standard answers were controlled within 100 words. Approximately, 50 electronic test papers were recovered from each experiment. The experimental results were compared with the experimental results of the original algorithm. The experimental results showed that the gap between the optimized evaluation algorithm and the original algorithm was relatively small; however, there is still room for improvement [3].
Wang et al. also suggested a scoring system [5], but, according to the existing subjective question evaluation method, it cannot effectively identify professional terms. Moreover, it is easy to omit the semantic relationship between texts in the process of scoring, resulting in a large difference between the scoring results and the manual scoring results. Due to this problem, the proposed scoring system cannot effectively meet the basic requirements of the real examination. To solve these problems, a subjective question evaluation method based on domain ontology and dependency syntax is proposed in the same work [5]. This method integrates various factors such as distance similarity, information similarity, and common words similarity and introduces the domain ontology into the process of subjective questions scoring [6]. The main purpose of the proposal is to improve the effectiveness of subjective questions scoring results. Experimental results show that compared with existing methods, the proposed method is closer to the manual scoring method in subjective questions evaluation. However, because the process is too complex, therefore, it has impacts on the accuracy of review [5].
3. Key Techniques for English Subjective Questions Evaluation
3.1. English Sentence Similarity Method Based on Multifeature Fusion
3.1.1. Similarity of Multifeature Fusion
On the basis of improving the semantic features, this paper comprehensively considers the word frequency, sentence length, and English word order of English sentences and puts forward the similarity calculation of English sentences based on multifeature fusion. In fact, this can effectively improve the accuracy of similarity calculation of the English sentences [7, 8]. The specific calculation flow is shown in Figure 1.
Suppose and are two sentences waiting to calculate the similarity, the feature similarity based on the surface of the sentence is , and the semantic feature similarity based on the English sentence is . The weighted fusion of the two feature similarities shows that the final similarity of the sentence is . The calculation process can be divided into three steps.
Step 1. Calculate the similarity of and surface features.
The features of sentence surface include word frequency, sentence length, and word order. The calculation process of the similarity is given as follows. The vector based spatial model method makes the calculation method of English sentence similarity based on word frequency, which mainly represents the sentence as a sentence, but mutually independent words form a vector, and uses the cosine value of the angle between the two vectors to calculate the similarity in English sentences. and are effectively mapped into the space of vector, expressed as and . The feature similarity based on English word frequency is expressed asIn formula (1), , where represents the frequency of the English word in , represents the index of inverse text frequency, represents the total number of English sentences in the corpus, and represents the number of English words and sentences included in the corpus.
Step 2. Interpolation.
The interpolation between the length of two sentences can reflect the similarity of English sentences [9, 10]. Generally, the similarity of English sentences can be inversely proportional to the interpolation of two sentence lengths. Assuming that the length of is and the length of is , the feature similarity based on English sentences is expressed asFor the same words in two sentences, the sequential relationship of word position must be considered. The reverse order number of the same words in two sentences can be used as a measure to express the similarity of the two sentences. If is the reverse order number of the sequence composed of the same words in and in , represents the largest reverse order number in the sequence of the same number of times in and . Based on word order feature, similarity is expressed as
Step 3. Weighted summation.
By weighted summation of the distribution weights of the similar calculation structures of the above three features of English sentences, it can be obtained that the similarity based on English sentences is expressed asIn formula (4), is expressed as an adjustable parameter, and and are satisfied. The similarity of the English sentences is calculated by combining the sentence features, and the final similarity of the two English sentences is expressed asIn formula (5), represents the features based on the surface of the English sentences, represents the improved semantic features, and the similarity value of English sentences meets , .
3.1.2. Calculation of English Features Weights Based on Analytic Hierarchy Process
The analytic hierarchy process (AHP) is a decision-making method combining quantitative calculation and qualitative analysis [11]. It mainly decomposes the complex multifactor decision-making problem into the problem of mutual comparison and weight calculation between different levels of self-factors. This paper uses analytic hierarchy process to select the feature weight of the English sentences [12, 13].
We construct the hierarchical structure model of English sentence similarity and construct the structural model of the English sentence similarity according to the hierarchical structure system of analytic hierarchy process, which is shown in Figure 2.
The comparison matrix is established [14]. The comparison matrix is established according to the relative importance of different sentence features in the criterion layer, which is represented in Table 1.
Among the surface features of English sentences, the feature of word frequency is an important factor affecting the similarity of English sentences, which is very important [15, 16]. The characteristics of English sentence length and word order have roughly the same influence on English sentences, so the degree of importance is also the same [17].
3.1.3. The Consistency Test
Indicators of consistency are expressed as given in the following formula:
In formula (6), represents the dimension of vector; that is, . The smaller the value of , the greater the consistency and vice versa. The is calculated as 0 according to formula (6), which indicates that the consistency is met.
After the value is obtained, the index of the random consistency must be defined prior to . The ratio formula of consistency is expressed as
According to formula (7), it can be calculated that is 0 and its value is less than 0.1. Therefore, the weight vector ; that is, the similarity weight of English sentences based on word frequency feature is , the similarity weight of English sentences based on sentence length feature is , and the similarity weight of English sentences based on word order feature is . It is concluded that the weight of English sentence similarity based on surface features is , and the weight of improved English sentence similarity based on semantic features is .
3.2. Recognition of English Subjective Questions Based on Deep Learning
Combined with the above multifeature fusion, the loss of aggregation cross entropy is replaced by the attention mechanism. When calculating different types of English characters, it is not necessary to consider the feature order [11]. The network is used to accurately predict and mark the number of characters in each category in English subjective questions to minimize the loss function. The calculation formula is expressed as follows:
In formula (8), represents the prediction results of English characters, and the occurrence times of English characters of category are equal to the probability of the given times of English labels under the condition of [18, 19].
The feature sequence dimension of subjective questions is , where represents the length of the sequence and represents the number of categories of English characters. This paper defines that the feature sequence vector of the English sentence output is , the feature vector of English words at time is , and the prediction probability of English words of category at time t is . The total probability of category in the whole English character sequence is .
Next, we adjust the lost English character function from the perspective of regression problem, and the calculation formula is expressed by the following equation:
In formula (9), represents the length of the predicted English text, represents the length of the label English text, and represents the number of white space characters in the English string; that is, .
In order to avoid the disappearance of gradients, the cumulative probability of the English character is normalized to , and the number of English characters is normalized to . The entropy of the intersection between and can be expressed as
As a final stage, in the final output probability matrix, the final sequence of the English text characters is obtained by greedy search method, so as to complete the recognition of English subjective questions.
4. Evaluation Model Based on Deep Neural Network
Based on the discussion of various mathematical models presented in the above section, we integrate them all to design a review model in this section. Combined with the above multifeature fusion English sentence similarity calculation and binary English subjective questions recognition, a review model based on deep neural network is constructed. Figure 3 shows the overall framework of the review model [20, 21]. The framework is composed of three layers: (i) the LSTM network layer; (ii) self-characteristic layer; and (iii) the full link layer. A detailed discussion of all layers is sketched in the subsequent sections.
The LSTM network layer:
The calculation formula of embedding English words into sequences and transferring them to the long and short memory network is
In formula (11), and represent the vector of English word input at time , respectively. The deep neural network model controls the data flow of English sentences in recursive operation through output, input, and forgetting gates. The following formula describes the depth neural network function:
At time , the deep neural network outputs a hidden vector , which reflects the English semantic content of English subjective questions at .
Self-characteristic layer:
This layer mainly describes how to extract the self-features of English sentences from vectors obtained from deep neural network. The English subjective questions vector and distance English data information vector have internal relations between adjacent sentences of English subjective questions. These relations are described in the network model. It should be noted that is the hidden layer of English subjective questions, is the vector of position, is the hidden layer of distance information, and is the vector of in position. Suppose represents the length of English sentences [22, 23]. Calculating the similarity between the position of the vector in the subjective question and the sentence at can be called the feature in the similarity of English sentences, which can be calculated as described by the following formula:
In addition, the similarity of the sentences of the vectors and at the same position of the subjective question is calculated, which is called the cross feature, and the calculation formula is expressed by the following equation:
Both subjective in questions features and cross features are connected with vectors and directly input into the next layer. In addition to the internal and cross features of English subjective questions, there are also two outputs in this layer, the hidden layer of the paper and the hidden layer of distance information. The different layers of the car are treated differently. One is to directly take the vector of the last position of and , and the second is to take the vector of the mean value in the corresponding time [24, 25]. Name these two vectors - and . The vector is input into the full link layer.
The full link layer:
Four vectors are obtained from the characteristic layer of subjective questions, which are , , , and . Connect these vectors into a vector. Then, input the vector to the layer.
The layer mainly classifies the English data output of the full link layer. The classification is mainly realized by equation and expressed by the following formula:
In formula (15), represents the vector of English data input, represents the weight vector of English words, and represents the offset of English words. Figure 4 visualizes the flow chart of subjective question scoring.
5. Analysis of Experimental Results
To evaluate the performance of the scoring method, the method is implemented in Python language, and the learning library used is PyTorch. The specific environment configuration of the experiment is shown in Table 2. This paper takes the test paper of the operating system course as the experimental material, selects 100 test papers of the test, inputs the English subjective questions into the computer, and also inputs the results of manual marking. When the results of manual marking are entered, each test question will be scored by two teachers, so as to improve the accuracy of manual marking. Among the answers of 100 test questions, 20 are randomly selected, with a full score of 20, as the test case, and the other 80 are used as the training set of system application, so as to make comparative analysis in all aspects.
Table 3 shows the comparison of sentence similarity in English subjective questions. As can be seen from Table 3, the similarity of English subjective questions sentences in the method proposed in this paper is significantly higher than that in the traditional model methods suggested in [1, 3]. Furthermore, the proposed method can improve the accuracy of English subjective questions evaluation.
5.1. Evaluation of the Accuracy
Accuracy is an important parameter to evaluate the English subjective question evaluation based on deep neural network described in this paper. It is expressed by formulas (16) and (17) to comprehensively evaluate the accuracy of subjective questions evaluation:
In formulas (16) and (17), represents the accuracy of single answer evaluation, and represents the total number of test papers.
5.2. Recall Rate of Subjective Questions Evaluation and Identification Results
The recall rate represents the number of results obtained by subjective question identification method, which is expressed by the following formula:
In formula (18), represents correctly identified information, and represents the information to be identified.
5.3. Effective Measurement of Automatic Recognition
This index mainly indicates the efficiency of the subjective questions recognition. According to this index, the effect of subjective questions recognition can be clarified, which is expressed by the following formula:
This paper selects the methods proposed in this paper based on deep neural network, methods suggested in [1, 3] for analysis, and compares the evaluation accuracy of the three methods, the recall rate of recognition results, and the effectiveness of automatic recognition. The results, in terms of accuracy, recall rate, and effectiveness of automatic recognition, are shown in Figures 5–7, respectively.
A detailed analysis of the data shown in Figure 5 shows that the effects of different methods are effectively reflected in this index experiment. The methods proposed in the text have relatively high accuracy in the evaluation of subjective questions and can evaluate multiple character information, while the methods proposed in [1, 3] can only evaluate a small amount of subjective questions and cannot analyze and recognize all character information of the subjective questions with high accuracy. This shows that the evaluation accuracy of the method proposed in this paper is high.
After verifying the accuracy of subjective questions evaluation, the recall rate of subjective questions recognition is verified and analyzed. According to the experimental results, as shown in Figure 6, it can be seen that there are great differences in the recall rate of subjective questions recognition of the three methods. The recall rate of recognition results based on deep neural network proposed in [1] and text is better, and it can recognize a variety of English text information. The recall rate of the recognition results of the method proposed in [3] is relatively low, and it cannot recognize all the characters of subjective questions. Therefore, this method cannot get good recognition results [26]. Combined with the above results, the effective measures of the automatic recognition of subjective questions in different methods are studied, which is shown in Figure 7.
The evaluation system of English subjective questions is a kind of artificial intelligence technology. It has the characteristics of timely feedback, comprehensive feedback, and objectivity of feedback. It can effectively reduce the workload of teachers, meet the needs of English learners to improve their subjective questions answering ability, assist teachers to complete English teaching activities, improve the quality of English teaching, and realize personalized English teaching. Therefore, this paper makes an in-depth study on English subjective questions evaluation based on deep neural network.
6. Conclusions and Future Work
The evaluation system of English subjective questions is a kind of artificial intelligence technology. It has the characteristics of timely feedback, comprehensive feedback, and objectivity of feedback. It can effectively reduce the workload of teachers, meet the needs of English learners to improve their subjective question answering ability, assist teachers to complete English teaching activities, improve the quality of English teaching, and realize personalized English teaching. Therefore, in this paper we studied an in-depth analysis on English subjective questions evaluation and proposed a scoring system based on the deep neural networks. The results, in terms of accuracy, recall rate, and effectiveness of automatic recognition, verified the validity of the proposed model.
In the future, we will consider other features of the English language and text and other more advance machine learning methods to improve the accuracy and robustness of the scoring system. Moreover, the grammar should be taken into account when designing automatic text recognition and review systems. Finally, the attention network and LSTM models along with aggregation methods should be used to minimize the training time of the model [6, 27]. The computational time would essentially increase through either increasing the number of features from the English language perspective or increasing the amount of data for learning purposes. In the future, we will study the impact of the learning process on the accuracy of the automatic text recognition and training times.
Data Availability
Data are available upon request from the corresponding author.
Conflicts of Interest
The author declares that there are no conflicts of interest.