Abstract
Sentiment analysis is a prominent research topic in natural language processing, with applications in politics, news, education, product review, and other sectors. Especially in the education sector, sentiment analysis can assist educators in finding students’ feelings about a course on time, altering the teaching plan appropriately and timely to improve the quality of education and teaching. For students, the sentiment analysis can identify emotions, academic performance, behaviour, and so on; the primary purpose of this research paper is to analyze students’ emotions, self-esteem, and efficacy based on closed-ended questionnaires. This paper proposes Quest_SA, which uses the sentiment analysis technique to identify students’ emotions based on the answer provided by a closed-ended questionnaire. The polarity value is assigned for each questionnaire scale. The students’ responses are then gathered using a closed-ended questionnaire, and the student’s emotions are classified using a polarity-based method of sentiment analysis. Finally, sentiment scores and emotion variance were used to evaluate the outcomes. According to the sentiment ratings, students have favourable sentiments and emotions such as unhappy, somewhat happy, and happy. The real-world closed-ended questionnaires such as emotional intelligence, Eysenck, personality, self-determination scale, self-efficacy, Rosenberg’s self-esteem, positive and negative affect schedule, and Oxford happiness questionnaires were used to examine the academic performance with the proposed sentiment analysis. This study inferred that the proposed sentiment analysis preprocessing method with polarity scores is as accurate as the standard value calculation.
1. Introduction
Sentiment analysis is a technique for detecting polarity and recognizing emotion toward a certain object, such as a person, a concept, or an activity. The purpose of sentiment analysis is to determine people’s opinions, identify the emotions they express, and categorize them as positive, negative, or neutral. Natural language processing (NLP) and machine learning (ML) techniques are used by sentiment analysis systems to identify, retrieve, and synthesize information and opinions from large amounts of text [1].
In general, sentiment analysis was done at three levels: document, sentence, and aspect. Document Level Sentiment Analysis discovers the user sentiments by evaluating the entire document. The goal of sentence-level research is to establish the polarity of individual sentences rather than the entire document; as a result, it is more precise. Finally, aspect-level sentiment analysis identifies elements or attributes mentioned in reviews and categorizes users’ reactions to them. The architecture of a broad sentiment analysis system is shown in Figure 1.
The whole system may employ a set of lexicons and linguistic resources. The document analysis module is a critical component of the system design since it employs linguistic resources to annotate preprocessed documents with sentiment annotations. The system's output—positive, negative, or neutral—is represented by annotations in several visualization tools. Depending on the sentiment analysis form, annotations may be utilized in various ways. For example, in document-based sentiment analysis, annotations may be applied to the entire document; in sentence-based sentiment analysis, annotations can be applied to specific sentences; and in aspect-based sentiment analysis, annotations can be applied to certain subjects or entities.
Sentiment analysis has been used in various settings to achieve a variety of goals, most notably in professional and economic networks. A few examples of well-known sentiment analysis business applications include product and service reviews [3], financial marketing approaches [4], and customer relationship management [5]. The most common use of sentiment analysis in social media apps is to analyze a company’s reputation on Twitter or Facebook [6] and investigate people’s reactions to a crisis, for example, COVID-19 [7]. Another important application area is politics [8], where sentiment research might aid candidates in their election campaigns.
Sentiment analysis and opinion mining have got a lot of attention in the educational community [9]. Unlike the previously stated sectors of social and commercial networks, which focus on a single user, education sentiment analysis research covers a variety of views, including teachers/instructors, students/learners, decision-makers, and institutions. Sentiment analysis is largely used to improve teaching, management, and assessment by examining learners’ attitudes and behaviour toward courses, platforms, institutions, and teachers.
Sentiment analysis is utilized to investigate the relationship between learners’ sentiments and drop-out rates in massive open online courses and the relationship between performance and retention and learners’ emotions [10]. Finally, sentiment analysis has examined several teacher-related aspects expressed in student reviews or comments on discussion forums in terms of teacher viewpoints [11].
Students are frequently obliged to engage in postcourse questionnaires at the end of each academic term to obtain information about their experiences. This procedure allows teachers and administrators to review student assessments and improve learning processes. There are both closed- and open-ended questions on the survey. Closed-ended questions, frequently used in Likert-scale inquiries, try to capture students’ evaluations in numerical ratings. Students can provide written comments or ideas in response to open-ended questions, which reflect their personal views and perceptions. This paper considers the closed-ended questions for identifying students’ emotions using sentiment analysis. The students’ responses are collected using a closed-ended questionnaire, and the students’ emotions are specified using a polarity-based sentiment analysis algorithm. The outcomes were assessed using sentiment scores and emotion variance. According to the sentiment ratings, students have positive sentiments and emotions such as unhappy, somewhat happy, and happy.
The remainder of this study paper is structured as follows: the research backdrop is described in Section 2, which includes sentiment analysis and a questionnaire. After that, in Section 3, the recommended methodology is explained. Finally, in Section 4, the proposed work’s performance is evaluated using standard questionnaires such as the Oxford happiness inventory, self-determination scale, Rosenberg’s self-esteem, self-efficacy, emotional intelligence, Eysenck personality questionnaire, and positive and negative affect schedule, and the conclusion and future work of this research work are presented.
2. Related Work
2.1. Sentiment Analysis
Sentiment analysis [12] evaluates emotional representation through language that comprises acquiring dynamic data, processing and analyzing data, and classifying a piece of text. The three main sentiment analysis tasks are facial expression identification, polarity detection, and affective computing [13]. Text sentiment analysis is a realistic approach for emotion mining in natural language processing widely used in public opinion monitoring, artificial intelligence, and corporate analytics. The three primary methods for text sentiment analysis are a machine learning-based technique, a dictionary-based approach, and a hybrid approach [14].
In machine learning-based techniques, sentiment classifiers are trained using a prelabeled data set. A classifier can be created to determine the polarity of textual inputs using methods such as naive Bayes, support vector machine, maximum entropy, and Word2vec, which are commonly used in sentiment analysis [15]. The dictionary-based approach uses a predeveloped lexicon, which includes the contradiction of words or phrases, to compute sentiment ratings and detect the polarity of a given text. The sentiment score is based on open source or bespoke sentiment dictionaries and can be computed using numerous semantic criteria [16]. A hybrid approach to sentiment classification combines machine learning and dictionary approaches. In general, the machine learning-based method is more effective although it takes a long time to classify the data [17]. The dictionary-based technique, on the other hand, has the advantage of not requiring any training data to determine sentiment and is substantially faster than machine learning in terms of computing time.
Customer product review [18], sale predictions [19], social media data [20], sarcasm detection [21], and the economic domain [22] are just a few examples of where sentiment analysis has been used. In the subject of education, sentiment analysis has recently gained interest. Reference [23] employed a lexicon-based approach to judging document-level polarity on students’ feedback to evaluate teachers.
Reference [24] introduced sentiment analysis provided by students at the end of a teacher evaluation course. The text processing capability of KNIME was utilized to build a pipeline for analyzing student feelings. This method recommends categorizing feedback as good, negative, or neutral using a sentiment score. Reference [25] proposed a hybrid technique for analyzing student input emotions that blends machine learning and lexicon-based methodologies. Textual feedback, usually given at the end of a course, provides useful insights into the general level of teaching and suggests practical ways to enhance teaching methods. Reference [26] planned to evaluate students’ text feedback and estimate instructional success levels using a lexicon-based technique. A lexicon of English sentiment phrases is built to get the polarity of terms as a linguistic source. In a sentiment-based eSystem,(i) for film reviewing, client happiness is measured using sentiment analysis with hybrid fuzzy and deep neural network [27], (ii) for modern business, knowledge discovery and sentiment analysis is used [28]. Selection for the best SVM hyperparameter values is done by applying natural optimizing techniques [29]. (iii) for nontraditional learning, expansion of hybrid reality-based education is done [30].
2.2. Open-Ended Questionnaire
Open-ended questions are survey questions that allow respondents to respond in an open text format, conveying their complete understanding, feelings, and knowledge. It implies that the answer to this question is not limited to a few options. Open-ended questions are commonly used in qualitative market research. A question with an open-ended response allows the viewer to answer depending on their knowledge and experience. The viewer’s detailed and elaborate knowledge leaves the potential for additional discussion and improvement. An open-ended question provides opportunities for both the researcher and the respondent to learn.
Figure 2 shows some examples of open-ended questionnaires.
The open-ended questionnaire has many merits, but it is difficult to analyze and organize the data into reports. Too many questions can directly harm the response rate. Moreover, the open-ended questionnaire may provide irrelevant information.
2.3. Closed-Ended Questionnaire
A questionnaire is a research tool that consists of questions or other prompts designed to gather data from a respondent. There are two types of questionnaires: structured and unstructured questionnaires. Quantitative data were collected via structured questionnaires. Quantitative questionnaires are used to evaluate or verify the accuracy that has already been developed. The questionnaire is meticulously constructed and designed to collect precise data. It also starts a formal investigation, contributes data, double-checks previously gathered data, and aids in invalidating any previous idea. Unstructured surveys are used to gather qualitative information. For example, qualitative questionnaires are used when collecting exploratory data to prove or reject a theory. They employ a minimal structure and a few branching questions, but nothing restricts a respondent’s options. To acquire specific responses from people, the questions are more open-ended. This research work considers structured quantitative questionnaires (closed-ended questionnaires). An investigation of the association between self-esteem and students’ academic performance was done in [31]. The authors of [32] worked on research contemplated on educational data mining.
Closed-ended questions, such as “yes” or “no” or multiple-choice questions, require respondents to choose from a limited set of predefined responses. Closed-ended inquiries are frequently used to gather statistical data from responders. It can take various shapes, but they are all driven by the requirement for respondents to have particular choices. Figure 3 depicts many sorts of closed-ended questions.
Table 1 shows the sample questions and the Likert scale used in each questionnaire.
The Oxford happiness questionnaire was developed by psychologists [33] at Oxford University. In the Oxford questionnaire, the (R) indicates reverse scoring. For example, if the student gives “1,” cross it out and change it to “6.” The emotional intelligence questionnaire is a self-evaluation tool. Self-awareness, self-regulation, motivation, empathy, and social skills are the five characteristics that characterize emotional intelligence, according to [34]. The self-esteem of an individual is assessed using Rosenberg’s self-esteem scale. The score of negative question items is inverted for analysis such that the positive and negative things have the same meaning. The final test result might be between 10 and 40. A person with a score of less than 14 has a problem with low self-esteem and needs assistance. The Eysenck personality test is a self-reporting tool [35]. It has 48 items: 12 for each of the personality traits of neuroticism, extraversion, and psychoticism and 12 for the lying scale. “Yes” or “no” is the binary response to each inquiry. Each dichotomous item was given a value of 1 or 0, with a maximum score of 12 and a minimum of 0. The self-determination scale (SDS) was developed to examine how self-determined people perform individually. It is thus regarded as a reasonably stable feature of people’s personalities that reflects (1) increased awareness of their feelings and sense of self and (2) a sense of control over their behaviour. The general self-efficacy scale is a 10-item psychometric scale that assesses optimistic self-beliefs in one’s ability to cope with various life challenges. Positive and negative affect schedule (PANAS) is a scale of several words that express feelings and emotions. The overall score is computed by adding 10 positive items together and then 10 negative items. For both sets of objects, the scores range from 10 to 50. A greater total positive score suggests a stronger beneficial influence. A lower total negative score suggests a lesser level of negative impact.
3. Methodology
Sentiment analysis is a computational study that evaluates individuals’ thoughts, assessments, and opinions regarding persons, situations, entities, concepts, activities, and items and their characteristics. Its goal is to find underlying opinions on a specific entity automatically. Sentiment analysis is mainly used for commercial applications such as product reviews, recommendations, marketing analysis, and public relations [36]. In the field of education, sentiment analysis is the process of determining a student’s feelings. In education, sentiment analysis can help with learning process improvement, performance improvement, study discontinuance reduction, teaching process improvement, and course satisfaction.
Emotion is commonly defined as a person’s mental state, including attitudes, feelings, and actions. Nowadays, public sentiment on a particular context can be easily known by extracting the opinions from a wealth of publicly available information on platforms such as Facebook, Youtube, Twitter, and Instagram.
Emotion detection, Reddit, Twitter, and others are becoming increasingly popular as a new study horizon in NLP. It could also be used in health services (as a tool for psychoanalysis), education (identifying learner dissatisfaction), and other fields [37]. This paper proposes Quest_SA to find students’ affective traits using polarity-enabled sentimental analysis in a closed-ended questionnaire. Figure 4 shows the architecture of the proposed Quest_SA.
The proposed Quest_SA contains two phases: online and offline processes. The students are requested to take an online closed-ended questionnaire in the online phase.
The following seven kinds of questionnaires are used: emotional intelligence, Eysenck personality, self-determination scale, self-efficacy, Rosenberg’s self-esteem, positive and negative affect schedule, and Oxford happiness. The selected scale value is converted into a polarity value for each question. Table 2 shows the polarity assignment for the questionnaire scale.
In the offline phase, the sentiment analyzer uses the polarity value to predict the students’ emotions. The polarity values are given based on the positive and negative sense. The positive Likert scale is given a positive score, and negative Likert scale is assigned a negative score.
4. Proposed QUEST_SA: Questionnaire Evaluation Using Sentiment Analysis
In this section, the performance of the proposed work is analyzed. Seven real-world questionnaires are used in this experiment. Table 3 shows the details of the questionnaire and the calculation in determining the result is shown.
The same standard calculation given in Table 3 is calculated for each questionnaire with the respective polarity value shown in Table 2. The result is given based on the polarity: if the result has negative polarity, then the scale is low; else, if the result has positive polarity, then the scale is high; and if the result is zero, then the scale is moderate. For instance, in Rosenberg’s self-esteem, if the score is negative, the result is low self-esteem, and if the score is positive, the result is high self-esteem.
5. Results and Discussions
This section evaluates the performance of the proposed through experiments. The research work uses seven kinds of questionnaires such as emotional intelligence (EI), Eysenck personality (EP), self-determination scale (SDS), general self-efficacy (GSE), Rosenberg’s self-esteem (RSE), positive and negative affect schedule (PNAS), and Oxford happiness (OH) and collects response from 1,000 students. The collected response was analyzed based on the standard and polarity-based evaluation. Finally, the obtained results are calculated and evaluated using MAE (mean absolute error) and accuracy.
Table 4 shows the emotional intelligence standard and the proposed polarity-based results. In addition, it shows the comparison of the result for all possible results. The result shows that the percentage of result deviation is very low between the standard evaluation and the proposed evaluation. Figure 5 shows the EI questionnaire scale value: low, average, and high. Table 5 gives the sample code of SentimentAnalyzer.
Table 6 shows the MAE and accuracy comparison of EI for different responses. Again, the lower number of the responses (200 and 400) produces a lower error.
Table 7 shows the Eysenck personality standard and the proposed polarity-based results. Figure 6 shows the questionnaire scale value for psychoticism, extroversion, and neuroticism. The standard and polarity evaluation produce the same result for all scale values.
Table 8 shows the MAE and EP accuracy comparison for a different number of responses. Again, the result produces zero error and 100% accuracy for all different numbers of responses.
Table 9 shows the self-determination scale standard and the proposed polarity-based results. Figure 7 shows the SDS questionnaire scale value: low and high.
Table 10 shows the MAE and SDS’s accuracy comparison for a different number of responses.
Table 11 shows the MAE and accuracy comparison of GSE for a different number of responses.
Table 12 shows the general self-efficacy standard and proposed polarity-based result. Again, the standard and polarity evaluation results for low-scale values. Figure 8 shows the GSE questionnaire scale value low and high.
Table 13 shows Rosenberg’s self-esteem standard and proposed polarity-based result. Figure 9 shows the RSE questionnaire scale value: low, normal, and high.
Table 14 shows the MAE and accuracy comparison of RSE for a different number of responses.
Table 15 shows the positive and negative affect schedule standard and the proposed polarity-based result. The standard and polarity evaluation produce the same result for all the scale values.
Figure 10 shows the PANAS questionnaire scale value: positive, negative, and neutral.
Table 16 shows the MAE and accuracy comparison of PANAS for the different numbers of responses. The result produces zero error and 100% accuracy for all different numbers of responses.
Table 17 shows the Oxford happiness questionnaire standard and proposed polarity-based result. Figure 11 shows the OH questionnaire scale value happy, moderately, happy, and unhappy. Table 18 gives MAE and the accuracy of the Oxford happiness questionnaire.
6. Conclusion
The task of sentiment analysis for questionnaire data was the focus of this study. The main goal was to develop a mechanism for analyzing questions and students’ emotions based on closed-ended responses. Quest SA is a tool for assessing questionnaire sentiments and students’ emotions proposed in this paper. The students’ replies are gathered using a closed-ended questionnaire, and the students’ emotions are identified using polarity-based sentiment analysis in this study. The performance of the study task is evaluated using seven real-time surveys (emotional intelligence, Eysenck personality, self-determination scale, general self-efficacy, Rosenberg’s self-esteem, positive and negative affect schedule, and Oxford happiness). The suggested Quest_SA accurately predicts students’ emotions compared to established questionnaire evaluation methods. The proposed system’s accuracy is comparable to that of the traditional method. When opposed to traditional evaluation, categorizing the result is simple. Because the traditional evaluation with range of values takes long time than the proposed evaluation with polarity score, multimodal SA techniques are probably going to be in high demand in the near future.
Table 15 shows the MAE and OH accuracy comparison for a different number of responses. Again, the results proved that the proposed system works similarly to the traditional system with good accuracy.
Data Availability
The data that support the findings of this study are not available in any public repository.
Conflicts of Interest
The authors declare that there are no conflicts of interest.