[Retracted] XAI-Based Reinforcement Learning Approach for Text Summarization of Social IoT-Based Content

Abdeljaber, Hikmat A. M.; Ahmad, Sultan; Alharbi, Abdullah; Kumar, Sudhir

doi:https://doi.org/10.1155/2022/7516832

Security and Communication Networks

On this page

Abstract Introduction Related Work Experimental Results and Analysis Conclusion Data Availability Conflicts of Interest Authors’ Contributions Acknowledgments References Copyright Related Articles

Research Article Retraction

!

This article has been Retracted. To view the article details, please click the ‘Retraction’ tab above.

Special Issue

Recent Advances in IoT and Blockchain-Based Security/Privacy in Advent Technology

View this Special Issue

Research Article | Open Access

Volume 2022 | Article ID 7516832 | https://doi.org/10.1155/2022/7516832

[Retracted] XAI-Based Reinforcement Learning Approach for Text Summarization of Social IoT-Based Content

Hikmat A. M. Abdeljaber,¹Sultan Ahmad,²Abdullah Alharbi,³and Sudhir Kumar⁴

Academic Editor: Mukesh Soni

Received24 Apr 2022

Revised30 May 2022

Accepted11 Jun 2022

Published04 Aug 2022

Abstract

The purpose of automatic text summarising technology is to condense a given text while properly portraying the main information in the original text in a summary. To present generative text summarising approaches, on the other hand, restructure the original language and introduce new words when constructing summary sentences, which can easily lead to incoherence and poor readability. This research proposes a XAI (explainable artificial intelligence)-based Reinforcement Learning-based Text Summarization of Social IoT-Based Content using Reinforcement Learning. Furthermore, standard supervised training based on labelled data to improve the coherence of summary sentences has substantial data costs, which restricts practical applications. In order to do this, a ground-truth-dependent text summarization (generation) model (XAI-RL) is presented for coherence augmentation. On the one hand, based on the encoding result of the original text, a sentence extraction identifier is generated, and the screening process of the vital information of the original text is described. Following the establishment of the overall benefits of the two types of abstract writings, the self-judgment approach gradient assists the model in learning crucial sentence selection and decoding the selected key phrases, resulting in a summary text with high sentence coherence and good content quality. Experiments show that the proposed model's summary content index surpasses text summarising ways overall, even when there is no pre-annotated summary ground-truth; information redundancy, lexical originality, and abstract perplexity also outperform the current methods.

1. Introduction

With the rapid development of the Internet, the network contains a massive base and various forms of data content. Quickly locating critical information from it is a primary problem for efficient information retrieval. For text data, automatic summarization technology can extract the core content from a given corpus and describe the main text of the original text with a relatively summary text, which is conducive to reducing the storage cost of text data and is a necessary means to improve the efficiency of text data retrieval. It has important practical significance and application value for the further realization of information integration.

Existing automatic text summarization methods can directly select basic sentences or paragraphs from the original text and generate abstract texts by sentence extraction [1]. Generative text summarization methods have become a research hotspot in text summarization [2]. Generally speaking, the generative text summarization method firstly encodes a given original text and obtains a vector (embedded) representation that can cover the original text information from the word and sentence levels. Finally, the above feature codes are decoded, that is, according to the decoding result, the corresponding vocabulary is selected from the given language (dictionary) to form an abstract text. Finally, the original text is re-expressed in text form. It can be seen that compared with the extraction method, the implementation of generative text summarization is more complex. Still, the summary text produced by it is more flexible and rich in vocabulary expression, and the effect of condensing the critical information of the original text is more ideal [3].

However, the problem faced is that the generative text summarization method needs to go through the process of original text encoding, encoding parsing, and feature decoding, and organize sentences with richer vocabulary to convey the original text, so it is easy to lead to the coherence of the generated summary sentences. In addition, the current generative text summarization methods involve artificially-annotated summaries for supervised training [3], resulting in the existing productive text summarization methods often facing the. Due to a scarcity of valuable resources, only relying on the “summary truth value” with strong sentence coherence in advance, and improving the sentence coherence of the abstract generated by the model according to the supervised training method, may have significant resistance in the practical applications [4]. Therefore, based on the generative text summarization model, this paper seeks an effective mechanism that can improve the sentence coherence of the summary generation model without intervention.

Specifically, on the one hand, in the summary text generation stage, the encoder (module A) first encodes the given source document (source document) to obtain the embedded representation of the original text; on this basis, the coherence measurement module (module B) The Transformer-XL [5] encoder is used further to encode the embedded representation of the original text, parse the context-related content features, and set a “key sentence classification layer” at the top of the coherence measurement module to generate sentence extraction identifiers, to filter out (or be the key) sentence coding results, to describe the process of extracting key sentences from the original text through the coherence measurement module; finally, the decoder (module C) is based on the critical sentence coding output by the coherence measurement module. And produces decoded results for the “extracted” key sentences, i.e., the original vocabulary distribution.

On the other hand, in the sentence coherence enhancement stage, the model XAI-RL first obtains the original vocabulary distribution output by the previous stage decoder (module C) and generates two types of summary texts by “selecting by probability” and “selecting by Softmax-greedy,” and the two types of abstracts are re-encoded by the encoder (module A); after that, the re-encoding results of the two types of abstracts are parsed by the coherence measurement module (module B), and the re-encoding results of the two types of abstracts are parsed by the coherence measurement module (module B). The recurrent self-attention weight of the semantic segment (segment) [5] is used as the coherence benefit of the summary sentence; the ROUGE [6] score of the generated summary text and the “pseudo-summary truth value” is used as the content income of the summary sentence, so that the sum of the above two benefits is calculated through the coherence measurement module to calculate the respective overall benefits of the two types of summary texts; here, the “true value of the pseudo-summary” is the optimal sentence set extracted from the original text through the ROUGE score. Second, construct the “cross-entropy loss” of the two types of abstracts, adopt the “self-critical policy gradient” in XAI-Based Reinforcement Learning [7], and use the “overall return difference” of the two types of abstracts to reward or reward the model parameter gradient. Penalty is forcing the overall return of the summary generated by “Softmax-greedy selection” to approach the overall recovery of the summary generated by “selecting by probability,” and improving the general baseline level of “Softmax-greedy selection” through “probability exploration,” and then improve the model value of the summary text generated by XAI-RL in terms of sentence coherence and sentence content. Finally, without the intervention of the abstract truth value, the abstract text with high sentence coherence and good content quality is generated.

To sum up, this paper proposes a coherence-enhancing-oriented truth-free text summarization model (XAI-RL), which combines “extraction and generation” to generate summary content based on the set of critical sentences extracted from the original text. At the same time, by re-encoding, coherence, and content revenue calculation of the initially generated summary text, based on the actual vocabulary distribution of the decoder, the “selected by probability” is obtained compared with “selected by Softmax-greedy.” Gain advantage” guides model gradient updates by maximizing this “profit advantage” to produce summary texts with higher sentence coherence. The experimental results show that the ROUGE [6] and METEOR [8] scoring indicators of the model XAI-RL are still better than the existing text summarization methods on the whole, even under the restriction of only given the original text. The summary texts also outperform existing methods in terms of sentence coherence, content importance, information redundancy, lexical novelty, and summary perplexity.

Currently, the sequence-to-sequence (Seq2Seq) structure based on the idea of “encoding-decoding” is the primary method for dealing with generative text summarization tasks [3]. The encoder and decoder in the traditional Seq2Seq structure often use the recurrent neural network (RNN) [9], long short-term memory (LSTM) [10], and bi-directional long short-term term memory (Bi-LSTM) [11], to generate summary texts with better sentence quality, many scholars have made related improvements to the summary mentioned above and generation models based on recurrent neural networks and their variants.

Author [12] proposed a hierarchical encoder that can capture the discourse structure of the input text from two levels of words and segments and injects the discourse structure features into the decoder to assist the decoder in generating summary texts. A high ROUGE score was achieved on the task of developing academic paper abstracts. Author [13] introduced an intra-decoder attention mechanism on the decoder side, that is, observing the first t − 1 bits of decoding results when decoding the t-th bit, the attention weight prevents the decoder from generating the duplicate content, which effectively reduces the redundancy of the summary text sentence content; at the same time, this work combines the Teacher Forcing algorithm [14] and the self-judgment policy gradient [7] to construct a hybrid XAI-Based Reinforcement Learning objective, which makes the model effectively avoid exposure bias when processing the original text, and generates summary text with high evaluation accuracy. Author [15] first divided the input original text into multiple segments, and based on Bi-LSTM, the model builds multiple agents; after that, each agent parses the allocated element, and transmits the parsing results of the part between the agents according to the multi-agent communication mechanism, and finally forms a “global observation” of the original text, which is defined by “Global observation” that generates summary text according to the “encoding-decoding” idea.

Although the above models have achieved improvement in the accuracy of summary generation, the recurrent neural network and its variants are all time-step-based sequence structures, which seriously hinders the parallel training of the model [16–18], resulting in the inference process being limited by memory, resulting in reduced encoding and decoding speed of the summary generation model, and increased training overhead [19–23]. On the other hand, the above works optimize the model to maximize the ROUGE index or maximum likelihood without considering the coherence or fluency of the summary sentence [24–26] and relying on the ground-truth value of the annotated summary text in advance. With supervised training, the data cost involved in model training is high. Therefore, further improvements to the summary generation models based on recurrent neural networks and their variants are needed [27–29].

The work on the coherence of summary sentences also includes: author optimize the model by encoding, decoding, and re-encoding the original text, constructing a summary similarity loss and a text reconstruction loss. A good language model calculates the negative log-likelihood of the generated summary text to measure sentence coherence; author used the BERTSCORE indicator to construct a distributed semantic gain and combined the payment with the self-judgment policy gradient to evaluate the model optimization. Human evaluation results show that this benefit can make model summaries more coherent; author optimize the extractor by applying an advantage actor-critic (A2C) at the sentence level after pretraining the decoder, to ensure that the model paraphrases the correct key sentences to generate coherent and fluent summaries [30, 31].

The above models optimize summary coherence to minimize the perplexity of the generated summary text. However, it is worth noting that the existing works all use manual evaluation methods when evaluating the coherence of summary sentences. There is a lack of a mechanism or process for automatic measurement of sentence coherence within the summary generation model [32–34]. To sum up, the current generative text summarization methods should meet or solve the following problems: first, it can generate coherent and highly readable summary texts based on the given original text; the processing mechanism for automatic coherence measurement of the generated summary sentences; thirdly, the labeling dependence of the summary ground-truth data in the model training process should be minimized to reduce the model training cost [35–37].

3. XAI-RL Summary Generation Model

3.1. Overall Model Architecture

As shown in Figure 1, the XAI-RL model is mainly divided into two stages: first, the summary text generation stage (① to ⑥ in Figure 1, marked by blue lines). First, the encoder (module A) uses the AL-BERT component to obtain the encoded representation Ea of the original text set D, and the coherence measurement module (module B, the top layer is the Sigmoid classification layer) obtains auxiliary information H and extracts the set of critical sentences Input_abs; here, the additional information H and the collection of essential sentences Input_abs are regarded as the feature analysis result of the encoded representation Ea; then, the decoder (module C) decodes Input_abs and H, and afterword search, the initial output for the content of the critical sentences is generated abstract text [38, 39]. It is worth noting that, as shown in Figure 1, when Input_abs and H are decoded by the decoder (module C) in the summary text generation stage, it is necessary to adopt “select by probability” and “select by Softmax-greedy” based on the original vocabulary distribution. Two strategies are used for vocabulary selection, resulting in summary texts under different selection strategies.

Second is the sentence coherence reinforcement stage (⑦ to ⑫ in Figure 1, marked by orange lines). First, the model XAI-RL resubmits the preliminary generated summary text in stage 1 (“selected by probability” or “selected by Softmax-greedy”) to the AL-BERT encoder (module A) for “summary re-encoding”; secondly, based on re-encode, the result to obtain the semantic segment-based recurrent self-attention weight embedded in the L-layer encoding component (Transformer-XL_Encoder) in the coherence measurement module (module B), which is used as the sentence coherence score of the summary text generated in stage 1, denoted as coherence benefits (rewards_coherence), thus introducing a sentence coherence measurement mechanism inside the model; in addition, calculating the ROUGE score of the summary text and pseudo-summary generated in stage 1 denoted as content revenue (rewards content); here, “ “Pseudo abstract” is composed of the top K sentences with the highest scores after calculating the ROUGE score of each sentence in the original text and the whole original text. Finally, the overall revenue generated by the model XAI-RL abstract (referred to as rewards) is composed of the sentence coherence benefit of the abstract text and the sentence content benefit of the abstract text. The general use (including both content and coherence) updates the model XAI-RL parameter gradient to guide the model to generate sentences with high coherence and good content quality without the intervention of artificially annotated ground-truth summaries (such as relying only on pseudo-summaries)—abstract text.

3.2. Stage 1: Summary Text Generation Stage

3.2.1. Text Encoding Representation

In the summary generation stage shown in Figure 2, the model XAI-RL utilizes the pretrained AL-BERT components as the encoder to obtain the encoded representation of the input text set E, denoted as F_a. Specifically, given n original texts, E = [E¹; E²; …; Eⁿ], where the i-th text and its i_m sentences are represented as Eⁱ = [, , …, ], and the jth sentence = [, , …, ] in the text E contains ij_len words; in addition, the model XAI-RL marks the [CLS] symbol before each sentence in the input text set E to distinguish different sentence. In particular, AL-BERT has fewer parameters and a faster encoding speed compared to BERT. Finally, after being processed by the AL-BERT encoder, the encoding of the n texts in the text set E is represented as F = [F¹; F²; …; Fⁿ]; among them, Fⁱ = [[, ], [, ], …, [, ]] is the coded representation of the i-th text; = [, , …, ], are sentenced as the encoded representation of sentij and [CLS] symbols, respectively.

3.2.2. Key Statement Selection and Auxiliary Information Acquisition

As shown in Figure 2, in the summary text generation stage, the coherence measurement module (the top layer is the Sigmoid classification layer) is responsible for parsing the text-encoded representation F_a output by the AL-BERT encoder to extract contextual information H across semantic segments; in addition, the top-level Sigmoid classifier discriminates critical sentences from the context information G to generate the extraction label Label_ext and then outputs the set of introductory sentences (encoded) inputs. In particular, G can provide corresponding context information for the collection of critical sentences Input_abs in the subsequent decoding process, thereby assisting the decoder to generate abstract text that summarizes the gist of the original text. Specifically, the coherence measurement module divides the encoded representation F_i of each text Dⁱ into i_u semantic segments of equal length (length l). For the τ + 1st segment in the text Dⁱ, the hidden state of the nth layer Transformer XL Encoder in the coherence measurement module is calculated according to formula:

Among them, SG(∙) means stop gradient update during backpropagation [5]; Positionwise_FFN(∙) means position-wise feedforward network; Relative_MHA(∙) means adopting. The relative multi-head attention layer of relative position encoding [5] is used. Here, the hidden layer state h͂n − 1τ + 1 is processed according to formula (2); where represents the nth layer Transformer - Self-attention weight for τ + 1st segment in XL Encoder. As a result, as shown in Figure 2, the hidden layer state output by the L-th Transformer-XL Encoder of the coherence measurement module is G = [G¹; G²; …; Fⁿ]; represents the final hidden layer state corresponding to the i_u sentence in the i-th text Di. Further, before the model XAI-RL adopts the AL-BERT component (module A) to encode the input text set D, the starting position of each sentence in D is marked with [CLS] symbols to distinguish from each other, and these symbols correspond to the hidden layer. A representation can characterize the statement following it [19]. Therefore, as shown in Figure 2, the coherence measurement module (module B) uses the hidden layer state of each [CLS] symbol in the secret layer state G to represent each sentence in the text set E after being passed by the L-layer coding and denoting is G_cls = [; ; …; ]; here, G_cls = [, , …, ] is the vector representation of i_m sentences in the i-th text Eⁱ. Moreover, as shown in formulas (1) and (2), Gicls should also contain corresponding context information (such as R and V)

After that, as shown in Figure 2, the Sigmoid classifier at the top level of the coherence measurement module uses the above text statement to represent G_cls to generate the extraction label Label_ext = [; ; …; ] to determine whether each sentence in the original text is critical; among them, = [, , …, ] represents the sentence extraction result of the i-th text Eⁱ; ∈ {0, 1} Indicates whether the nth sentence in the text Di should be extracted (if it is 1, it means that the sentence is a crucial sentence and should be removed; otherwise, it is not considered). Therefore, as shown in formula (3), the original text encoding result Ea output by the AL-BERT encoder (module A) and the sentence extraction result Label_ext output by the coherence metric module (module B) are multiplied bit by bit, and from F_a Filter out the set of critical sentences (encodings), denoted as Inputs. It is worth noting that the model XAI-RL uses the binary cross-entropy sentence extraction loss Loss_ext as shown in equation (4) to pretrain the coherence measurement module (module B) before the summary text generation stage begins. Specifically, Algorithm 1 is first used to extract pseudo-summaries from the given original text; in which the first line initializes the pseudo-summary set; starting from the second line, each textbook in the input text set E is processed separately; Line 6, respectively, calculate the ROUGE index between each sentence s_j in the text Dⁱ and the remaining documents (i.e., Dⁱ \s_j); Lines 7 to 11 indicate that the first k sentences with the most considerable ROUGE index value are taken as the pseudo-abstract Plabelⁱ corresponding to the text Dⁱ; Line 12, the text Di corresponding to the pseudo-abstract Plabelⁱ is added to the pseudo-summary set. Thus, as shown in equation (4), by comparing the extraction probability ŷk ∈ (0, 1) of the kth sentence in the i-th text in the coherence measurement module (output by the top-level Sigmoid classifier) and the pseudo-summary set. The kth sentence extraction result y_k ∈ {0, 1} of the i-th pseudo-summary Plabelⁱ is compared. Finally, without the intervention of the abstract truth value, the coherence measurement module is guided to obtain the relevant parameter information of key sentence extraction in advance [40].

4. Experimental Results and Analysis

This chapter conducts a series of experimental analyses on the ground-truth-dependent text summarization model (XAI-RL) proposed in this paper for coherence enhancement and discusses the model’s effectiveness in the summary generation process and summary generation quality. This paper uses Python 3.7 and Tensorflow-1.15 to implement the model, and the experimental running environment is GPU, NVIDIA GeForceGTX 1080Ti, 11 GB.

4.1. Dataset and Experimental Setup

First of all, this paper uses two typical automatic text summarization datasets, CNN/Daily Mail and XSum, for experiments [41, 42]. They both use news reports as text data and contain corresponding “gold standard” summarization ground-truth documents. In this paper, the original data set is divided into a training set, validation set, and test set. The training set is used for model training, the validation set is used for model parameter selection, and the test set is used for model evaluation. In particular, the “gold standard” summaries do not participate in the model XAI-RL training process and are only used for summarization generation quality assessment. As shown in Table 1, the average length of the original text and abstract text in CNN/Daily Mail is larger than XSum; XSum is a sentence written by humans as the notional truth value. Compared with CNN/DailyMail, the novelty of the ground truth in XSum is higher and contains more words that do not appear in the original text.

Secondly, in terms of model setting, let the word vector dimension be E, the number of hidden layer units be H, the number of self-attention heads be A, the feedforward layer dimension size is F, and the XAI-RL model adopts AL-BERT_large [30] (E = 128, H = 1.024, A = 16, F = 4.096) as the encoder, and the coherence metric module consists of L = 3 layers Transformer-XL Encoder (E = 1.024, H = 2.048, A = 32, F = 4.096), and the decoder consists of R = 6 Transformer-XL Decoder (E = 1.024, H = 2.048, A = 32, F = 4.096). In the summary text generation stage, a beam search algorithm with a width of 4 is used for vocabulary selection. The maximum length of the generated summary is determined by the average compression ratio of the original document and the summary document in the dataset (the ratio of document length), and the number of discarded words is low. In the sentence at 3, the coherence metric module and the decoder adopt the Adam optimizer [19] with a learning rate of 1E-3 and 0.05, respectively. The learning rates of both decreases with the number of iterations. The number of batch samples (batch_size, which is the size of the input text set D) is 16. In the sentence coherence enhancement stage, β1 = 0.3 and β2 = 0.2 are taken from the text content income shown in equation (6), and γ = 0.7 in the total income of equation (6). When the model is trained with the CNN/Daily Mail dataset, the input text set D takes the first M = 8 optimal records in one iteration for “experience playback” during the coherence enhancement stage; when training with XSum, D is in one iteration, the first M = 4 optimal records are taken.

Then, in terms of comparison methods, the abstract generation model XAI-RL proposed in this paper is compared with the existing extractive and generative automatic summarization methods. Among them, for the extraction method, MMS_Text, SummaRuNNer and HSSAS are used; for the generative method, Pointer-Generator + Coverage, Bottom-up, DCA (deep communicating agents), BERTSUMEXTABS, and PEGASUS.

Finally, for the evaluation indicators, this paper adopts ROUGE-N [6] (including ROUGE-1 and ROUGE-2, formula (12)), ROUGE-L (formula (13)) and METEOR [8] (formula (14)). The index evaluates the content quality of the generated text, and at the same time cooperates with manual evaluation to assess the summary text generated by the related model in terms of sentence coherence, content redundancy, and content importance. Here, in ROUGE-N, n represents the length of n-gram (n-gram), {RS} means the reference abstract, Count_match(grain) represents the same number of n-grams in the generated abstract as in the reference abstract, and Count(grain) is the total number of n-grams in the reference abstract. In ROUGE-L, X is the generated abstract, Y is the reference abstract, LCS(X, Y) represents the length of the longest common subsequence between the generated abstract and the reference abstract, m is the developed abstract length, and n is the reference abstract length; in METEOR, m is the number of tuples matching the reference abstract in the generated abstract, r is the connection abstract length, c is the developed abstract length, α, γ, β are balance parameters, dh is the generated conceptual and the reference abstract of the number of common subsequence’s.

4.2. XAI-RL Model Summary Generation Process Discussion

To explore the influence of different modules in the model XAI-RL on the experimental results, six ablative combinations, as shown in Table 2, are implemented in this paper. Specifically, Combination 1 uses module A (AL-BERT encoder) and module B (coherence metric module, only Transformer-XL Encoder) without a replaceable top layer for encoding and then uses module C (decoder) for encoding and decoding to produce digest. Combination 2 adds a sigmoid classification layer to module B based on combination 1, which aims to perform critical sentence selection on text-encoded representations and then generate summaries. Combination 3 has the same structure as Combination 2, but it pretrains module B; in particular, the above three combinations are all supervised training using the training set “gold standard” as the ground truth. Finally, combination 4 adopts the structure of Combination 3. In addition to pretraining module B, coherence enhancement is only performed by maximizing the coherence gain. During the reinforcement process, the extracted pseudo-summary is used as an alternative truth value; Combination 5 is similar to Combination 4 in that it only enhances the coherence by maximizing the content revenue; Combination 6 is the complete XAI-RL model in Figure 1, and the extracted pseudo-summary is still used as the alternative truth value.

CNN/Daily Mail and XSum validation sets evaluated the above six ablative combinations. The experimental results are shown in Tables 3 and 4 and Figures 3 and 4.

First, the evaluation results of Combination 2 are better than Combination 1, which indicates that after extracting key sentences by module B, the decoder can decode the critical content and generate higher-quality summaries. Second, combination 3 is better than combination 2, indicating that pretraining can make the parameter configuration of module B more reasonable and then select critical sentences more reasonably. Then, the evaluation results of combination four and combination 5 are better than those of combination 3, indicating that the revenue and sentence coherence enhancement method constructed in this paper can effectively improve the quality of abstract content. In particular, the ROUGE-L and METEOR indicators of combination 4 are better than those of combination 3, reflecting the improvement of sentence coherence by the coherence measurement and reinforcement in this paper. Finally, the combination of all the mechanisms 6 has the best evaluation result, reflecting the effectiveness of each module of the proposed model XAI-RL in a summary generation.

To sum up, for the XAI-RL model, firstly, by comparing combination two and combination 3, it can be found that after pretraining the coherence measurement module by the pseudo-summary, meaningful sentences and contextual semantic information can be better identified from the text encoding representation, to provide the decoder with semantic benchmarks and auxiliary input to generate summary content that can accurately summarize the main idea of the original text; secondly, by comparing combination three and combination 4, it can be found that the self.

4.3. Comparison between the XAI-RL Model and Existing Text Summarization Models

In this section, the accuracy of the XAI-RL model is compared with the existing extractive and generative methods on the test set to evaluate its summary generation quality. First, the evaluation results (average of 3 times) of the XAI-RL model and the comparison method on the CNN/Daily Mail dataset are shown in Table 5 (ROUGE-AVG is the average of ROUGE-1, ROUGE-2, and ROUGE-L). On the one hand, as shown in Figure 5, the evaluation results of the XAI-RL model are generally better than the existing extraction methods. This model outperforms other extraction-based baseline models in ROUGE-1 and ROUGE-2 indicators, indicating that it can effectively obtain the original subject information.

At the same time, its scores on ROUGE-L and METEOR indicators are higher than other extraction-based baseline models which indicate that the model can ensure the coherence of generated sentences when paraphrasing the acquired vital sentences. The core ideas of the compared extraction methods (MMS_Text, SummaRuNNer, Refresh, and HSSAS) can be summarized into three categories: one is to convert text into a graph structure (such as MMS_Text), by scoring nodes (sentences) to extract essential sentences to form summary texts; the second is to mine the latent features of the reader through the encoder and extract summary sentences in the order of probability matrix or sentence arrangement (such as SummaRuNNer and HSSAS). The third is to use XAI-Based Reinforcement Learning to build quality gains, and after updating the sentence selection strategy to maximize profits, extract abstract texts from the original document (such as Refresh). However, for the XAI-RL model proposed in this paper, its core idea is “extract first, then generate.” The coherence measurement module of the model can identify and extract critical sentences in the original text after pretraining, thus prompting the decoder to pay attention to the actual content. In addition, when decoding and generating, the XAI-RL model outputs auxiliary information H containing contextual semantics to the decoder, which further enriches the text feature information inside the model, and finally makes the quality of the abstract text generated by the model XAI-RL better than the “single” extraction model. On the other hand, as shown in Figure 6(b), the XAI-RL model is compared with existing generative methods (Pointer-Generator + Coverage [11], Bottom-up, DCA [15], BERTSUMEXTABS [19], and PEGASUS) and also achieved better accuracy overall. The model outperforms other generative baseline models in ROUGE-1 and ROUGE-2 indicators, indicating that it can correctly paraphrase the original text message. At the same time, its scores on both ROUGE-L and METEOR metrics are higher than other generative baseline model, which indicates that this model is more capable of generating coherent and smooth summary content. Its performance improvement can be attributed to: first, in developing the abstract, as shown in Figure 2, the model XAI-RL is based on the pretraining components (such as the AL-BERT encoder and the pretraining coherence measurement module), and the text encoding results are further used. The semantic segment is the division unit and is additionally encoded by the L = 3-layer Transformer-XL component through the semantic segment-based recurrent self-attention mechanism to increase the strength of feature parsing. Second, in the coherence enhancement process shown in Figure 4, the model XAI-RL re-encodes the generated summary text to calculate the coherence gain; at the same time, the extracted pseudo-summary is used to create the summary to calculate the content gain by maximizing the two. The weighted sum of revenue strengthens the model summary text generation process. Further, it improves the model text generation quality from the content and sentence coherence levels.

Secondly, the evaluation results of the XAI-RL model and the compared method on the XSum dataset (average of 3 times) are shown in Table 6, and the corresponding histogram is shown in Figure 6. Overall, the model still achieves optimal results. In particular, the XSum dataset is only used to test generative methods due to its high novelty corresponding to the “gold standard” abstract. The results shown in Table 6 and Figure 6 further illustrate the “extract first, then generate” design principle followed by the model XAI-RL, the recurrent self-attention weight based on semantic segment, and the reinforcement process based on content benefit and coherence benefit can effectively improve the summary generation quality.

5. Conclusion

Using automated text summarization technology to condense the core content of the text is a necessary means to reduce the cost of text data storage and improve the efficiency of information retrieval. To quickly generate high-quality and readable summaries of texts while avoiding the ground-truth dependence of model training, the XAI-RL-based text summarization model for coherence enhancement (XAI-RL) proposed in this paper utilizes the Transformer-XL of the attention mechanism to build a coherence measurement module and uses the extracted pseudo-summary to pretrain it, which can effectively identify and remove the important textual information. In addition, it can automatically measure the coherence of the generated abstract during the re-encoding process and generate text coherence benefits, which can be introduced into the coherence enhancement process of the model, which can promote the model generation to be closer to the original theme, more readable summary content. Experiments show that the evaluation accuracy of the XAI-RL model incorporating coherence metric and coherence enhancement is better than the other existing methods in multiple sets of experiments. The future work of this paper will further improve the effectiveness of self-attention weights on coherence measurement. By constructing various measurement methods, the coherence factors such as semantic connection, grammatical regularity, and coreference disambiguation will be considered from multiple perspectives to improve the sentence coherence for the next-generation models.

Data Availability

The data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Authors’ Contributions

Hikmat A. M. Abdeljaber wrote the paper, Sultan Ahmad validated the paper, Abdullah Alharbi designed the methodology and proofread the paper, Sultan Ahmad validated the software, and Sudhir Kumar proposed the method.

Acknowledgments

The authors deeply acknowledge Taif University for supporting this research through Taif University Researchers Supporting Project number (TURSP-2020/231), Taif University, Taif, Saudi Arabia.

References

J. Jeong, I.-M. Kim, and D. Hong, “Deep reinforcement learning-based task offloading decision in the time varying channel,” in Proceedings of the 2021 International Conference on Electronics, Information, and Communication (ICEIC), pp. 1–4, Jeju, Korea (South), January 2021.
View at: Publisher Site | Google Scholar
H. Tünnermann and A. Shirakawa, “XAI based reinforcement learning for coherent beam combining,” in Proceedings of the 2018 Conference on Lasers and Electro-Optics Pacific Rim (CLEO-PR), vol. 56, pp. 1-2, Hong Kong, 2018.
View at: Google Scholar
Y. George, “Dueling deep Q-network for unsupervised inter-frame eye movement correction in optical coherence tomography volumes,” in Proceedings of the 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), vol. 42, pp. 1595–1599, France, 2021.
View at: Publisher Site | Google Scholar
F. Zhou, “Multi-objective distributional XAI based reinforcement learning for large-scale order dispatching,” in Proceedings of the 2021 IEEE International Conference on Data Mining (ICDM), vol. 52, pp. 1541–1546, New Zealand, 2021.
View at: Publisher Site | Google Scholar
W. Jaśkowski, “Mastering 2048 with delayed temporal coherence learning, multistage weight promotion, redundant encoding, and carousel shaping,” IEEE Transactions on Games, vol. 10, no. 1, pp. 3–14, March 2018.
View at: Publisher Site | Google Scholar
T. Tiong and I. Saad, “Deep XAI based reinforcement learning online offloading for SWIPT multiple access edge computing network,” in Proceedings of the 2021 IEEE 11th International Conference on System Engineering and Technology (ICSET), vol. 9, pp. 240–245, Malaysia, 2021.
View at: Publisher Site | Google Scholar
B. Keller, M. Draelos, K. Zhou et al., “Optical coherence tomography-guided robotic ophthalmic microsurgery via reinforcement learning from demonstration,” IEEE Transactions on Robotics, vol. 36, no. 4, pp. 1207–1218, 2020.
View at: Publisher Site | Google Scholar
M. Bilkis, M. Rosati, and J. Calsamiglia, “Reinforcement-learning calibration of coherent-state receivers on variable-loss optical channels,” in Proceedings of the 2021 IEEE Information Theory Workshop (ITW), vol. 24, pp. 1–6, China, 2021.
View at: Publisher Site | Google Scholar
K. Chen, Z. Liang, W. Liang, H. Zhou, L. Chen, and S. Qin, “Cross-overlapping hierarchical XAI based reinforcement learning in humanoid robots,” in Proceedings of the 2021 33rd Chinese Control and Decision Conference (CCDC), vol. 85, pp. 3340–3345, China, 2021.
View at: Publisher Site | Google Scholar
R. Tommy and K. R. Athira, “Self-supervised learning of robot manipulation,” in Proceedings of the 2020 4th International Conference on Automation, Control and Robots (ICACR), vol. 6, pp. 22–25, Italy, 2020.
View at: Publisher Site | Google Scholar
Q.-D. L. Tran and A.-C. Le, “A deep XAI based reinforcement learning model using long contexts for chatbots,” in Proceedings of the 2021 International Conference on System Science and Engineering (ICSSE), vol. 12, pp. 83–87, Vietnam, 2021.
View at: Publisher Site | Google Scholar
A. Abuduweili, B. Yang, and Z. Zhang, “Control of delay lines with XAI based reinforcement learning for coherent pulse stacking,” in Proceedings of the 2020 Conference on Lasers and Electro-Optics (CLEO), vol. 42, pp. 1-2, Washington, DC, USA, 2020.
View at: Google Scholar
H. Tünnermann and A. Shirakawa, “Tiled aperture beam combining with reinforcement learning,” in Proceedings of the 2020 Conference on Lasers and Electro-Optics (CLEO), pp. 1-2, San Jose, CA, USA, May 2020.
View at: Google Scholar
N. Pfeifer, B. V. Zimpel, G. A. G. Andrade, and L. C. V. dos Santos, “A XAI based reinforcement learning approach to directed test generation for shared memory verification,” in Proceedings of the2020 Design, Automation & Test in Europe Conference & Exhibition (DATE), vol. 52, pp. 538–543, France, 2020.
View at: Publisher Site | Google Scholar
L. Huang, S. Bi, and Y.-J. A. Zhang, “Deep reinforcement learning for online computation offloading in wireless powered mobile-edge computing networks,” IEEE Transactions on Mobile Computing, vol. 19, no. 11, pp. 2581–2593, 2020.
View at: Publisher Site | Google Scholar
S. Jha, S. Ahmad, H. A. Abdeljaber, A. A. Hamad, and M. B. Alazzam, “A post COVID machine learning approach in teaching and learning methodology to alleviate drawbacks of the e-whiteboards,” Journal of Applied Science and Engineering, vol. 25, no. 2, pp. 285–294, 2021.
View at: Google Scholar
H. Tünnermann and A. Shirakawa, “AI controlled tiled aperture coherent beam combining,” in Proceedings of the 2019 Conference on Lasers and Electro-Optics Europe & European Quantum Electronics Conference (CLEO/Europe-EQEC), vol. 41, p. 1, Germany, 2019.
View at: Publisher Site | Google Scholar
T. Hamagami, T. Shibuya, and S. Shimada, “Complex-Valued Reinforcement Learning,” in Proceedings of the 2006 IEEE International Conference on Systems, Man and Cybernetics, pp. 4175–4179, Taipei, Taiwan, October 2006.
View at: Publisher Site | Google Scholar
T. Yoshida, H. Hori, and S. Nakasuka, “A XAI Based Reinforcement Learning approach to cooperative problem solving,” Proceedings International Conference on Multi Agent Systems (Cat. No.98EX160), vol. 32, pp. 479-480, 1998.
View at: Publisher Site | Google Scholar
T. Zhang, S. Shen, S. Mao, and G.-K. Chang, “Delay-aware cellular traffic scheduling with deep reinforcement learning,” GLOBECOM 2020 - 2020 IEEE Global Communications Conference, vol. 5, pp. 1–6, 2020.
View at: Publisher Site | Google Scholar
H. Qinxia, S. Nazir, M. Li, H. Ullah Khan, W. Lianlian, and S. Ahmad, “AI-enabled sensing and decision-making for IoT systems,” Complexity, vol. 2021, Article ID 6616279, 9 pages, 2021.
View at: Publisher Site | Google Scholar
G. Dhiman and R. Sharma, “SHANN: an IoT and machine-learning-assisted edge cross-layered routing protocol using spotted hyena optimizer,” Complex & Intelligent Systems, vol. 74, pp. 1–9, 2021.
View at: Google Scholar
S. Ahmad, H. A. M. Abdeljaber, J. Nazeer, M. Y. Uddin, V. Lingamuthu, and A. Kaur, “Issues of clinical identity verification for healthcare applications over mobile terminal platform,” Wireless Communications and Mobile Computing, vol. 2022, pp. 1–10, 2022.
View at: Publisher Site | Google Scholar
M. S. Mekala, G. Dhiman, G. Srivastava et al., “A DRL-based service offloading approach using DAG for edge computational orchestration,” IEEE Transactions on Computational Social Systems, vol. 1, 9 pages, 2022.
View at: Publisher Site | Google Scholar
K. Yadav, A. Jain, N. M. Osman Sid Ahmed, S. A. Saad Hamad, G. Dhiman, and S. D. Alotaibi, “Internet of thing based koch fractal curve fractal antennas for wireless applications,” IETE Journal of Research, vol. 6, pp. 1–10, 2022.
View at: Publisher Site | Google Scholar
B. Sumathy, A. Chakrabarty, S. Gupta et al., “Prediction of diabetic retinopathy using health records with machine learning classifiers and data science,” International Journal of Reliable and Quality E-Healthcare, vol. 11, no. 2, pp. 1–16, 2022.
View at: Publisher Site | Google Scholar
Y. Dong, X. Yu, A. Alharbi, and S. Ahmad, “AI-based production and application of English multimode online reading using multi-criteria decision support system,” Soft Computing, vol. 30, pp. 1–11, 2022 May.
View at: Publisher Site | Google Scholar
W. Viriyasitavat, L. D. Xu, A. Sapsomboon, G. Dhiman, and D. Hoonsopon, “Building trust of Blockchain-based Internet-of-Thing services using public key infrastructure,” Enterprise Information Systems, vol. 22, pp. 1–24, 2022.
View at: Publisher Site | Google Scholar
A. Alferaidi, K. Yadav, Y. Alharbi, W. Viriyasitavat, S. Kautish, and G. Dhiman, “Federated Learning Algorithms to Optimize the Client and Cost Selections,” Mathematical Problems in Engineering, vol. 2022, Article ID 8514562, 9 pages, 2022.
View at: Google Scholar
G. Dhiman, J. Rashid, J. Kim, S. Juneja, W. Viriyasitavat, and K. Gulati, “Privacy for healthcare data using the byzantine consensus method,” IETE Journal of Research, vol. 55, pp. 1–12, 2022.
View at: Publisher Site | Google Scholar
F. Ahmadi Zeidabadi, M. Dehghani, P. Trojovský, Š. Hubálovský, V. Leiva, and G. Dhiman, “Archery algorithm: a novel stochastic optimization algorithm for solving optimization problems,” Computers, Materials & Continua, vol. 72, no. 1, pp. 399–416, 2022.
View at: Publisher Site | Google Scholar
N. Singh, E. H. Houssein, S. B. Singh, and G. Dhiman, “HSSAHHO: a novel hybrid Salp swarm-Harris hawks optimization algorithm for complex engineering problems,” Journal of Ambient Intelligence and Humanized Computing, vol. 4, pp. 1–37, 2022.
View at: Publisher Site | Google Scholar
S. Swain, B. Bhushan, G. Dhiman, and W. Viriyasitavat, “Appositeness of optimized and reliable machine learning for healthcare: a survey,” Archives of Computational Methods in Engineering: State of the Art Reviews, vol. 11, pp. 1–23, 2022.
View at: Publisher Site | Google Scholar
R. Kumar and G. Dhiman, “A comparative study of fuzzy optimization through fuzzy number,” International Journal of Modern Research, vol. 1, no. 1, pp. 1–14, 2021.
View at: Google Scholar
P. K. Vaishnav, S. Sharma, and P. Sharma, “Analytical review analysis for screening COVID-19 disease,” International Journal of Modern Research, vol. 1, no. 1, pp. 22–29, 2021.
View at: Google Scholar
I. Chatterjee, “Artificial intelligence and patentability: review and discussions,” International Journal of Modern Research, vol. 1, no. 1, pp. 15–21, 2021.
View at: Google Scholar
V. K. Gupta, S. K. Shukla, and R. S. Rawat, “Crime tracking system and people’s safety in India using machine learning approaches,” International Journal of Modern Research, vol. 2, no. 1, pp. 1–7, 2022.
View at: Google Scholar
T. Sharma, R. Nair, and S. Gomathi, “Breast cancer image classification using transfer learning and convolutional neural network,” International Journal of Modern Research, vol. 2, no. 1, pp. 8–16, 2022.
View at: Google Scholar
S. K. Shukla, V. K. Gupta, K. Joshi, A. Gupta, and M. K. Singh, “Self-aware execution environment model (SAE2) for the performance improvement of multicore systems,” International Journal of Modern Research, vol. 2, no. 1, pp. 17–27, 2022.
View at: Google Scholar
M. Muqeem, S. Ahmad, J. Nazeer, M. F. Farooqui, and A Alam, “Selection of requirement elicitation techniques: a neural network based approach,” International Journal of Advanced Computer Science and Applications, vol. 13, no. 1, 2022.
View at: Publisher Site | Google Scholar
A. Kumar Dutta, J. Sampson, S. Ahmad et al., “Hybrid deep learning enabled air pollution monitoring in its environment,” Computers, Materials & Continua, vol. 72, no. 1, pp. 1157–1172, 2022.
View at: Publisher Site | Google Scholar
R. Gopi, M. Mathapati, B. Prasad et al., “Intelligent DoS attack detection with congestion control technique for vanets,” Computers, Materials & Continua, vol. 72, no. 1, pp. 141–156, 2022.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2022 Hikmat A. M. Abdeljaber et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Security and Communication Networks

Recent Advances in IoT and Blockchain-Based Security/Privacy in Advent Technology

[Retracted] XAI-Based Reinforcement Learning Approach for Text Summarization of Social IoT-Based Content

Abstract

1. Introduction

2. Related Work

3. XAI-RL Summary Generation Model

3.1. Overall Model Architecture

3.2. Stage 1: Summary Text Generation Stage

3.2.1. Text Encoding Representation

3.2.2. Key Statement Selection and Auxiliary Information Acquisition

4. Experimental Results and Analysis

4.1. Dataset and Experimental Setup

4.2. XAI-RL Model Summary Generation Process Discussion

4.3. Comparison between the XAI-RL Model and Existing Text Summarization Models

5. Conclusion

Data Availability

Conflicts of Interest

Authors’ Contributions

Acknowledgments

References

Copyright