Construction and Optimization of Fuzzy Rule-Based Classifier with a Swarm Intelligent Algorithm

Mao, Li; Chen, Qidong; Sun, Jun

doi:https://doi.org/10.1155/2020/9319364

Mathematical Problems in Engineering

On this page

Abstract Introduction Experimental Results Conclusion Data Availability Conflicts of Interest Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2020 | Article ID 9319364 | https://doi.org/10.1155/2020/9319364

Construction and Optimization of Fuzzy Rule-Based Classifier with a Swarm Intelligent Algorithm

Li Mao,¹Qidong Chen,¹and Jun Sun¹

Academic Editor: Jose A. Lozano-Galant

Received18 Sept 2019

Revised10 Mar 2020

Accepted17 Mar 2020

Published27 Apr 2020

Abstract

In this paper, we propose a particle swarm optimization method incorporating quantum qubit operation to construct and optimize fuzzy rule-based classifiers. The proposed algorithm, denoted as QiQPSO, is inspired by the quantum computing principles. It employs quantum rotation gates to update the probability of each qubit with the corresponding quantum angle updating according to the update equation of the quantum-behaved particle swarm optimization (QPSO). After description of the principle of QiQPSO, we show how to apply QiQPSO to establish a fuzzy classifier through two procedures. The QiQPSO algorithm is first used to construct the initial fuzzy classification system based on the sample data and the grid method of partitioning the feature space, and then the fuzzy rule base of the initial fuzzy classifier is optimized further by QiQPSO in order to reduce the number of the fuzzy rules and thus improve its interpretability. In order to verify the effectiveness of the proposed method, QiQPSO is tested on various real-world classification problems. The experimental results show that the QiQPSO is able to effectively select feature variables and fuzzy rules of the fuzzy classifiers with high classification accuracies. The performance comparison with other methods also shows that the fuzzy classifier optimized by QiQPSO has higher interpretability as well as comparable or even better classification accuracies.

1. Introduction

Pattern classification, as one of the most important problems in the field of pattern recognition and machine learning, attempts to assign each input value to one of a given set of classes [1]. It involves design of a classifier and the use of the classifier to classify the given data. Generally, in practical classification problems, uncertainty or noise interference is always contained in the data, making it difficult to classify the data accurately with classical crisp classification methods [2]. In contrast, fuzzy sets and fuzzy logic provide a means for soft classification of data that allows us to analyze the data samples in a more sensitive way. For example, the use of fuzzy rules for classification is known to be a good method for classification of knowledge representation, which is similar to human knowledge expression, so that it leads to efficient, transparent, and easily interpretable fuzzy classifiers [3]. During the past few decades, fuzzy classification has been widely used in many fields, including speech recognition, character recognition, text classification, image processing, and weather forecast, to name only a few [4–9].

Fuzzy rule-based classification for a practical problem employs two kinds of methods of extracting fuzzy rules [10]. The first one is to generate fuzzy inference rules based on the professional knowledge of experts in a particular field, while the second one is to reason out corresponding fuzzy IF-THEN rules from a set of given data [9]. In this paper and other related literature, the so-called fuzzy classification employs the second type of method to generate a set of fuzzy classification rules. For the second type of methods, researchers also have proposed different approaches for the purpose of effectively extracting the fuzzy classification rules. For example, Abe and Lan [11] proposed to carry out clustering on the sample data first and then define a fuzzy rule for each cluster. Castellano and Fanelli [12] designed a grid partition method to generate the initial fuzzy classifier and then used the artificial neural network to reduce the number of the fuzzy rules. An important class of approaches in this type is to employ a training algorithm to perform the training task or to optimize fuzzy classification rules [13]. Many heuristic methods, such as genetic algorithm (GA), simulated annealing, genetic programming, and particle swarm optimization (PSO), have been used to optimize the fuzzy rules of classifiers [14–21]. These methods act as the training algorithms to construct or optimize the fuzzy rule base to maximize the classification accuracy and minimize the number of the fuzzy rules on the given training samples.

The goal of this paper is to develop a novel effective method to optimize the fuzzy classifier based on a modified version of quantum-behaved particle swarm optimization (QPSO), which incorporates quantum computing principles into the original QPSO algorithm. As a variant of PSO, the QPSO algorithm has been shown to successfully solve a wide range of optimization problems due to its stronger search ability [22–27]. However, since the QPSO was designed aiming for continuous optimization problems, it cannot be directly applied to optimization of fuzzy classification rules, which is known as a discrete optimization problem. Hence, in this paper, we incorporate a quantum bit operation into the QPSO algorithm and discretize the algorithm to be suitable for optimizing fuzzy rule-based classifiers. Quantum computing principles have been employed to design evolutionary algorithms by many researchers [28–33]. For example, Han and Kim [30] proposed the quantum-inspired evolutionary algorithm (QEA), which is characterized by the concepts of qubits (Q-bit) and superposition of states. Hammed et al. [33] designed a quantum-inspired particle swarm optimization in which the rotation angle in the rotation gate is represented by the velocity of the particle. The proposed algorithm is motivated by these works and is denoted as quantum-inspired QPSO (QiQPSO), in which each individual particle is encoded as a string of qubits and is updated by a rotation gate with the rotation angle evolving according to the iterative equation of the original QPSO.

The paper is structured as follows. In Section 2, we describe how to construct a fuzzy rule-based classification system. Section 3 presents the proposed QiQPSO, and Section 4 describes the procedure of how to construct an initial fuzzy rule-based classifier and optimize its fuzzy rule base further by using the QiQPSO algorithm. Experiment results are presented and analyzed in Section 5. Finally, the paper is concluded in Section 6.

2. Fuzzy Rule-Based Classifiers

Fuzzy rule-based classifier is an important application of fuzzy sets and is essentially a set of IF-THEN classification rules [2]. The real-world classification problems generally contain uncertainty noise interference, which can be tackled effectively by fuzzy rule-based classification methods. The fuzzy rule-based classification system has many advantages such as its consistence with representation of human knowledge, better classification performance, understandability, and strong readability and interpretability.

There are three types of fuzzy classifiers corresponding to three different types of consequents of the fuzzy rules [2]. The first type of consequents only indicates the class that the input data belongs to. That is, it has the following form:where y_(o(k)) denotes the class represented by the o(k)-th input. The second type of consequents of fuzzy rules is expressed bywhere denotes the class that the -th input data belongs to and refers to the degree of confidence with which class the input data belongs to. The third type of the fuzzy classifier has the following form of consequents:where indicates the class that the -th input data belongs to, denotes the all possible values with the scope of the output variable, that is, represents all possible classes, and refers to the degree of confidence.

In this paper, we adopt the form in (3) for the consequents of fuzzy rules. As such, the fuzzy classification system can be formulated in detail as follows. Suppose that there are n samples belonging to M classes. For the fuzzy classifier, there are m training samples with being the p-th training data, and denotes the class that belongs to. The problem of classifying the n samples into M classes can be defined as follows. For the input variable , the IF-THEN rules for the fuzzy classifier are given bywhere denotes the-th fuzzy set for the input variable (or feature variable) in the antecedent of the fuzzy rule, denotes the output variable of the consequent of the fuzzy rule (i.e., the class the sample belongs to), and represents the degree of confidence for the IF-THEN fuzzy rule, .

For the fuzzy classification system represented by (4), we define the membership function of the input variable in the antecedent of the IF-THEN rule on fuzzy set :

This membership is essentially Gaussian one including three parameters, denoting the center of the membership function and and representing the left and right widths of the membership function, respectively, which are not necessarily equal. We can select the value of the three parameters to determine the membership function for the antecedent of the fuzzy rules.

The center of the membership function is the center of the fuzzy partition. When selecting and , we introduce a constant, which is known as an overlap factor and reflects the overlap degree of the membership:in order to improve the interpretability of membership functions. Thus and can be determined by

3. The Proposed QIQPSO

In this paper, we propose a quantum-inspired quantum-behaved particle swarm optimization (QiQPSO), of which the position of particle i at the t-th iteration is represented as a string of qubits. Unlike the bits representing the information in either 0’s or 1’s in the classical computing, in quantum computing, a qubit could take the value of 0, 1, or a superposition of both [33]. Superposition allows the possible states to represent both 0 and 1 simultaneously based on its probability. In this section, we first give a brief introduction of QPSO and then propose the QiQPSO algorithm.

3.1. A Brief Introduction to QPSO

Quantum-behaved particle swarm optimization (QPSO) is a variant of PSO motivated by quantum mechanics and the trajectory analysis of the canonical PSO [22, 23, 34, 35, 36]. In the QPSO with L individuals, each individual is treated as a volume-less and spin-less particle in the N-dimensional space, with the current position vector of particle i at the t-th iteration represented by . Besides, each particle i also has a vector , called personal best (pbest) position, which is the best previous position (the position giving the best objective function value or fitness value) of particle i. The whole swarm has vector , known as the global best (gbest) position, which is the position of the best particle among all the particles in the population. Consider the following minimization problem:where is an objective function continuous almost everywhere and is the feasible space. Note that always has many local minima, which leads to its difficulty in finding a global minima when minimizing it. Then, can be updated byand accordingly can be found by , where . The particle in the QPSO updates its position in the following equation [22-27]:for and , where

In (10), is called the mean best (mbest) position defined by the average of the pbest positions of all particles, i.e., , Is a sequence of random numbers uniformly distributed on (0, 1), and parameter is known as the contraction-expansion (CE) coefficient, which can be adjusted to balance the local and global search of the algorithm during the optimization process.

3.2. Quantum-Inspired QPSO

In the quantum computing and quantum information, the quantum state is modeled by the Hilbert space of wave functions and is defined as where α and β are the complex numbers defining probabilities at which the corresponding state is likely to appear when a qubit collapses, for instance, when reading or measuring. Probability fundamentals stated thatwhere gives the probability that a qubit is in the OFF (0) state and gives the probability that a qubit is in the ON (1) state. The probability of can be represented as a quantum angle , where satisfies the probability fundamental ofwhere the parameter has been used to calculate and update the probability in quantum-inspired EA [29, 30] and quantum-inspired PSO [33].

In this paper, we proposed a quantum-inspired quantum-behaved particle swarm optimization. In this method, the position of particle i at the t-th iteration is represented as a string of qubits, which corresponds to a vector of quantum angles , where denotes the quantum angle of the j-th qubit and the vector of the quantum angles corresponding to the personal best position of particle i is .

The vector of quantum angles corresponds to the global best position of the swarm and is denoted as . Thus, according to equation (10), the quantum angle for each qubit is updated aswhere is the quantum angle corresponding to the j-th qubit of the mbest position of the swarm at the t-th iteration. However, we have to obtain the increment of the quantum angle to update the probability by using a rotation gate. Thus, we further get the increment of the quantum angle for each qubit bywith which we can update the probability corresponding to each qubit by the following qubit rotation gate:where represents the probability of the j-th qubit of particle i at the t-th iteration, with which we can collapse the corresponding qubit to be “0” or “1”.

4. Optimizing Fuzzy Classifier with QiQPSO

When using QiQPSO to optimize a fuzzy rule-based classifier, we first construct the initial fuzzy classifier with QiQPSO based on the fuzzy rule base obtained directly from the samples by using the grid partition method, and then we further optimize the fuzzy rule base to reduce the number of the rules with the classification accuracy unchanged. In the section, we describe these two procedures in detail.

4.1. Construction of the Initial Fuzzy Classification System with QiQPSO

When using QiQPSO to construct a fuzzy rule-based classifier, we should determine the antecedents of the fuzzy IF-THEN rules firstly. Then, the determined antecedent trains the known data to achieve the consequent of the fuzzy rules. The antecedent of the fuzzy rules contains information about the number of input variables, the partition of input space, and the number of input space partitions. Here, we present how to use QiQPSO to establish the initial fuzzy rule-based classifier, by adopting Gaussian membership functions and using the grid method for partitioning the input space (i.e., the feature space).

4.1.1. Determining Antecedent of Fuzzy Rules

(1)Selection of Feature Variables.

For the purpose of improving the interpretability of a fuzzy system model, the number of the selected input variables (i.e., feature variables) should be as few as possible. The method employed in this paper is to implement feature variable selection by using QiQPSO during the construction of the initial fuzzy classification system. To this end, we represent each particle in QiQPSO as a string of qubits, with the first n (n is the number of the feature variables) qubits, each reflecting whether the corresponding feature variable is selected or not. If the j-th bit is “1”, then the j-th feature variable (or input variable) is selected; otherwise, the feature variable is not selected.(2)Fuzzy Partition of the Feature Space.

We adopt the following procedure, i.e., the grid method, to partition the feature space. First, the feature space is divided evenly, with the number of partitions and the center of each partition determined. Then, QiQPSO is employed to select the midpoints to implement fuzzy partition of the feature variables. Specifically, suppose that the domain of the j-th feature variable has even partitions, , which is a qubit string composed of 0’s and 1’s. If a bit is “1,” the center of this even partition is selected as the valid center of the fuzzy partition; otherwise, the center is not selected. The number of all the valid centers in this string, namely, the sum of all 1’s in the string, is known as the number of fuzzy partitions .

Let and be the maximum and the minimum of the feature variable . The k-th center of the fuzzy partition of is then given bywhere is the index of the k-th bit whose value is “1” in the string. The center of the fuzzy partition is regarded as the center of the membership function. In order to guarantee the interpretability of the membership function, according to (6), we havewhich means the membership function overlaps at constant as shown in Figure 1.

Thus, according to (7), the left width and the right width are given bywhere is the overlap factor and it guarantees that when the k-th membership function value is , its corresponding variable value coincides with the centers of the (k-1)th and the (k+1)th membership functions. Such membership functions generally have high interpretability.

4.1.2. Encoding of the Particle Position in QPSO

The position of each particle is encoded as a qubit string with two sections, one of which represents the feature variable selection and the other of which reflects the selection of the centers of feature space partitions. As mentioned above, each qubit in a particle position string has a corresponding quantum angle which is updated and then determines the probability of the qubit value according to equations (16) and (17), respectively.

4.1.3. Encoding of the Particle Position in QiQPSO

An important objective in constructing the initial fuzzy classification system is to maximize the number of the correctly classified samples. Thus, the classification accuracy should be one of the most important components in the fitness function. Another objective is to minimize the number of the selected features as the number of the fuzzy rules increases exponentially with the feature number. Thus, the number of the selected feature variables should be another component in the fitness function. The other component in the fitness function is the number of the fuzzy rules since too many rules reduce the interpretability of the fuzzy classifier.

As such, the fitness function for the construction of the initial fuzzy classification system with the QiQPSO is given by

4.1.4. Determining the Consequents of Fuzzy Rules

As mentioned in Section 2, the consequents of the fuzzy rules include the index y of the class that the sample belongs to and the corresponding degree of confidence. When using the QiQPSO algorithm to construct the initial fuzzy classifier, we can determine the antecedents of the fuzzy rules for a given collapsed qubit string of the particle and then obtain the consequents of the fuzzy rules by the following procedure.

First, according to the antecedents of fuzzy rules and the sample data, we determine that reflects the extent to which the samples belong to class T:

Then, we can obtain the class index y in the consequent of the fuzzy rules by using the following formula:

Finally, we can compute the corresponding degree of confidence aswhere

4.1.5. Procedure of Constructing the Initial Fuzzy Classifier with QiQPSO

The procedure of constructing the initial fuzzy classifier with QiQPSO is described as follows: Step 1: set the parameters of the QiQPSO algorithm: the number of maximum iterations of the algorithm , the swarm size L, the overlapping factor, and the weights of the components in the fitness function, etc. Step 2: set the number of the iteration t = 1 and randomly initialize the particle swarm, namely, randomly initialize the quantum angle of each qubit of each particle and collapse each qubit to be “0” or “1”. Step 3: do the following procedures. Step 4: while , where i is the particle index, do the following procedures. Step 5: according to the qubit string of the k-th particle, determine the selected feature variables, the fuzzy partition of the feature space, and thus the antecedents of the fuzzy rules. Step 6: calculate the parameters of the consequents of the fuzzy rules according to equations (24) to (28). Step 7: calculate the fitness function values of the string of each particle according to (22). Step 8: update and accordingly. Step 9: calculate the increment of each quantum angle according to (16), update the probability of each qubit according to (17), and collapse each qubit to “0” or “1” accordingly. Step 10: set i = i + 1. If , return to Step 4; otherwise, continue to the next step. Step 11: set t = t + 1. If , go to Step 3; otherwise, continue to the next step. Step 12: output the collapsed qubit string of the particle with the highest fitness value and determine the fuzzy classification model based on this binary string.

4.2. Optimizing the Fuzzy Classifier with QiQPSO

The initial fuzzy classification system constructed by QiQPSO may have many redundant and invalid fuzzy rules since the feature space for the membership functions is partitioned by the grid method. Thus, we need to optimize the fuzzy classifier by removing the redundant and invalid rules without changing the interpretability of the classifier. In this paper, QiQPSO is also employed to achieve this purpose.

4.2.1. Encoding of the Particle Position

Each particle position is encoded as a qubit string with each qubit having a corresponding quantum angle. The value of each collapsed qubit represents whether a fuzzy rule should be included in the fuzzy classifier or not. If the qubit is “1,” the corresponding fuzzy rule is selected; otherwise, it is removed from the fuzzy rule base of the classifier.

4.2.2. Fitness Function

The purpose of optimizing the fuzzy rule base of the fuzzy classifier is to delete the redundant and invalid rules in order to minimize the number of the fuzzy rules without reducing the classification accuracy of the classifier. Thus, there should be two components in the fitness function, one of which is the number of the correctly classified samples and the other of which is the number of fuzzy rules. As a result, the following fitness function is employed for fuzzy rule base optimization:where and are the weights of the two components.

4.2.3. The Procedure of the Fuzzy Rule Base Optimization with QiQPSO

With the above specification, the procedure of optimizing the fuzzy rule base of the classifier is presented as follows: Step 1: set the parameters of the QiQPSO algorithm: the number of maximum iterations of the algorithm , the swarm size L, the weights of the components in the fitness function, etc. Step 2: set the number of the iteration t = 1 and randomly initialize the particle swarm, namely, randomly initialize the quantum angle of each qubit of each particle and collapse each qubit to be “0” or “1”. Step 3: do the following procedure. Step 4: for each particle i, do the following procedure. Step 5: according to the collapsed qubit string of the i-th particle, determine the fuzzy rules selected into the fuzzy classifier and find the number of selected fuzzy rules . Step 6: classify all the samples by the given fuzzy classifier and determine the number of the correctly classified samples . Step 7: calculate the fitness function values of the collapsed qubit string of each particle according to (29). Step 8: update and accordingly. Step 9: calculate the increment of each quantum angle according to (16), update the probability of each qubit according to (17), and collapse each qubit to “0” or “1” accordingly. Step 10: set i = i + 1. If , return to Step 4; otherwise, continue to the next step. Step 11: set t = t + 1. If , go to Step 3; otherwise, continue to the next step. Step 12: output the binary string of the particle with the highest fitness value and determine the fuzzy classification model based on this binary string.

5. Experimental Results

In this section, we first introduce the data sets used and parameter setting in experiments in detail in Section 5.1. In Section 5.2, experimental results for each data set are discussed separately, and we also had done experiments on those data sets with other state-of-the-art algorithms and compared them with the proposed algorithm QiQPSO.

5.1. Data Sets and Experiment Settings

The experiments were carried out on various high-dimensional classification problems to establish and optimize fuzzy classifiers for these data sets by using the proposed QiQPSO algorithm, and Table 1 summarizes the most important characteristics of these data sets. Iris data consist of 150 samples with four dimensions (namely, calyx length, calyx width, petal length, and petal width) and is divided into three classes, each of which is composed of 50 samples. The samples of the first and the second classes are separated completely, while the second and the third classes intersect. The Wine data set has 178 samples of chemical compositions of three different wines, each sample having 13 attributes. The Glass data set is composed of 9-dimensional 210 samples that can be classified into 6 categories. The Seeds data set consists of 210 data that can be divided into three categories and each sample has seven attributes. In order to verify the performance of the proposed algorithm on more complex real-world problems, Wdbc and Satimage are selected as testing data sets. These two data sets are both high-dimension real-world problems from the UCI repository [11] containing 569 and 6435 samples, respectively.

In all the experiments, the maximum number of iterations that the QiQPSO algorithm executed for was set to be 100 and the swarm size was L = 20. Note that these two parameters are set empirically according to the dimension of the optimization problems. That is, higher-dimensional problems generally needs larger maximum number of iterations and larger swarm size. The CE coefficient of the algorithm decreased linearly from 1.0 to 0.5. When constructing the initial fuzzy classifiers, the weights in the fitness function was set to be , , and . The overlap factor for the membership functions was set as . When the fuzzy rule bases were optimized with QiQPSO, the weights in the fitness function were set as and .

5.2. Experimental Results and Analysis

5.2.1. Results for Iris Data Set

For the Iris data set, we denote the four features, i.e., calyx length, calyx width, petal length, and petal width, as , , , and , respectively. The initial fuzzy classifier constructed by QiQPSO for this data is shown in Table 2. It can be seen that the initial classification system only uses two feature variables, namely, and , both of which employ four Gaussian membership functions to divide the feature space as shown in Figure 2. Thus, the initial classifier has 16 fuzzy rules in total.

From Table 2, it is shown that the initial fuzzy classifier for the Iris data set contains two invalid fuzzy rules (i.e., R13 and R14), suggesting that there is no data under these two fuzzy rules. Using this fuzzy classifier, we can find that the number of the correctly classifier samples is 143 although there are intersections between the second and the third classes.

The final fuzzy classifier further optimized by QiQPSO is presented in Table 3, in which only 9 fuzzy rules are included. More specifically, the rules in the final fuzzy classifier are

The final fuzzy classifier further optimized by QiQPSO was then tested on the Iris data set. Although the classification accuracy remains the same, the number of the fuzzy rules reduced from 16 to 9 after the fuzzy rule base of the initial classifier was optimized. This means that after the rule base was optimized by QiQPSO, the redundant and invalid fuzzy rules were removed so that the interpretability of the classifier is improved.

5.2.2. Results for Wine Data Set

For the Wine data set, the 13 attributes (i.e., compositions) are alcohol , malic acid , ash content, ash content alkalinity, magnesium, total phenol, falconoid, phenolic , anthocyanin, color intensity, chroma, OD280/OD315 dilute wine, and amino acid. Among them, alcohol , flavonoid, chroma, OD280/OD315 dilute wine, and amino acid are the input variables of the initial fuzzy classification system established by QiQPSO, and the feature space of each variable is partitioned by using Gaussian membership functions as shown in Figure 3.

For the wine data set, the initial fuzzy classifier consists of 32 fuzzy rules with five input variables. It can correctly classify 161 samples. After the fuzzy rule base was optimized by QiQPSO, the improved classifier contains only 10 fuzzy rules. More specifically, these rules are expressed as

The comparison is between the initial fuzzy classification system and the one improved by QiQPSO. It is shown that the initial classification system established by QiQPSO has 5 input variables, 10 fuzzy sets for the antecedents, and 32 fuzzy rules. With this initial classifier, the classification accuracy is 90.45% for the data set. After the fuzzy rule base was optimized by QiQPSO, the number of fuzzy rules reduced to 10 with the classification accuracy unchanged.

5.2.3. Results for Glass Data Set

Each sample in the Glass data set has nine features. However, after optimized by QiQPSO, the initial fuzzy classifier only needs five features, i.e., x₃, x₄, x₆, x₈, and x₉, and the feature space of each variable is partitioned by 2 Gaussian membership functions. Thus, the initial fuzzy classifier established by the algorithm has fuzzy rules. Due to the space limitation, these fuzzy rules are not listed here. After the fuzzy rule base was optimized by QiQPSO, the improved classifier only needs 17 fuzzy rules. Comparing to the initial fuzzy classification system, the one with the fuzzy rule base optimized by QiQPSO can classify the sample data with an accuracy of 84.11% the same as the initial fuzzy classification system, but the optimized classifier only has 17 fuzzy rules, which means that the interpretability of the classifier is enhanced by QiQPSO.

5.2.4. Results for Seeds Data Set

Each sample in the Seeds data set has 7 features. After feature selection by the QiQPSO algorithm, the dimensionality of the data is reduced to 5, only x₁, x₂, x₄, x₅, and x₇ being selected as the input variables of the initial fuzzy classification system. The feature space of each input variable is partitioned by 3 Gaussian membership functions so that the initial fuzzy classifier has fuzzy rules, which are not given here due to the space limitation. The fuzzy rule base was further optimized by QiQPSO, and the resulting fuzzy classifier only needs 12 rules.

The classifiers classify the sample data with an accuracy of 84.86%, but the one with the fuzzy rule base optimized only has 12 rules, compared to the 243 rules in the initial system.

5.2.5. Results for Wdbc Data Set

Each sample in the Wdbc data set has 64 features. However, after being optimized by QiQPSO, the initial fuzzy classifier only needs 10 features, and the feature space of each variable is partitioned by 3 Gaussian membership functions. Thus, the initial fuzzy classifier established by the algorithm has fuzzy rules. After the fuzzy rule base was optimized by QiQPSO, the improved classifier only needs 97 fuzzy rules. It is shown that both classifiers can classify the sample data with an accuracy of 79.31%, but the optimized classifier only has 200 fuzzy rules, which means that the interpretability of the classifier is enhanced after it is the QiQPSO.

5.2.6. Results for Satimage Data Set

Each sample in the Satimage data set has 36 features. However, after optimized by QiQPSO, the initial fuzzy classifier only needs 10 features, and the feature space of each variable is partitioned by 3 Gaussian membership functions. Thus, the initial fuzzy classifier established by the algorithm has fuzzy rules. After the fuzzy rule base was optimized by QiQPSO, only 153 fuzzy rules were selected for the final classification. The accuracy of these two algorithms is both 81.53%, but the optimized classifier only has 153 fuzzy rules, implying that the optimized classifier has higher interpretability than the original one.

5.2.7. Performance Comparison with Other Methods

In this section, we first compared the proposed algorithm to C4.5, and then we used other optimization methods, namely, genetic algorithm (GA) and the binary PSO (BPSO) [37], to establish the initial fuzzy classifiers and further optimize the fuzzy rule bases for the six data sets and to make a performance comparison between these methods and QiQPSO. Just as the construction of the initial fuzzy classifiers with QiQPSO, we also used GA or BPSO to build the initial fuzzy classification systems based on the system directly established from the sample data.

We all know that using particle swarm optimization to optimize a problem enhances the computational complexity. However, as shown in Table 4, although the time complexity of the proposed algorithm is not better than C4.5, the proposed algorithm has a much higher classification accuracy than C4.5. Therefore, it is still acceptable although the time complexity is higher.

The parameter settings for GA and BPSO are as follows. For GA, the crossover probability was set to be 0.5 and the mutation probability was 0.05. The population size was 50, and the maximum number of generations was also 100. For BPSO, the swarm size and the maximum number of iterations were set to be 20 and 100, respectively. The fitness functions for constructing the initial classifiers and optimizing the fuzzy rule bases were the same as those used in the experiments for QiQPSO. The experimental results are shown in Table 5, where the results of the fuzzy classifier established directly from the sample data and further optimized by GA are also listed for comparison.

For the Iris data set, the fuzzy classifier constructed directly from the sample data has 256 rules. After optimized by GA, the number of the rules decreases to 26 with a classification accuracy of 95.33%. The fuzzy classification system constructed by GA consists of 16 rules, and the final classifier improved by the algorithm only needs 12 rules that correctly classify 94.67% of the samples. The classifier constructed and further optimized by BPSO has the same number of the final fuzzy rules and the same classification accuracy as those by GA. QiQPSO generated the best classifier with the least number of rules and the highest classification accuracy (95.33%).

For the Wine data set, the fuzzy classifier constructed directly from the sample data and optimized by GA has the highest classification accuracy (95.51%) but needs 70 fuzzy rules. QiQPSO obtained the final classifier with the second highest classification accuracy,, but only 10 fuzzy rules are needed to achieve this classification accuracy. GA and BPSO also showed the same performance in constructing and optimizing the fuzzy classifier for the data set, but their performance is obviously poorer than that of QiQPSO.

For the Glass data set, the fuzzy classifier constructed directly from the samples and optimized by GA has the highest classification accuracy (87.85%), but it needs 118 fuzzy rules. Although the final fuzzy classifier based on QiQPSO obtained the second highest classification accuracy (84.11%), the number of its fuzzy rules is far smaller, meaning that the classifier has the highest interpretability. Between the GA-based and the BPSO-based classifiers, they have the same number of fuzzy rules, but the latter has higher classification accuracy.

For the Wdbc data set, the fuzzy classification system constructed directly from the samples and improved by GA employs 200 fuzzy rules to obtain a classification of 79.31%. QiQPSO obtained the final classifier with the second highest classification accuracy (77.51%), which only needs 97 fuzzy rules. It is shown that BPSO had better performance than GA in constructing and optimizing the fuzzy classifier for this data set.

For the Satimage data set, the fuzzy classification system constructed directly from the samples and improved by GA employs 150 fuzzy rules to obtain a classification of 81.53%. QiQPSO yielded the final classifier with the second highest classification accuracy (80.65%), which only needs 12 fuzzy rules. It is shown that BPSO had better performance than GA in constructing and optimizing the fuzzy classifier for this data set.

For the seeds data set, the fuzzy classification system constructed directly from the samples and improved by GA employs 34 fuzzy rules to obtain a classification of 90.95%. QiQPSO obtained the final classifier with the second highest classification accuracy (90.40%), which only needs 153 fuzzy rules. It is shown that BPSO had better performance than GA in constructing and optimizing the fuzzy classifier for this data set.

Overall, from Table 5, we can see that the fuzzy classifiers constructed and optimized by QiQPSO have satisfactory classification accuracies, though not the best ones in general, and has the least number of fuzzy rules. This implies that QiQPSO can generate more interpretable fuzzy classifiers than its competitors with comparable or even better classification accuracies.

6. Conclusion

In this paper, we first proposed the quantum-inspired QPSO (QiQPSO) and then applied the algorithm in optimizing fuzzy rule-based classifiers. Since the original QPSO was developed for continuous optimization problems, we incorporate the quantum computing principles into the QPSO to discretize the particle position and the corresponding operations. In QiQPSO, each particle position is encoded as a string of qubits, the quantum angle of each qubit is updated in line with the equation of QPSO, and a rotation gate is used to update the probability of the qubit according to which the qubit is collapsed to “0” or “1.” When QiQPSO is employed to optimize a fuzzy rule-based classifier, the algorithm is first used to construct the initial fuzzy classifier based on the sample data and the grid method of partitioning the feature space. The objectives in construction of the initial fuzzy classification system are to reduce the number of feature variables, minimize the number of fuzzy rules, and maximize the classification accuracy. After the initial classifier is established, its fuzzy rule base is further optimized by QiQPSO in order to minimize the number of fuzzy rules with the classification accuracy unchanged. The proposed method was finally tested on six well-known data sets to construct and optimize the corresponding fuzzy classifiers. The experimental results showed that the QiQPSO can effectively construct the initial fuzzy classifier and optimize its fuzzy rule base. It is also shown that the fuzzy rule-based classifiers optimized by QiQPSO have higher interpretabilities than those optimized by other methods, with comparable and even better classification accuracies.

Data Availability

The data in the manuscript are from the benchmark data set that is accessible on the website.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China (project numbers: 61673194, 61672263, and 61672265) and in part by the national first-class discipline program of Light Industry Technology and Engineering (project number: LITE2018-25).

References

L. A. Zadeh, “Fuzzy logic,” Computer, vol. 21, no. 4, pp. 83–93, 1998.
View at: Google Scholar
R. O. Duda, P. E. Hart, and D. G. Stork, “Pattern classification,” vol. 2, China Machine Press, Beijing, China, 2nd edition, 2004.
View at: Google Scholar
K. M. Alexiev and O. I. Georgieva, “Improved fuzzy clustering for identification of takagi-sugeno model,” in Proceedings of the 2nd international IEEE conference on intelligent system, vol. 1, pp. 213–218, IEEE, Varna, Bulgaria, Bulgaria, June 2004.
View at: Google Scholar
H. M. Lee, C. M. Chen, J. M. Chen et al., “An efficient fuzzy classifier with feature selection based on fuzzy entropy,” IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 31, no. 3, pp. 426–432, 2001.
View at: Google Scholar
Z. Xing, W. Hu, L. Jia, and Q. Chen, “Research on fuzzy modeling based on modified fuzzy clustering and its application,” in Proceedings of the 8th IEEE Conference on Control, Automation, Robotics and Vision, vol. 3, pp. 2293–2296, IEEE, Kunming, China, December 2004.
View at: Google Scholar
V. Snášel, P. Krömer, J. Platoš et al., “The evolution of fuzzy classifier for data mining with applications,” Simulated Evolution and Learning, Springer, Berlin, Germany, 2010.
View at: Google Scholar
N. R. Sakthivel, V. Sugumaran, and B. B. Nair, “Automatic rule learning using roughset for fuzzy classifier in fault categorization of mono-block centrifugal pump,” Applied Soft Computing, vol. 12, no. 1, pp. 196–203, 2012.
View at: Publisher Site | Google Scholar
A. Lemos, W. Caminhas, and F. Gomide, “Adaptive fault detection and diagnosis using an evolving fuzzy classifier,” Information Sciences, vol. 220, pp. 64–85, 2013.
View at: Publisher Site | Google Scholar
T. A. Runkler and R. H. Palm, “Identification of nonlinear systems using regular fuzzy c-elliptotype clustering,” in Proceedings of the Fifth IEEE International Conference on Fuzzy Systems, vol. 2, pp. 1026–1030, IEEE, New Orleans, LA, USA, September 1996.
View at: Google Scholar
J. M. Dias and A. Dourado, “A self-organizing fuzzy controller with a fixed maximum number of rules and an adaptive similarity factor,” Fuzzy Sets and Systems, vol. 103, no. 1, pp. 27–48, 1999.
View at: Publisher Site | Google Scholar
S. Abe and M. S. Ming-Shong Lan, “A method for fuzzy rules extraction directly from numerical data and its application to pattern classification,” IEEE Transactions on Fuzzy Systems, vol. 3, no. 1, pp. 18–28, 1995.
View at: Publisher Site | Google Scholar
G. Castellano and A. M. Fanelli, “Modeling fuzzy classification systems with compact rule base. Computational intelligence for modelling, control & automation: evolutionary computation & fuzzy logic for intelligent control,” Knowledge Acquisition & Information Retrieval, vol. 2, p. 210, 1999.
View at: Google Scholar
M. Valenzuela-Rendón, “The fuzzy classiﬁer system: a classiﬁer system for continuously varying variables,” in Proceedings of the Fourth Intemational Conference on Genetic Algorithms, Morgan Kauffman), San Diego, CL, USA, July 1991.
View at: Google Scholar
Y. Shi, R. Eberhart, and Y. Chen, “Implementation of evolutionary fuzzy systems,” IEEE Transactions on Fuzzy Systems, vol. 7, no. 2, pp. 109–119, 1999.
View at: Google Scholar
L. Sánchez, I. Couso, and J. A. Corrales, “Combining GP operators with SA search to evolve fuzzy rule based classifiers,” Information Sciences, vol. 136, no. 1-4, pp. 175–191, 2001.
View at: Publisher Site | Google Scholar
H. Ishibuchi and T. Yamamoto, “Effects of three-objective genetic rule selection on the generalization ability of fuzzy rule-based systems,” Evolutionary Multi-Criterion Optimization, Springer, Berlin Germany, 2003.
View at: Google Scholar
P. Angelov, X. Zhou, and F. Klawonn, “Evolving fuzzy rule-based classifiers,” in The Proceedings of 2007 Symposium on Computational Intelligence in Image and Signal Processing, pp. 220–225, IEEE, Piscataway, NJ, USA, April 2007.
View at: Google Scholar
C. F. Juang, C. M. Hsiao, and C. H. Hsu, “Hierarchical cluster-based multispecies particle-swarm optimization for fuzzy-system optimization,” IEEE Transactions on Fuzzy Systems, vol. 18, no. 1, pp. 14–26, 2010.
View at: Google Scholar
P. Kromer, J. Platos, V. Snasel et al., “Fuzzy classification by evolutionary algorithms,” in The Proceedings of 2011 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 313–318, IEEE, Anchorage, AK, USA, October 2011.
View at: Google Scholar
E. K. Aydogan, I. Karaoglan, and P. M. Pardalos, “hGA: hybrid genetic algorithm in fuzzy rule-based classification systems for high-dimensional problems,” Applied Soft Computing, vol. 12, no. 2, pp. 800–806, 2012.
View at: Publisher Site | Google Scholar
S. García-Galán, R. P. Prado, and J. E. Muñoz Expósito, “Rules discovery in fuzzy classifier systems with PSO for scheduling in grid computational infrastructures,” Applied Soft Computing, vol. 29, pp. 424–435, 2015.
View at: Publisher Site | Google Scholar
J. Sun, B. Feng, and W.-B. Xu, “Particle swarm optimization with particles having quantum behavior,” in Procedings of 2004 Congress on Evolutionary Computation, pp. 325–331, IEEE, Portland, OR, USA, June 2004.
View at: Google Scholar
J. Sun, B. Feng, and W.-B. Xu, “A global search strategy of quantum-behaved particle swarm optimization,” in Proceedings of 2004 IEEE Conference on Cybernetics and Intelligent Systems, pp. 111–116, IEEE, Singapore, December 2004.
View at: Google Scholar
J. Sun, W. B. Xu, and B. Feng, “Adaptive parameter control for quantum-behaved particle swarm optimization on individual level,” in Proceedings of 2005 IEEE International Conference on Systems, Man and Cybernetics, pp. 3049–3054, IEEE, Waikoloa, HI, USA, October 2005.
View at: Google Scholar
J. Sun, W. Fang, V. Palade, X. Wu, and W. Xu, “Quantum-behaved particle swarm optimization with Gaussian distributed local attractor point,” Applied Mathematics and Computation, vol. 218, no. 7, pp. 3763–3775, 2011.
View at: Publisher Site | Google Scholar
J. Sun, X. Wu, V. Palade, W. Fang, C.-H. Lai, and W. Xu, “Convergence analysis and improvements of quantum-behaved particle swarm optimization,” Information Sciences, vol. 193, pp. 81–103, 2012.
View at: Publisher Site | Google Scholar
J. Sun, W. Fang, X. Wu, V. Palade, and W. Xu, “Quantum-behaved particle swarm optimization: analysis of individual particle behavior and parameter selection,” Evolutionary Computation, vol. 20, no. 3, pp. 349–393, 2012b.
View at: Publisher Site | Google Scholar
A. Narayanan and M. Moore, “Quantum-inspired genetic algorithms,” in Proceedings of 1996 IEEE International Conference on Evolutionary Computation, pp. 61–66, IEEE, Nagoya, Japan, May 1996.
View at: Google Scholar
K. H. Han and J. H. Kim, “Genetic quantum algorithm and its application to combinatorial optimization problem,” in Proceedings of the 2000 Congress on Evolutionary Computation, vol. 2, pp. 1354–1360, IEEE, La Jolla, CA, USA, July 2000.
View at: Google Scholar
K. H. Han and J. H. Kim, “Quantum-inspired evolutionary algorithm for a class of combinatorial optimization,” Evolutionary Computation, IEEE Transactions, vol. 6, no. 6, pp. 580–593, 2002.
View at: Google Scholar
C. Hui, Z. Jiashu, and Z. Chao, “Chaos updating rotated gates quantum-inspired genetic algorithm,” in The Proceedings of the 2004 International Conference on Communications, Circuits and Systems, vol. 2, pp. 1108–1112, IEEE, Chengdu, China, June 2004.
View at: Google Scholar
G. Zhang, W. Jin, and L. Hu, “A novel parallel quantum genetic algorithm,” in Proceedings of the Fourth International Conference on Parallel and Distributed Computing, Applications and Technologies, pp. 693–697, IEEE, Chengdu, China, August 2003.
View at: Google Scholar
Y. Wang, X.-Y. Feng, Y.-X. Huang et al., “A novel quantum swarm evolutionary algorithm and its applications,” Neurocomputing, vol. 70, no. 4-6, pp. 633–640, 2007.
View at: Publisher Site | Google Scholar
H. N. A. Hammed, N. K. Kasabov, and S. M. Shamsuddin, “Quantum-inspired particle swarm optimization for feature selection and parameter optimization in evolving spiking neural networks for classification tasks,” Evolutionary Algorithm, pp. 133–148, 2011.
View at: Google Scholar
J. Kennedy and R. Eberhart, “Particle swarm optimization,” in Proceedings of 1995 IEEE International Conference on Neural Network, pp. 1942–1948, IEEE, Ann Arbor, MI, USA, December 1995.
View at: Google Scholar
M. Clerc and J. Kennedy, “The particle swarm - explosion, stability, and convergence in a multidimensional complex space,” IEEE Transactions on Evolutionary Computation, vol. 6, no. 1, pp. 58–73, 2002.
View at: Publisher Site | Google Scholar
J. Kennedy and R. Eberhart, “A discrete binary version of the particle swarm algorithm,” in Proceedings of 1997 IEEE International Conference on Systems, Man, and Cybernetics, vol. 5, pp. 4104–4141, IEEE, Orlando, FL, USA, October 1997.
View at: Google Scholar

Copyright

Copyright © 2020 Li Mao et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies