Spectral-Spatial Hyperspectral Image Classification Based on Homogeneous Minimum Spanning Forest

Poorahangaryan, F.; Ghassemian, H.

doi:https://doi.org/10.1155/2020/8884965

Mathematical Problems in Engineering

On this page

Abstract Introduction Discussion Conclusion Data Availability Conflicts of Interest References Copyright Related Articles

Special Issue

Advances in Numerical Optimisation: Theory, Models, and Applications

View this Special Issue

Research Article | Open Access

Volume 2020 | Article ID 8884965 | https://doi.org/10.1155/2020/8884965

Spectral-Spatial Hyperspectral Image Classification Based on Homogeneous Minimum Spanning Forest

F. Poorahangaryan¹and H. Ghassemian²

Academic Editor: Li-Tao Zhang

Received17 Sept 2020

Revised13 Oct 2020

Accepted25 Oct 2020

Published12 Nov 2020

Abstract

The combination of spectral and spatial information is known as a suitable way to improve the accuracy of hyperspectral image classification. In this paper, we propose a spectral-spatial hyperspectral image classification approach composed of the following stages. Initially, the support vector machine (SVM) is applied to obtain the initial classification map. Then, we present a new index called the homogeneity order and, using that with K-nearest neighbors, we select some pixels in feature space. The extracted pixels are considered as markers for Minimum Spanning Forest (MSF) construction. The class assignment to the markers is done using the initial classification map results. In the final stage, MSF is applied to these markers, and a spectral-spatial classification map is obtained. Experiments performed on several real hyperspectral images demonstrate that the classification accuracies obtained by the proposed scheme are improved when compared to MSF-based spectral-spatial classification approaches.

1. Introduction

Hyperspectral imagery (HIS) records reflectance values of the electromagnetic spectra in more than hundred spectral bands for each spatial position in the image. Although this valuable spectral information increases the ability to classify materials, due to the high dimensionality of the pixels, more complicated algorithms are required for hyperspectral image analysis. Several scholars investigated on hyperspectral image dimension reduction [1, 2].

Classification can be defined as assigning a unique label to each pixel in the image such that pixels with similar spectral signatures belong to the same class. In the first beginning, many pixel-wise classification techniques [3] were introduced to classify each pixel in the image independently based on only its spectral information. To further increase classification accuracies, many algorithms were designed based on a combination of spectral and spatial information [4–6]. It means that information obtained from the neighborhood of pixels and their spectrums is considered in the classification process.

Another approach for the integration of spatial information into the spectral-spatial classification process is based on image segmentation. Numerous segmentation models have been successfully performed, including watershed [7], partitional clustering [8], hierarchical segmentation [9], and multilevel segmentation [10]. These approaches divide the image into homogeneous regions based on a homogeneity measure. The results of these studies indicate the considerable improvement of classification accuracies. However, the automatic segmentation of hyperspectral images is a challenging task. Thus, the marker-controlled segmentation technique was suggested to solve the mentioned problem [9, 11]. Tarabalka et al. [11] proposed a marker-controlled segmentation for automated selection of a single hierarchical segmentation level. In that, at least one pixel for each spatial object is selected, which is called marker. Then, regions are grown from the markers. In the other work [9], markers are defined from probabilistic support vector machine (SVM) results, and then Minimum Spanning Forest (MSF) is constructed.

Although the results of the experiments of two mentioned studies indicate that the marker-controlled segmentation has good performance in hyperspectral image classification, the markers were selected based on the performance of SVM classifiers. If another pixel-wise classifier is used, it may lead to different markers. Therefore, the classification result will not be the certainty. Moreover, the different parameter settings in the same classifiers may afford the described problem [12]. Recently, the spectral-spatial classifier is proposed based on algebraic multigrid (AMG) method and hierarchical segmentation (HSEG) algorithm [12]. In this scheme, the AMG method is performed on the hyperspectral image, and a multigrid structure is generated. Then, the vertices of the obtained structure are regarded as seeds for the HSEG algorithm. In this step, the results of the segmentation and pixel-wise classification map are combined via applying the majority vote decision rule. Finally, the optimal grid level is selected, and a final classification map is obtained. Although this approach increases the classification accuracy, the selection of the optimal grid level may be the main challenge. The use of multilevel spanning forest has also been considered in recent research [13, 14]. In this approach, the combination of multiscale filters and MSF has been used. Although these studies have achieved a high degree of accuracy in classification, they used random markers to create MSFs. Some researchers also used this strategy to create MSF [15]. Although the use of random markers reduces the complexity of the method, if one of the areas of the image does not have a candidate in the markers, it will remove the area in the final classification map. To overcome this problem, scholars have used multilevel techniques. However, these approaches need a huge consuming time.

In this paper, a spectral-spatial hyperspectral image classification method based on MSF is presented, which used a new strategy for the selection of markers. The main property of our method is that the selection of the markers does not depend on the pixel-wise classification results. At first, we present a new index called the homogeneity order to extract the pixels. Then the results are considered as the input to K-nearest neighbor (KNN) that searches in feature space. The obtained pixels are regarded as markers for the MSF algorithm, and the spatial-spectral classification map is produced. The remainder of the paper is organized as follows. Section 2 describes the proposed method for hyperspectral image classification. Experimental results and the related discussions are presented in Section 3. Section 4 gives the conclusions.

2. Methodology

Figure 1 presents the general framework of the proposed method.

In Figure 1, the input of our method is a hyperspectral image with the b-spectral band and the size of , which is considered as the matrix of order . The first classification process is done on the image using SVM. Meanwhile, the local homogeneous pixels are identified. This part provides a step-by-step explanation of how local homogeneous pixels are extracted in the next step. KNN is performed in order to select the pixels that are in the local pixel neighborhood in the feature space. The results of SVM classification are used to allocate a class to each marker. Such pixels are considered as spectral-spatial markers for the MSF algorithm, and, finally, the spatial-spectral classification is constructed.

2.1. Extracting Local Homogeneous Pixels

In order to determine the local homogeneous pixels, homogeneity index of the image pixels is measured. represents the set of local spatial neighbors for central pixel with coordinate and it is obtained from the following equation:where and is an odd number that determines the width of the neighborhood window. Local neighboring pixels are presented as , and s signifies the maximum number of neighboring pixels. For example, in an 8-scale neighborhood shown in Figure 2, s = 8.

The homogeneity index of pixel in each b-spectral band, , is obtained as follows:

The homogeneity index is the average of the spectral distance between pixels and central pixel . As the distance decreases, the central pixels tends to move into a more homogeneous region. In order to determine the homogeneity order, the matrix H is constructed as follows:where N = m × n signifies the total number of image pixels and B signifies the number of the spectral bands. Matrix A is a subset of matrix H whose number of rows can be obtained as follows: where .

Then, the Pixel Homogeneity Order (PHO) is determined by the number of occurrences of each pixel in Matrix A. After this step, N_h pixels with a higher PHO are selected.

2.2. Determination of K-Neighbors in Feature Space

The image is mapped into feature space. Pixels having the minimum distance from the one extracted in the previous step are selected. This method is based on the fact that neighboring pixel in the feature space is more likely to belong to the same class. Each homogeneous pixel obtained from the previous step is considered as the central pixel, and other pixels are selected using the K-nearest neighbor in the feature space. Figure 3 is a simple description of neighbor pixels selection in the feature space. In this figure, the green-blue pixel is a homogeneous pixel with five nearest neighbors in the feature space shown as the yellow pixel.

2.3. Spectral-Spatial Marker Generation

If N_h represents the number of homogeneous pixels, the number of final spectral-spatial markers can be obtained aswhere K is the number of neighborhoods obtained by KNN method. The results of classification obtained from SVM help allocate a class to extracted markers.

If presents the class of i^th local neighbor of j marker, the class of the marker is obtained by the following equation:where .

All the obtained pixels are considered as markers in MSF algorithm. For more details of MSF construction, see [7].

3. Results of the Experiment and Discussion

Three hyperspectral benchmark images have been used: Salinas, Indian Pines, and University of Pavia (PaviaU). We choose a subset of the Salinas dataset called SalinasA, which comprises 83×86 pixels located in the of Salinas. Table 1 explains the major features of each dataset including the sensor, image size, the number of spectral bands, wavelength, spatial resolution, and the number of classes.

In order to evaluate the proposed method, three indexes, overall accuracy (OA), average accuracy (AA) and kappa coefficient, have been used. AA is the mean of the percentage of correctly classified pixels for each class. The kappa coefficient gives the percentage of correctly classified pixels corrected by the number of agreements that would be expected purely by chance. All runs are implemented in MATLAB R2014b with a laptop with 2.40 GHz central processing unit (Intel(R) Core(TM) i7-5500), 8 GB memory, and Windows 7 operating system. The number of training samples of each data is presented in Tables 2–4. In order to measure homogeneity index, an 8-scale neighborhood is used.

3.1. Analyzing the Effectiveness of Parameters on the Proposed Method Performance

Three parameters “N_h”, “TH,” and “K” have impact on the results of the method. Classification accuracy is evaluated by changing the number of homogeneous pixels. It helps to examine the impact of parameter “TH” on the method. The number of homogeneous pixels changes from 200 to 1800 with Step 200 in SalinasA and from 500 to 6000 with Step 500 in Indian Pines. It also changes from 5000 to 40000 with Step 5000 in PaviaU. Figure 4 represents the evaluation of the three data.

(a)

(b)

(c)

It has been observed that the optimal amounts of parameter “TH” in SalinasA, Indian Pines, and PaviaU are 3, 4, and 4, respectively. The optimal amount of parameter “TH” is determined in such a way that the difference of the obtained AA caused by changes in the number of the markers is smaller in the optimal value “TH” compared to other values. It is observed, after careful examination of the images, that mixed pixels are less likely to be found in images with high spatial resolution, for example, SalinasA and PaviaU. So there is not too much difference in the homogeneity order of the pixels and to achieve a more accurate homogeneity order, matrix A should be bigger compared to the situation where the image has a low spatial resolution because, in low spatial resolution images, pixel homogeneity indexes are not close in different bands. Having information about the spatial resolution of the image, we can fix the parameter “TH” in the following way:(i)TH = 3, if there is a low spatial resolution image(ii)TH = 4, if there is a high spatial resolution image

With obtained values for parameter “TH” being considered, the K-nearest neighbors around homogeneous pixels are selected. The classification accuracy of the proposed method is evaluated by changing the parameter “K” from 1 to 8, which happens when the number of homogeneous pixels (N_h) changes. Figure 5 shows this evaluation.

(a)

(b)

(c)

In order to determine the optimal values of K and N_h, the best classification accuracy value has been taken into account. It is

The results show that the optimal values of “K” for the three datasets, SalinasA, Indian Pines, and PaviaU, are 5, 2, and 2, respectively. Besides, the optimal values of N_h for the mentioned datasets are 600, 1000, and 30000, respectively. The optimal amount of the parameter N_h in the two datasets SalinasA and Indian Pines is about 8% of the total number of image pixels. However, this amount in dataset PaviaU is more than the one stated before (about 15%). Now, we will discuss the reason for the difference: PaviaU is on urban data which contains a lot of small regions. Thus, the number of markers should be high enough to maximize the likelihood of marker selection in all regions. Otherwise, out-of-marker regions are more likely to be eliminated. In the three datasets, the classification accuracy will reduce if the number of markers is fewer than the optimal value because many more numbers of image pixels are considered as markers.

We set K = 2 as the default parameter which gives the best performance in these experiments. Although the optimal value for SalinasA is achieved at 5, the difference between AA at k = 5 and k = 2 is less than 0.1%. So, we can set K = 2.

3.2. Results of Classification and Comparison with Some Existing Methods

Tables 2–4 show information about classes, the number of class samples, the number of training samples, and the results of classification for Salinas A, Indian Pines, and PaviaU, respectively. Our method is named as HKNN-MSF-MV. The results of our method are given in this section. Besides, other aspects of this method are examined and given as follows:(i)HMSF : Proposed method when the KNN step is ignored.(ii)F–HKNN-MSF-MV : In this case, WMF is used to smooth images homogeneity considering optimal value for the parameters; the method (HKNN-MSF-MV) is performed on the images extracted from the filtering stage. In this paper, the WMF parameters are set according to previous research [14].

Figure 6 indicates grand trust map, classification map obtained from SVM, and our proposed methods for SalinasA. It is clear that, by the visual comparison of the images, the proposed method improves the SVM classification. Furthermore, the filtering stage helps increase the homogeneity of the pixel in such a way that the image obtained from this method has less noise compared to other methods. In addition, in this part, the method has been compared with several hyperspectral image classification methods including (1) classification by applying grown MSF on the markers obtained from probable SVM (SVM-MSF) [9], (2) classification by constructing on MSF on the markers obtained from multiclassifiers results (MC-MSF) [9], (3) classification by constructing MSF on random markers (RD-MSF) [15], (4) classification by combination of multiscale filter and MSF (MSWMF-MSF) [14], and (5) multiscale modified MSF (MSMMSF) [13]. It is worth mentioning that training samples of all the methods should be equal in terms of number and position in order to provide an accurate evaluation. Table 2 shows the results obtained from various methods for SalinasA.

(a)

(b)

(c)

(d)

(e)

Table 2 indicates that the proposed methods can effectively enhance the OA and Kappa of SVM. However, the obtained AA of the proposed method without the filtering step is less than some of the MSF-based methods (SVM-MSF, MSWMF-MSF, and MSMMSF). This difference is related to the spectral within-class variability of the Lettuce_romaine_4wk class. In Figure 7, the spectral values of fourteen samples of some classes are presented in various spectral bands. This figure illustrates the within-class variability of Lettuce_romaine_4wk class, apparently. So, the homogeneity index for most of the pixels can be high, and the number of representatives of this class in the marker map can be reduced. When the number of class markers reduces, the class is more likely to be misclassified. This problem is solved by using F–HKNN-MSF-MV. In this method, due to the spatial filtering, the adjacent pixel consistency and therefore the homogeneity of the areas increase.

(a)

(b)

(c)

(d)

It is clear that the accuracy of the individual classification for almost all of the classes is also improved by the proposed F–HKNN-MSF-MV method. For example, the accuracy of the Lettuce_romaine_4wk class has been improved from 90.58 to 100% in comparison with the HKNN-MSF-MV.

Figures 8 and 9 indicate grand trust map, classification map obtained from SVM, and our proposed methods for Indian Pines and PaviaU, respectively. Furthermore, the numbers of training and test samples and the accuracy of individual classification for different methods for Indian Pines and PaviaU are presented in Tables 3 and 4, respectively. We can see that the proposed method improves the OA, AA, and Kappa of SVM. Moreover, the HKNN-MSF-MV method shows the best classification performance in terms of OA, AA, and Kappa in comparison with the other MSF-based methods. However, compared with MSMMSF, it is not so efficient. The reason is edge-preservation filtering used in MSMMSF. However, the filtering-based proposed method, F–HKNN-MSF-MV, leads to a dramatic increase in the classification accuracy compared to other methods.

(a)

(b)

(c)

(d)

(e)

(a)

(b)

(c)

(d)

(e)

4. Conclusion

In this paper, a new approach is presented for spectral-spatial classification. This method introduced the homogeneity order index and applied the K-nearest neighbors in feature space for robust spectral-spatial markers extraction. The obtained markers are used in the MSF algorithm and the spatial-spectral classification map is produced. In the previous MSF-based methods, the performance of the initial classification has essential effects on the marker selection and also the final classification results. This is a challenge, but the proposed method can solve this problem. The presented method was also compared to some state-of-the-art methods. These comparisons demonstrate that the classification accuracies obtained by the proposed scheme are improved when compared to MSF-based spectral-spatial classification approaches.

Data Availability

The previously reported data that were used to support this study are available at http://www.ehu.eus/ccwintco/index.php/Hyperspectral_Remote_Sensing_Scenes.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

References

P. Chen, L. Jiao, F. Liu, S. Gou, J. Zhao, and Z. Zhao, “Dimensionality reduction of hyperspectral imagery using sparse graph learning,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 10, no. 3, pp. 1165–1181, 2017.
View at: Publisher Site | Google Scholar
A. Kianisarkaleh and H. Ghassemian, “Marginal discriminant analysis using support vectors for dimensionality reduction of hyperspectral data,” Remote Sensing Letters, vol. 7, no. 12, pp. 1160–1169, 2016.
View at: Publisher Site | Google Scholar
A. Plaza, J. A. Benediktsson, J. W. Boardman et al., “Recent advances in techniques for hyperspectral image processing,” Remote Sensing of Environment, vol. 113, pp. S110–S122, 2009.
View at: Publisher Site | Google Scholar
L. He, J. Li, C. Liu, and S. Li, “Recent advances on spectral-spatial hyperspectral image classification: an overview and new guidelines,” IEEE Transactions on Geoscience and Remote Sensing, vol. 56, no. 3, pp. 1579–1597, 2018.
View at: Publisher Site | Google Scholar
M. Imani and H. Ghassemian, “Edge preserving based collaborative representation for spectral-spatial classification,” International Journal of Remote Sensing, vol. 38, no. 20, pp. 5524–5545, 2017.
View at: Publisher Site | Google Scholar
A. Kianisarkaleh and H. Ghassemian, “Spatial-spectral locality preserving projection for hyperspectral image classification with limited training samples,” International Journal of Remote Sensing, vol. 37, no. 21, pp. 5045–5059, 2016.
View at: Publisher Site | Google Scholar
Y. Tarabalka, J. Chanussot, and J. A. Benediktsson, “Segmentation and classification of hyperspectral images using watershed transformation,” Pattern Recognition, vol. 43, no. 7, pp. 2367–2379, 2010.
View at: Google Scholar
Y. Tarabalka, J. A. Benediktsson, and J. Chanussot, “Spectral–spatial classification of hyperspectral imagery based on partitional clustering techniques,” IEEE Transactions on Geoscience and Remote Sensing, vol. 47, no. 8, pp. 2973–2987, 2009.
View at: Publisher Site | Google Scholar
Y. Tarabalka, J. Chanussot, and J. A. Benediktsson, “Segmentation and classification of hyperspectral images using minimum spanning forest grown from automatically selected markers,” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 40, no. 5, pp. 1267–1279, 2010.
View at: Publisher Site | Google Scholar
P. Ghamisi, M. S. Couceiro, F. M. Martins, and J. A. Benediktsson, “Multilevel image segmentation based on fractional-order Darwinian particle swarm optimization,” IEEE Transactions on Geoscience and Remote Sensing, vol. 52, no. 5, pp. 2382–2394, 2014.
View at: Publisher Site | Google Scholar
Y. Tarabalka, J. C. Tilton, J. A. Benediktsson, and J. Chanussot, “A marker-based approach for the automated selection of a single segmentation from a hierarchical set of image segmentations,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 5, no. 1, pp. 262–272, 2012.
View at: Publisher Site | Google Scholar
H. Song and Y. Wang, “A spectral-spatial classification of hyperspectral images based on the algebraic multigrid method and hierarchical segmentation algorithm,” Remote Sensing, vol. 8, no. 4, p. 296, 2016.
View at: Publisher Site | Google Scholar
F. Poorahangaryan and H. Ghassemian, “A multiscale modified minimum spanning forest method for spatial-spectral hyperspectral images classification,” EURASIP Journal on Image and Video Processing, vol. 2017, no. 1, p. 71, 2017.
View at: Publisher Site | Google Scholar
F. Poorahangaryan and H. Ghassemian, “A hybrid multi-scale spatial filtering and minimum spanning forest for spectral-spatial hyperspectral image classification,” Journal of the Indian Society of Remote Sensing, vol. 46, no. 3, pp. 345–353, 2018.
View at: Publisher Site | Google Scholar
K. Bernard, Y. Tarabalka, J. Angulo, J. Chanussot, and J. A. Benediktsson, “Spectral-spatial classification of hyperspectral data based on a stochastic minimum spanning forest approach,” IEEE Transactions on Image Processing, vol. 21, no. 4, pp. 2008–2021, 2012.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2020 F. Poorahangaryan and H. Ghassemian. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies