A Novel Deep Convolutional Neural Network Based on ResNet-18 and Transfer Learning for Detection of Wood Knot Defects

Gao, Mingyu; Song, Peng; Wang, Fei; Liu, Junyan; Mandelis, Andreas; Qi, DaWei

doi:https://doi.org/10.1155/2021/4428964

Journal of Sensors

On this page

Abstract Introduction Methods Experimental Results and Discussion Conclusions Data Availability Conflicts of Interest Acknowledgments References Copyright Related Articles

Special Issue

Deep Learning and Artificial Intelligence for Non-Vision Sensors and Imaging

View this Special Issue

Research Article | Open Access

Volume 2021 | Article ID 4428964 | https://doi.org/10.1155/2021/4428964

A Novel Deep Convolutional Neural Network Based on ResNet-18 and Transfer Learning for Detection of Wood Knot Defects

Mingyu Gao,¹Peng Song,²Fei Wang,^3,4Junyan Liu,^3,4Andreas Mandelis,^5,6and DaWei Qi¹

Academic Editor: Ruizhen Yang

Received14 Jul 2021

Revised27 Jul 2021

Accepted02 Aug 2021

Published14 Aug 2021

Abstract

Wood defects are quickly identified from an optical image based on deep learning methodology, which effectively improves wood utilization. Traditional neural network techniques have not yet been employed for wood defect detection due to long training time, low recognition accuracy, and nonautomatical extraction of defect image features. In this work, a model (so-called ReSENet-18) for wood knot defect detection that combined deep learning and transfer learning is proposed. The “squeeze-and-excitation” (SE) module is firstly embedded into the “residual basic block” structure for a “SE-Basic-Block” module construction. This model has the advantages of the features that are extracted in the channel dimension, and it is fused in multiscale with original features. Instantaneously, the fully connected layer is replaced with a global average pooling; consequently, the model parameters could be reduced effectively. The experimental results show that the accuracy has reached 99.02%, meanwhile the training time is also reduced. It shows that the proposed deep convolutional neural network based on ReSENet-18 combined with transfer learning can improve the accuracy of defect recognition and has a potential application in the detection of wood knot defects.

1. Introduction

Wood knot defect detection is an important part in the production of wood products and finally affects the quality of wood products. Rapid detection of wood knot defects on the surface of the wood can effectively improve the qualification rate of wood products. Consequently, it is important to quickly identify the wood knot defects in a short time [1–4]. Although the traditional manual recognition is widely used and accurate, it is still a subjective [5] and inefficient method to identify wood knot defects [6]. With the rapid development of digital image processing and computer vision, artificial intelligence technology can improve the recognition speed and accuracy at a certain extent [7–9]. Among them, deep learning is the most potential method in the field of artificial intelligence.

In recent years, wood knot defect recognition based on the artificial neural network and image analysis processing has been widely studied [10–15]. Because of its simple basic structure, the neural network can fit various data in theory. Because of this, large-scale neural network combination is needed. Due to the limitation of hardware, the current tools are not enough to run this complex network, resulting in its slow evolution. At present, with the development of robots and so on, the demand for computer vision technology based on CNN (convolutional neural network) is gradually increasing. Therefore, the neural network still has a great application value in the future. In the field of wood defect detection, the accurate recognition of wood needs to collect the defect image by camera or X-ray and then recognize it by image processing and artificial intelligence. In order to accurately identify the wood knot defects, image features must be extracted first. For example, Lin et al. in 2015 proposed a method to classify wood knot defects by combining the aspect ratio, grayscale, and variance feature extraction method of the back propagation (BP) network [10]. The accuracy of this method can reach 86.67%. In the same year, Mu et al. proposed a wood defect classification method by extracting the perimeter, area, aspect ratio, and mean grayscale value of the defect, combined with the radial basis function (RBF) neural network with the accuracy over 85% [11]. In 2019, Ji et al. proposed a wood defect classification method based on Hu moment invariant feature extraction and a combination of wavelet moment with BP network [12]. The crack identification accuracy of this method can reach 98%. However, due to the shape of flying knot scar and hole being similar, it is easy to induce a misclassification in some cases. Due to the quite unique shape of each wood knot defect, it is difficult and complex to identify the defect by extracting the image features manually [13]. Therefore, a convolutional neural network (CNN) which can automatically learn the wood knot features is needed to replace the complex manual defect feature extraction. In 2019, Liu et al. proposed a CNN based on split-shuffle-residual (SSR) for real-time classification of rubber boards [14]. Comprehensive experiments show that the algorithm is superior than other classification methods and the latest deep learning classification network at that time has an accuracy of 94.86%, but there is still room for improvement. In 2021, a new method based on transfer learning and ResNet-34 convolutional neural network for recognizing wood knot defects was presented by Gao et al. [15]. The experimental results show that the classification accuracy of this method can reach 98.69%. Although both methods are practical, with the increase of network depth, the model parameters become more complex and the amount of calculation becomes larger. To solve these problems and improve the accuracy of the model, a high accuracy wood knot defect detection method based on the convolutional neural network is required.

In this paper, a model based on the attention mechanism and deep transfer residual convolutional neural network structure named ReSENet-18 is proposed to detect wood knot defects. This paper is arranged and structured as follows. Firstly, the dataset of wood knot defects is acquired and preprocessed. Then, the proposed ReSENet-18 model is introduced. A squeeze-and-excitation-basic block (SE-Basic-Block) is added, and the fully connected layer is replaced by a global average pooling layer to adjust the network structure. At the same time, combined with the ideology of transfer learning, the ReSENet-18 network is pretrained on ImageNet. Subsequently, the network is trained and tested by using the dataset of wood knot defects. Finally, based on a benchmark dataset, the test results are compared and analyzed with other deep learning models.

2. Image Processing and Methods

2.1. Dataset

In order to realize the classification and recognition of wood knot defects, firstly, the image information of 448 wood knot defects of spruce trees with seven kinds of knot defects were collected on the website of Computer Laboratory of Department of Electrical Engineering, University of Oulu [16–18] (shown in Figure 1), and made them into a dataset that can simulate the actual use scene of ReSENet-18 model. Then, the preprocessing operations such as image scaling and adding noise were carried out to realize data augmentation. Finally, the dataset was divided into three parts: a training set, a verification set, and a testing set for training, verification, and testing.

(a)

(b)

(c)

(d)

(e)

(f)

(g)

2.2. Data Preprocessing and Augmentation

The dataset of wood knot defects with 448 images was divided into a training set, a verification set, and a testing set according to the ratio of 6 : 2 : 2, which refers to 268 training images, 90 verification images, and 90 testing images, respectively (Table 1). The powerful generalization ability of the convolutional neural network is based on a large amount of data; thus, the model will induce the overfitting problem when the amount of data is not large enough which greatly limits the generalization ability [19–22]. Data augmentation technology [23, 24] was always used to expand the dataset of wood knot defects using color digital image processing technology to expand the data set and add it to the original image dataset; the problem of insufficient data can be easily solved.

The preprocessing of wood knot defect images was completed by simulating the change of angle, noise, and color of different tree species. In order to simulate these changes, the images of wood knots were mirrored horizontally, rotated by 180°, and mirrored vertically to simulate different angles of actual images. By the operation of increasing hue by 10, the color of different defects in actual image acquisition is simulated. At the same time, in order to simulate the noise that may appear in the process of image acquisition, appropriate Gaussian noise and salt-and-pepper noise are added to the image of wood knot defect to further enhance the dataset. In this work, the results of the data augmentation are shown in Figure 2. After data augmentation, the size of the dataset of wood knot defects is expanded from 448 images to 3136 images (7 times expansion). The number of training set, verification set, and testing set is 1885, 636, and 615, respectively, which can effectively reduce the overfitting phenomenon of the convolutional neural network during the training phase.

(a) Decayed knot

(b) Dry knot

(c) Edge knot

(d) Encased knot

(e) Horn knot

(f) Leaf knot

(g) Sound knot

2.3. Improved Deep Convolutional Neural Network- (DCNN-) Based Method

2.3.1. ResNet-18

ResNet-18 consists of a convolutional layer and eight residual building blocks. A residual building block is the basic structure of the ResNet-18 network. The structure of the residual building block [25, 26] is shown in Figure 3. A kind of short-cut is used to skip the convolutional layer [27]. The input vector and the vector output through the convolutional layer can be added directly [28] and then output through the rectified linear unit (ReLU) activation function. This method can powerfully alleviate the problem of a vanishing gradient or exploding gradient caused by the increase of neural network depth and can eventually improve the recognition accuracy of wood knot defects.

The output of the residual building block is written as follows: where presents the residual function and and stand for the input and output, respectively.

ResNet-18 consists of 17 convolutional layers, a max-pooling layer with the filter size of , and a fully connected layer. A classical ResNet-18 model involves 33.16 million parameters, in which ReLU activation function and batch normalization (BN) are applied to the back of entire convolutional layers in “basic block.” The structure of ResNet-18 is shown in Table 2 [27].

2.3.2. SE-Basic-Block Module

The SE-Basic-Block module has been used during the champion of ImageNet 2017 classification competition [29]. The structure is shown in Figure 4, which mainly includes squeeze and excitation [30]. The input image has the size of , where and represent the width and height, respectively, and represents the number of channels. The structure of the SE module is uncomplicated and easy to implement. It can be easily embedded into the existing network framework. The SE module mainly studies the correlation between channels, which only increases a small amount of calculation but can achieve better results.

The attention mechanism of the SE module is mainly realized by multiplying the fully connected layer and input vector for feature fusion. Assume that the size of the input image is , after passing through the global pooling layer and the fully connected layer, the input image is stretched to and then multiplied with the original image to give weights to each channel. In the denoising task, each noise point is given weight, the low weight noise points are removed automatically, and the high weight noise points are retained. During this process, the network running efficiency can be improved, the parameters and computational cost can be reduced, and the recognition accuracy is improved [31]. As shown in Figure 4, by processing the feature map of convolutional, a one-dimensional vector with the same number of channels is obtained as the evaluation score of each channel [32], and then, the score is used for the corresponding channel to get the result.

The SE module can be embedded into the residual basic block of ResNet-18. Figure 5 shows the combined structure of the SE module and residual basic block module.

2.3.3. Transfer Learning of DCNNs

Due to the small size of the data in this experiment and a certain depth of the proposed network (ReSENet-18), it is easy to induce the overfitting problem in the training process, which leads to a poor recognition ability [33, 34]. In this case, the transfer learning is used to pretrain the deep learning model and then retrain for the wood knot defect detection task using the dataset in this study, which can make our model converge rapidly; thus, a lot of training time can be saved. The deep learning model includes a hierarchical architecture with various layers to learn the complex features of images with wood knot defects [35, 36]. Finally, all these layers are connected to the final fully connected layer classifier to obtain the final results. In the transfer learning, ResNet, VGG and AlexNet models have been trained in ImageNet [37], so that the better classification performance of wood knot defects can be achieved with less training time.

2.3.4. Global Average Pooling

The fully connected layer is usually used as a classifier of CNN, but too many parameters of the fully connected layer will increase the calculation amount of the network and thus slow down the training speed and also easily appear the overfitting problem [38]. Global average pooling (GAP) is a global average of all pixels in the feature map of each channel and obtains the output of each feature map [39–41]. GAP directly removes the features of black box in the fully connected layer and gives each channel practical significance; then, the vectors composed of these output features will be sent to the classifier for classification directly [42]. Figure 6 shows the comparison between the fully connected layer and the global average pool layer.

(a)

(b)

2.3.5. Overall Architecture

ReSENet-18 is a deep neural network based on the residual structure and attention mechanism (Figure 7). The main features of the architecture are described below.

ReSENet-18 network based on deep learning consists of a ResNet-18, a residual basic block and a squeeze-and-excitation (SE) module. The ReSENet-18 model has 22 layers, including 8 parts: Conv1, SE-Basic-Block, Conv2_x, Conv3_x, Conv4_x, Conv5_x, a global average pooling layer, and a fully connected layer. The first part (Conv1) includes a convolutional layer, a batch normalization layer, a ReLU activation function, and a max-pooling layer. The convolutional layer with the kernel size of , stride of 2, padding of 3, and the max-pooling layer with the kernel size of , stride of 2, padding of 1 were employed. Adding the max-pooling layer helps to reduce the dimensions and the parameters of model, to expand the receptive fields, and to retain important feature information. The second part (SE-Basic-Block) consists of a residual basic block and a squeeze-and-excitation (SE) module. There are two convolutional layers in the second part. The SE module was embedded into the residual basic block to form a SE-Basic-Block. The structure of the SE-Basic-Block module is shown in Figure 5. In the proposed SE-Basic-Block module, two convolutional layers with the kernel size of and the stride of 1 were used. The first convolutional layer is followed by a BN layer and a ReLU activation function, while the second convolutional layer is only followed by a BN layer. As discussed above, the SE module mainly includes two parts. The first is squeeze, which makes the input image global average pooling; then, the feature map is compressed into a vector. The second is excitation, which is composed of two fully connected layers and two activation functions (ReLU and Sigmoid). The input of the first fully connected layer is , and the output is , where is a scaling parameter which is used to reduce the number of channels so as to reduce the amount of calculation. The input of the second fully connected layer is , and the output is . In this paper, is used. After getting the vector of , the initial feature map and the vector of will be scaled. The size of the original feature map is , the weight value of each channel output by the SE module is multiplied by the two-dimensional matrix of the corresponding channel of the original feature map, and the final output result is obtained. Parts three to six (Conv2_x, Conv3_x, Conv4_x, and Conv5_x) are shown in Figure 2. The seventh part (global average pool) uses AdaptiveAvgPool function, and the output size of this layer was set to . The eighth part (fully connected layer) is the classifier of ReSENet-18. Its output was set to 7, which corresponds to the types of datasets to train and classify.

ReSENet-18 takes RGB image with the random size as input, and then, the image is adjusted to in batch. The input layer of ReSENet-18 is followed by a series of convolutional blocks and a subsampling layer. The CNN structure used in this paper is a variant of ResNet-18, and the feature extraction part of this network is similar to ResNet-18. We used 17 convolutional layers of ResNet-18 to self-study the features of input RGB images from low to high. With the deepening of convolutional layers, the resolution of feature map is reduced, and more abstract high-level features are extracted. Then, inspired by the success of SENet, we use a SE-Basic-Block module to improve the receptive field. The SE-Basic-Block was embedded between Conv1 and Conv2_x. To improve the sensitivity of the model to channel features, the original features are recalibrated in the channel dimension, so that the model can automatically learn the importance features of different channels. Then, the fully connected layer is replaced by the global average pooling layer to reduce the training parameters, to accelerate the convergence speed of the model, and to improve the classification accuracy of the model.

Finally, we use transfer learning in ReSENet-18 to maximize the collected data and prevent overfitting problem. During the training phase, the weights of 17 convolutional layers except SE-Basic-Block are initialized by the pretraining model of ResNet-18. In this paper, all the parameters would not be frozen. After loading the pretraining weights, the current dataset of wood knot defects was used to retrain the whole model, which can not only improve the accuracy and speed of training but also improve the recognition ability of the model in the current dataset of wood knot defects. This is very important for effective and stable feature learning.

2.4. Training

The proposed ReSENet-18 was used and trained on one GPU (GTX 960M 2G). The experimental environment is presented in Table 3. The parameter configuration is shown in Table 4. The model using the Adam optimization algorithm and the cross-entropy loss function was trained for 200 epochs, whose batch size is 128 and learning rate is 1e-4.

The flow diagram of the detection process of wood knot defects is shown in Figure 8. First, the images of knot defects were collected from logs. The original datasets were classified by experienced professionals according to the types of defects. Then, the datasets were divided into a training dataset, a verification dataset, and a testing dataset. Subsequently, the proposed ReSENet-18 model was trained on the dataset of wood knot defects. Finally, the model is used to detect the defect types of each image in the testing dataset.

Figure 9 shows the process of training the model using the training and validation datasets. The best accuracy is 99.062%, the best loss is about 0.044, and the overall accuracy in the test phase is about 99.02%.

(a)

(b)

3. Experimental Results and Discussion

3.1. Comparisons of Model Performance

To evaluate the performance of the proposed model, the dataset was randomly divided and trained 10 times in our case. The classification accuracy of these 10 models is 99.02%, 98.20%, 98.20%, 98.20%, 98.20%, 98.36%, 97.71%, 97.87%, 99.02%, and 98.20%, respectively. The average classification accuracy of the 10 models is % and the variance is 0.40%, which indicates a good stability. Taking the first model with the accuracy of 99.02% as an example, the confusion matrix is established by analyzing the predicted labels and true labels of the testing dataset, as shown in Figure 10. All the correct predictions are on the square of the diagonal.

(a)

(b)

Figure 10 shows that the recognition accuracy of the model for dry knot, edge knot, and horn knot are 100%. In the testing set, the total number of images is 611. The classification accuracies for decayed knot defect and encased knot defect are both 95%, which is due to the small number of decayed knot images and the quite shape difference of encased knot. For leaf knot defect, the classification accuracy is 98% due to the similarity of geometric features between the horn knot and the leaf knot which is easy to be mixed. The classification accuracy of sound knot is 99% which is due to the largest number of sound knot images.

To evaluate the performance of the ReSENet-18, the precision (), recall (), f1-score (F1), and false alarm rate (FAR) were applied for the evaluation shown as follows: where , , , and represent the confusion matrix components.

Table 5 shows the precision, recall, f1-score, and false alarm rate of ReSENet-18 for the seven types of wood knot defect and the other five models for comparison. It can be seen from Table 5 that the four indicators of ReSENet-18 are the best in the recognition of five knots (decayed knot, dry knot, edge knot, encased knot, and horn knot) compared with other five classical CNN models, i.e., LeNet-5, AlexNet, VGGNet-16, GoogLeNet, and MobileNet V2. In the recognition of leaf and sound knots, some indicators of ReSENet-18 are slightly worse than other models. For example, the precision of ResNet-18 is higher than that of LeNet-5 and MobileNet V2, but slightly worse than that of AlexNet, VGGNet-16, and GoogLeNet. Among the other three indicators (, F1, and FAR), ReSENet-18 is still the best of the six models. In the recognition of sound knot, recall and false accept rate of ReSENet-18 are slightly worse than GoogLeNet and precision and f1-score are better than GoogLeNet. Compared with LeNet-5, AlexNet, VGGNet-16, and MobileNet V2, all the indicators of ReSENet-18 are better than them. Based on the above analysis, although there is still room for improvement in a few indicators, compared with the other five methods, it can be seen that ReSENet-18 still has a good performance in the identification of wood knot defects.

3.2. Comparison of the Accuracy of Different Models

The ReSENet-18 model was modified, and the test results are shown in Table 6. The accuracy of without ReLU, BN, or SE Basic-Block models is 84.78%, 96.07%, and 98.53%, respectively. The accuracy of adding a BN layer and a ReLU is 96.24%. The accuracy of adding a logsoftmax classifier and replacing Adam with NLLLoss, adding a ReLU and a fully connected layer, and adding a softmax classifier are 14.08%, 14.08%, and 14.40%, respectively. By comparing the performance of different structures of the ReSENet-18 model, the experimental results show that the ReSENet-18 model has the highest recognition accuracy.

3.3. Convergence and Prediction Accuracy Analysis of ReSENet-18 Network Model

3.3.1. Comparison with ReSENet-18 Model and Its Component Models

As demonstrated above, ReSENet-18 is composed of ResNet-18, SE module, and residual basic block. Therefore, to test the performance of the ReSENet-18 model, it was compared with ResNet-18 (composed of residual basic block) and SENet (composed of SE module). Figure 11 shows the loss curve and accuracy curve of ResNet-18, SENet, and ReSENet-18 during the training phase, which are trained by the wood knot defects dataset. One could learn from Figure 11 that our proposed method has the lowest loss value and highest accuracy compared with ResNet-18 and SENet.

(a)

(b)

The classification results of wood knot defects are shown in Table 7, in which the italic entries represent the number of knot defects correctly identified by the corresponding model, and bold entries represent the total number of the wood knot defects. From Table 7 and Figure 12, ReSENet-18 has the best recognition effect among seven kinds of wood knot defects. Compared with other networks, the ResNet-18 network is relatively shallow in depth, so some degrees of underfitting phenomenon might appear, which leads to a low accuracy on the testing set. The SENet network has the largest number of layers among the three networks, but it can be seen from Table 7 and Figure 12 that the result of identification of SENet is not the best among them due to the increase of network layers and the appearance of overfitting phenomenon. The accuracy of the proposed model with lightweight of ResNet-18 in the testing dataset reaches 99.02% (Figure 12). At the same time, it can combine the features of channel into the network, which improves the feature extraction ability.

(a)

(b)

Based on the above analysis, ReSENet-18 has been proved to have the highest accuracy and fastest convergence speed in the wood knot defect dataset than other models.

3.3.2. Comparison with Classical CNN Model

Figure 13 shows the training of ReSENet-18 and other five CNN models which was mentioned in Section 3.1. These networks are trained through the dataset of wood knot defects. It can be seen that the ReSENet-18 network has the highest accuracy and the fastest convergence speed than other models on the wood knot defect dataset.

(a)

(b)

Table 8 compares the number and accuracy of the six network models on the testing dataset. The results show that the LeNet-5 model has the minimal training parameters, which may lead to the underfitting of the network which leads to the lowest accuracy. The parameters of GoogLeNet and MobileNet V2 models are slightly more than LeNet-5, but they are more complex than LeNet-5. It can be seen from Table 8 that their accuracy is improved compared with that of LeNet-5. VGGNet-16 has the maximum parameters among the six models, and AlexNet follows. However, the accuracy is lower than that of ReSENet-18 even through the increase of parameters and longer training time. Compared with VGGNet-16, ReSENet-18 is a kind of lightweight network. At the same time, it can use SE-Basic-Block to weight and recalibrate features. It has stronger feature extraction ability and higher accuracy. It can be seen that, compared with other models, we have improved the performance of the ReSENet-18 model by adding appropriate parameters, while maintaining the robustness and efficiency of the model. Among the six models, the recognition accuracy of the ReSENet-18 model is the highest.

3.3.3. Transfer Learning

A pretraining model of ResNet-18 which includes 1.2 million color images and 1000 categories is used in this study. The weight of the pretrained model is taken as the initial weight of the dataset of wood knot defects. Figures 14 and 15 show the influence and prediction results of transfer learning on the classification accuracy and convergence speed of the ReSENet-18 model. It can be seen that the convergence speed and accuracy of the model have been improved after using transfer learning. The experimental results show that the accuracy of the model with transfer learning is 2.29% higher than that of the model without transfer learning on the testing dataset. Therefore, better convergence can be achieved using transfer learning.

(a)

(b)

(a)

(b)

3.3.4. Data Augmentation

Figure 16 shows the convergence and recognition accuracy curves of ReSENet-18 with and without data augmentation. Under the same experimental conditions, ReSENet-18 was trained on 3136 images after data augmentation, and the final classification accuracy in testing dataset reached 99.02% while the accuracy was 80.68% before data augmentation, which is shown in Figure 17.

(a)

(b)

(a)

(b)

3.3.5. Comparison of Optimization Algorithms

The optimization algorithm has an important influence on the model performance. In this study, the Adam optimization algorithm is used and compared with SGD, AdaGrad, and AdaMax, which are shown in Figure 18. The results show that the model with Adam algorithm has fastest convergence speed. Table 9 shows the prediction results of these four optimization algorithms under the same environment. The results show that the accuracy in the testing phase is 70.21% for SGD algorithm, 91.98% for AdaGrad algorithm, 95.91% for AdaMax algorithm, and 99.02% for Adam algorithm. It can be seen that the ReSENet-18 model has the best training effect using the Adam optimization algorithm.

(a)

(b)

3.3.6. Recognition Results of Different Kinds of Wood Knot Defects

“Correct recognition” was used to mark in green, and the “Wrong recognition” was marked in grey in this study. Details of the identification, such as the name and probability of wood knot defects, are displayed next to each label. Figure 19 shows seven wood knot defects and the corresponding identification results.

It can be seen that most of the wood knot defects in the image were correctly identified. Due to the shape of some wood knot defects being similar to other defects, there is no clear feature to extract under this background to induce a few regions incorrectly identified. In addition, the shape of the defect is blurred due to the low resolution of some images, which also makes the extracted features different from those in the training set. In the most cases, our method (ReSENet-18) still has a high accuracy.

4. Conclusions

In conclusion, a novel convolutional neural network model ReSENet-18 is proposed. In the feature extraction part of the network, the SE module is embedded into the residual basic blocks to form SE-Basic-Block. The classifier of the network selects the global average pool to replace the fully connected layer after the convolutional layer at the end to speed up the convergence speed and reduce the model parameters. 2521 images of wood knot defects were used for training after 200 training epochs. Experimental results show that the accuracy of ReSENet-18 in the test phase reaches 99.02%, which is 8.19% higher than the classical ResNet-18 (90.83%). In addition, when various wood knot defects are detected by this method, a large amount of image preprocessing and manual feature extraction are not required, which greatly improves the recognition efficiency. This means that ReSENet-18 has a potential application in wood nondestructive testing and knot defect identification, and it provides a feasible solution for future wood knot defect identification.

Data Availability

The datasets, codes, and weight files used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

This work was supported by the Project funded by the China Postdoctoral Science Foundation under Grant 2020M670902 and Heilongjiang Postdoctoral Fund under Grant LBH-Z20156. J. Liu acknowledges the Strategic Cooperation Program of the World Top Universities funded by Harbin Institute of Technology. A. Mandelis acknowledged the Natural Sciences and Engineering Research Council of Canada for a Discovery grant.

References

R. Norlander, J. Grahn, and A. Maki, “Wooden knot detection using ConvNet transfer learning,” in Scandinavian Conference on Image Analysis, pp. 263–274, Cham, 2015.
View at: Google Scholar
W. Pölzleitner and G. Schwingshakl, “Real-time surface grading of profiled wooden boards,” Industrial Metrology, vol. 2, no. 3-4, pp. 283–298, 1992.
View at: Publisher Site | Google Scholar
Z. Qiu, A simple machine vision system for improving the edging and trimming operations performed in hardwood sawmills, Doctoral dissertation, Virginia Tech, 1996.
D. Schmoldt, P. Li, and A. Abbott, “Machine vision using artificial neural networks with local 3D neighborhoods,” Computers and Electronics in Agriculture, vol. 16, no. 3, pp. 255–271, 1997.
View at: Publisher Site | Google Scholar
D. Yadav and A. K. Yadav, “A novel convolutional neural network based model for recognition and classification of apple leaf diseases,” Traitement du Signal, vol. 37, no. 6, pp. 1093–1101, 2020.
View at: Publisher Site | Google Scholar
X. Zhu, M. Zhu, and H. Ren, “Method of plant leaf recognition based on improved deep convolutional neural network,” Cognitive Systems Research, vol. 52, pp. 223–233, 2018.
View at: Publisher Site | Google Scholar
D. Patrício and R. Rieder, “Computer vision and artificial intelligence in precision agriculture for grain crops: a systematic review,” Computers and Electronics in Agriculture, vol. 153, pp. 69–81, 2018.
View at: Publisher Site | Google Scholar
J. Lemley, S. Bazrafkan, and P. Corcoran, “Deep learning for consumer devices and services: pushing the limits for machine learning, artificial intelligence, and computer vision,” IEEE Consumer Electronics Magazine, vol. 6, no. 2, pp. 48–56, 2017.
View at: Publisher Site | Google Scholar
T. He, Y. Liu, Y. Yu, Q. Zhao, and Z. Hu, “Application of deep convolutional neural network on feature extraction and detection of wood defects,” Measurement, vol. 152, p. 107357, 2020.
View at: Publisher Site | Google Scholar
L. Wenshu, S. Lijun, and W. Jinzhuo, “Study on wood board defect detection based on artificial neural network,” The Open Automation and Control Systems Journal, vol. 7, no. 1, 2015.
View at: Google Scholar
H. Mu, M. Zhang, D. Qi, and H. Ni, “The application of RBF neural network in the wood defect detection,” International Journal of Hybrid Information Technology, vol. 8, no. 2, pp. 41–50, 2015.
View at: Publisher Site | Google Scholar
X. Ji, H. Guo, and M. Hu, “Features extraction and classification of wood defect based on Hu invariant moment and wavelet moment and BP neural network,” in Proceedings of the 12th International Symposium on Visual Information Communication and Interaction, pp. 1–5, Shanghai, China, 2019.
View at: Google Scholar
T. He, Y. Liu, C. Xu, X. Zhou, Z. Hu, and J. Fan, “A fully convolutional neural network for wood defect location and identification,” IEEE Access, vol. 7, pp. 123453–123462, 2019.
View at: Publisher Site | Google Scholar
S. Liu, W. Jiang, L. Wu, H. Wen, M. Liu, and Y. Wang, “Real-time classification of rubber wood boards using an SSR-based CNN,” IEEE Transactions on Instrumentation and Measurement, vol. 69, no. 11, pp. 8725–8734, 2020.
View at: Publisher Site | Google Scholar
M. Gao, J. Chen, H. Mu, and D. Qi, “A transfer residual neural network based on ResNet-34 for detection of wood knot defects,” Forests, vol. 12, no. 2, p. 212, 2021.
View at: Publisher Site | Google Scholar
H. Kauppinen and O. Silven, “A color vision approach for grading lumber,” in Theory & Applications of Image Processing II-Selected Papers from the 9th Scandinavian Conference on Image Analysis, G. Borgefors, Ed., pp. 367–379, Singapore, 1995.
View at: Google Scholar
O. Silven and H. Kauppinen, “Recent developments in wood inspection,” International Journal of Pattern Recognition and Artificial Intelligence, vol. 10, no. 1, pp. 83–95, 1996.
View at: Publisher Site | Google Scholar
H. Kauppinen and O. Silven, “The effect of illumination variations on color-based wood defect classification,” in Proceedings of the 13th International Conference on Pattern Recognition (13th ICPR), pp. 828–832, Vienna, Austria, 1996.
View at: Google Scholar
L. Jiang, Y. Wang, Z. Tang, Y. Miao, and S. Chen, “Casting defect detection in X-ray images using convolutional neural networks and attention-guided data augmentation,” Measurement, vol. 170, p. 108736, 2021.
View at: Publisher Site | Google Scholar
S. Liu, H. Jiang, Z. Wu, and X. Li, “Rolling bearing fault diagnosis using variational autoencoding generative adversarial networks with deep regret analysis,” Measurement, vol. 168, p. 108371, 2021.
View at: Publisher Site | Google Scholar
W. Zhang, X. Li, X. Jia, H. Ma, Z. Luo, and X. Li, “Machinery fault diagnosis with imbalanced data using deep generative adversarial networks,” Measurement, vol. 152, p. 107377, 2020.
View at: Publisher Site | Google Scholar
H. Kim and M. Kang, “A comparison of methods to reduce overfitting in neural networks,” International journal of advanced smart convergence, vol. 9, no. 2, pp. 173–178, 2020.
View at: Google Scholar
N. Farda, J. Lai, J. Wang, P. Lee, J. Liu, and I. Hsieh, “Sanders classification of calcaneal fractures in CT images with deep learning and differential data augmentation techniques,” Injury, vol. 52, no. 3, pp. 616–624, 2021.
View at: Publisher Site | Google Scholar
S. Han, S. Oh, and J. Jeong, “Bearing fault diagnosis based on multiscale convolutional neural network using data augmentation,” Journal of Sensors, vol. 2021, 14 pages, 2021.
View at: Google Scholar
H. Ali, S. Kabir, and G. Ullah, “Indoor scene recognition using ResNet-18,” International Journal of Research Publications, vol. 69, no. 1, p. 7, 2021.
View at: Google Scholar
D. Zhu, Y. Yang, W. Zhai, F. Ren, C. Cheng, and M. Huang, “Geosot grid remote sensing intelligent interpretation model based on fine-tuning ResNet-18: a case study of construction land,” in IGARSS 2020-2020 IEEE International Geoscience and Remote Sensing Symposium, pp. 2535–2538, Waikoloa, HI, USA, 2020.
View at: Google Scholar
D. Sarwinda, R. Paradisa, A. Bustamam, and P. Anggia, “Deep learning in image classification using residual network (ResNet) variants for detection of colorectal cancer,” Procedia Computer Science, vol. 179, pp. 423–431, 2021.
View at: Publisher Site | Google Scholar
F. Zeng, W. Hu, G. He, and C. Yue, “Imbalanced Thangka image classification research based on the ResNet network,” Journal of Physics: Conference Series, vol. 1748, no. 4, pp. 042–054, 2021.
View at: Google Scholar
H. Ma, G. Han, L. Peng, L. Zhu, and J. Shu, “Rock thin sections identification based on improved squeeze-and-excitation networks model,” Computers & Geosciences, vol. 152, p. 104780, 2021.
View at: Publisher Site | Google Scholar
F. Qi, Z. Xie, Z. Tang, and H. Chen, “Related study based on Otsu watershed algorithm and new squeeze-and-excitation networks for segmentation and level classification of tea buds,” Neural Processing Letters, vol. 19, pp. 1–5, 2021.
View at: Google Scholar
Y. Ying, N. Zhang, P. Shan, L. Miao, P. Sun, and S. Peng, “PSigmoid: improving squeeze-and-excitation block with parametric sigmoid,” Applied Intelligence, vol. 9, pp. 1–3, 2021.
View at: Google Scholar
J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7132–7141, Salt Lake City, Utah, USA, 2018.
View at: Google Scholar
Y. Pathak, P. Shukla, A. Tiwari, S. Stalin, and S. Singh, Deep Transfer Learning Based Classification Model for COVID-19 Disease, Irbm, 2020.
View at: Publisher Site
V. Chouhan, S. Singh, A. Khamparia et al., “A novel transfer learning based approach for pneumonia detection in chest X-ray images,” Applied Sciences, vol. 10, no. 2, p. 559, 2020.
View at: Publisher Site | Google Scholar
M. Loey, G. Manogaran, M. Taha, and N. Khalifa, “A hybrid deep transfer learning model with machine learning methods for face mask detection in the era of the COVID-19 pandemic,” Measurement, vol. 167, p. 108288, 2021.
View at: Publisher Site | Google Scholar
Z. Wu, H. Jiang, K. Zhao, and X. Li, “An adaptive deep transfer learning method for bearing fault diagnosis,” Measurement, vol. 151, p. 107227, 2020.
View at: Publisher Site | Google Scholar
A. Abbas, M. Abdelsamea, and M. Gaber, “DeTrac: transfer learning of class decomposed medical images in convolutional neural networks,” IEEE Access, vol. 8, pp. 74901–74913, 2020.
View at: Publisher Site | Google Scholar
Q. Zhang, C. Bai, Z. Liu et al., “A GPU-based residual network for medical image classification in smart medicine,” Information Sciences, vol. 536, pp. 91–100, 2020.
View at: Publisher Site | Google Scholar
H. Lee, J. Park, and J. Hwang, “Channel attention module with multiscale grid average pooling for breast cancer segmentation in an ultrasound image,” IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, vol. 67, no. 7, pp. 1344–1353, 2020.
View at: Google Scholar
Y. Li and K. Wang, “Modified convolutional neural network with global average pooling for intelligent fault diagnosis of industrial gearbox,” Eksploatacja i Niezawodność, vol. 22, 2020.
View at: Google Scholar
W. Li, K. Liu, L. Zhang, and F. Cheng, “Object detection based on an adaptive attention mechanism,” Scientific Reports, vol. 10, no. 1, pp. 1–3, 2020.
View at: Google Scholar
X. Zhang and X. Zhang, “Global learnable pooling with enhancing distinctive feature for image classification,” IEEE Access, vol. 8, pp. 98539–98547, 2020.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2021 Mingyu Gao et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies