Abstract
The demand for an indoor localization system is increasing, and related research is also becoming more universal. Previous works on indoor localization systems mainly focus on the acoustic signals in Line of Sight (LOS) scenario to obtain accurate localization information, but their effectiveness in Nonline of Sight (NLOS) scenario remains comparatively untouched. These works are usually less efficient as the acoustic signals often bring diffraction, refraction, scattering, energy decays, and so on in NLOS environments. So the system needs adjusting accordingly in a complex NLOS scenario based on NLOS identification results. Therefore, the identification of NLOS acoustic signal turns out to be significant in the indoor localization system. If the system only uses original support vector machine (SVM) to complete NLOS identification, the result turns out to be poor by our test. To address this challenge, we propose a novel indoor localization system, named ZKLocPro, which utilizes an advanced swarm intelligence method to optimize the traditional SVM classification model to deal with NLOS acoustic signal identification. Its results can help the system adjust the localization process if necessary in a complex NLOS scenario. Obviously, it is also significant to build our own NLOS data set, which is suitable for an indoor localization system’s situation. Specifically, four methods are added: (1) new LOS and NLOS acoustic localization signal sample production, rearrangement, and reselecting process; (2) advanced parameter optimization process; (3) elitist strategy; and (4) inertia weight nonlinear decrement. The experimental result shows that our system is efficient and performs better than state-of-the-art congeneric works even in a complex NLOS scenario.
1. Introduction
Nowadays, the public’s demand for location-based services (LBS) in a complex NLOS scenario has become more robust, and LBS has affected many aspects of our works or daily lives in many situations. Although the Global Navigation Satellite System (GNSS) can cope with people’s demand for positioning outdoors, it really has limitations. In an indoor site, due to the complex structure of the building envelop, the satellite signal is blocked by unavoidable occlusion, and it makes the effective area unable to cover the indoor areas. So the research on an indoor localization system is also needed. For some indoor situations, the indoor localization system will provide better services for the majority of users with fabulous scheduling and optimization processes. It can help analyse the traffic of congested sections, and it is also an important factor to count the real-time number of people in various regions to provide navigation services or tips. It is also the foundation on subsequent development of trajectory tracking or the user behaviour prediction system. The result is supposed to improve the decision-making efficiency and consumption conversion rate for the managers, so as to obtain more benefits.
The existence of an indoor localization system improves people’s quality of life and brings convenience to users. These systems can be roughly divided into several types based on core technology used for localization, such as Global System for Mobile Communications (GSM) [1], Wi-Fi [2], Bluetooth [3], acoustic signals [4–6], geomagnetic field [7], and inertial navigation [8]. Besides, indoor localization technology based on acoustic signal is becoming popular because of its excellent compatibility, stability, and robustness, so it has gradually become a cutting-edge research topic and is being more widely used in a variety of indoor environments.
For the acoustic localization signal, if there is an obstacle with a length or width far greater than its wavelength from the transmitting end to the receiving end, the acoustic signal may be out of time synchronization due to incalculable time delay among acoustic anchors or erroneous calculations. Meanwhile, the complex NLOS scenarios will directly contribute to diffraction, refraction, scattering, reflection, and other irregular loss of acoustic signals [9–11], which can significantly degrade the positioning accuracy of indoor localization systems. Although our test scenario in this paper is not exactly the same as the deploying scenario, we have made the conditions of the experimental sites match the conditions of the deployment site as much as possible and still simulate all kinds of NLOS scenarios for our classification part in the system. Generally, as for the indoor localization system deployed in the classrooms, museums, or exhibition halls, the LOS and NLOS scenario descriptions are shown in Figure 1.
As for the indoor localization system based on acoustic signals in the NLOS scenario, in order to reduce its negative impact, the main process will be adjusted to improve the final positioning accuracy according to the classification result. Combined with [4, 7, 11–13], we use six smartphones to finish our experiments: one is for the main anchor, four is used to construct acoustic anchors, the last one is used for calculation reference, and one PC is regarded as the back-stage management part to set up our own system. We utilize the algorithm in [4, 11, 12] to complete the feature extraction of acoustic signals for our system due to its stability and expansibility. Moreover, NLOS identification processes at each anchor are added in all localization periods. MySQL is used to store data and information temporarily, during which the classification results interacted with other parts based on the Wi-Fi. At this time, if there are not any NLOS acoustic signals in this localization loop based on identification result, the process can be completed directly based on advanced Time Difference of Arrival (TDOA) without a time synchronization [13, 14] process and figure out the final localization results. As for each transmission path between the acoustic anchor and the reference anchor, if there is only one NLOS acoustic localization signal in this loop judging by classification result, the server will delete the only one NLOS acoustic signal and complete the localization process based on the advanced TDOA algorithm; otherwise, the process will not be carried out. From previous experimental results in [13, 14], we can see that the actual positioning accuracy of indoor localization systems based on advanced TDOA using only three LOS acoustic signals is even higher than that of three LOS acoustic signals plus one acoustic NLOS acoustic signal due to its less NLOS negative effect. In order to use the classification results reasonably, relevant steps also need to be designed and implemented. The overall process of our indoor localization system is shown in Figure 2. Therefore, the results obtained from the acoustic signal LOS and NLOS classification section will determine when to adjust the entire localization process. There is no doubt that the classification part of NLOS acoustic signal is the key, and it directly affects the final result of the indoor localization system in the NLOS scenario [4, 11–13].
The construction of NLOS scenarios and acoustic signal sample set are important for the classification part. However, in the previous development process of the same type of indoor localization system, the construction of the NLOS scenario is relatively simple: it often involves only one kind of NLOS occlusion mode, and there is quite a certain gap with the deploying scenario. The production process of acoustic localization signals in LOS and NLOS scenarios should be improved and build our new training set. For the occlusion mode of acoustic signal, several new methods and processes are used to reconstruct our own training data set for the SVM classifier. More details will be revealed in Section 5. Among them, the NLOS training sample set is divided into three different batches, which serves as the foundation on the following process. According to different occlusion modes in NLOS scenarios, they are divided into three batches. The advanced swarm intelligence optimization strategy is used to facilitate the SVM classification process in order to implement the acoustic signal NLOS identification process in the system easily and quickly. With the upgrading part of the system mentioned above, we will be able to deal with sudden changes and complex conditions in the NLOS scenario better. It means that the system will be able to cope with more than two kinds of NLOS scenarios. We implement the feature extraction part on an Android smartphone for our indoor localization system and the classification part on a PC platform. During all localization periods, all parts can interconnect with each other through Wi-Fi. This will facilitate the system localization process in the NLOS scenario and improve its positioning accuracy. In total, the main flow is shown in Figure 2.
2. Related Work
For the indoor localization system, according to the core localization algorithm adopted by the system, they can be divided into three main batches: the system based on Time of Arrival (TOA), Direction of Arrival (DOA), and TDOA. These three kinds of indoor localization systems are easily subject to NLOS environments. Previously, the BeepBeep system [15], OneBeep system [16], Guoguo system [17], ASSIST system [18], and other typical systems will sharply reduce the positioning accuracy because of the error of multipath transmission, incalculable energy loss, and so on. These factors are most likely caused by the NLOS scenario. From the above-mentioned documents and systems, we can see some characteristics of indoor localization systems based on acoustic signal. Firstly, the acoustic signal actually received by each receiving terminal is the combination of scattered signal and reflected signal. Then, the acoustic signal actually collected by the receiving terminal is the superposition of acoustic signals by multiple transmission paths. Although its frequency component is similar to the source, its phase components and energy both have changed greatly. Finally, when the acoustic signal is transmitted through multiple paths, the energy will continue to decrease with the increase in transmission distance. This irregular reduction will be difficult to calculate and analyse. It is easy to see that the calculation of arrival delay difference caused by the different reflection paths will become the key to the localization algorithm. No matter for distance-based localization algorithms such as TDOA and TOA or angle calculation-based algorithm like DOA, time delay calculation will determine the effect of positioning accuracy. Therefore, considering the system complexity, development cost, and positioning accuracy requirements, the core localization algorithm of the system in this system is TDOA, which only needs the clock synchronization process among the acoustic anchors, not between the acoustic anchor and the coordinates to be measured, that is, the main anchor in this system. In Section 3.1, we will explain the way to abandon the time synchronization process in the system. Then, TDOA evolves into advanced TDOA. However, the system will encounter NLOS paths in the daily use. The random appearance of NLOS paths will bring enormous errors to the localization process, which has become a common problem of all the indoor localization systems.
To address these challenges, our system chooses the new identification method to deal with the NLOS scenario. Specifically, the more accurate classification part will make the localization system’s chosen process more precise when dealing with NLOS scenarios. Our system adjusts the traditional localization process according to the NLOS classification results in time and then eliminates the negative impact on the NLOS scenario as possible. Previously, Li et al. and Zhang et al. designed and developed the SACLoc system and Linloc system and their related products based on acoustic signals, respectively, in [4, 5, 12]. These systems are using Linear Regression (LR), Linear Discriminant Analysis (LDA), SVM, and other models for NLOS identification in order to facilitate the main process according to identification results. However, judging by the measurement in the complex NLOS scenario, the acoustic signal NLOS identification performance of the Linloc system is still insufficient. It is also difficult to extend to other systems due to the difficulty of cross-platform implementation and clock synchronization process limitation. This has become a bottleneck for the indoor localization system.
The applications (APP) involved in this system are developed on Android Studio. Although the applications can also be completed in the iOS or MAC system in theory, the systems mentioned above have weak support for the development of high-frequency audio files, and relevant development toolkits or functional libraries are rare. At the same time, combining with the demand of the deploying sites on our indoor localization systems appears to be necessary. We also make our decisions on the climate of Guilin City and the conventional temperature of this place. We believe that there will be some humid and cold days in this place every year. At this stage, those terminals based on the iOS and MAC system may automatically shut down randomly. This kind of automatic shutdown is unpredictable and uncontrollable. Meanwhile, considering the development cost, the ideas of transplantation and development on these platforms mentioned above are abandoned.
Obviously, the previous methods of the NLOS shielding way in other indoor localization system is slightly simple, and targeted upgrades have been made for specific scenarios in this paper. As for the NLOS scenarios involved in this indoor localization system, the NLOS occlusion methods encountered are rigid body shielding and human body shielding. At the same time, their appearances of the two kinds of NLOS shielding ways are random in the real-world situation. For the general deployment of this system, this paper makes the same number of two kinds of NLOS acoustic signal samples, corresponding to two different NLOS shielding methods; that is, the default NLOS occurrence probability of both two kinds of NLOS situations is 50%. According to the actual demands, the weights of the two kinds of NLOS samples can be adjusted.
3. System Overview
As a classical machine learning model [19], SVM has unique advantages when the computing platform has limited performance with small feature dimensions and high correlation among their various dimensions. For the acoustic signal classification part of the system, SVM will be the core, which needs to cope with the results obtained by the feature extraction algorithms and affect the whole process in the NLOS scenario in time. In this paper, we use an advanced swarm intelligence optimization strategy in the system to optimize the SVM classification process, which will directly affect the final positioning accuracy.
3.1. Overall System Framework
Referring to our previous work in [13, 14], an indoor localization system called the ZKLocPro system is designed and constructed to cope with NLOS localization in the real-world situation without a clock synchronization process. An acoustic signal acquisition section is added on the main anchor for carrying out appropriate solutions according to the collected acoustic signal. Then, it is necessary to calculate the acoustic signal sending time interval according to the reference anchor planted in the system in order to avoid the strict demand of all the acoustic anchors sending acoustic signals on time precisely. At the same time, the overall wav file in one localization loop should be cut as planned, whose length can be decided on the acoustic signal time delay gain or acoustic signal sending time of each anchor before the identification process. After the Android platform completes the 9-dimension feature extraction, the classification process is completed on PC and the results are sent back to the Android platform, which implements NLOS identification of four small segments of acoustic signals obtained in one localization loop and then synergizes the final positioning accuracy after extra processing if necessary.
Referring to other localization systems based on wireless sensor networks or other mobile devices [4, 5, 10, 12–14, 20, 21], the whole localization system designed and constructed in this paper can be divided into five modules: anchor module, APP module, Local Area Network (LAN) module, background module, and classification module. The main anchor and APP module share one smartphone. The speakers used by all anchors are just the horn or the speakers equipped on the smartphones during our experiments. The overall framework of the system is shown in Figure 3.
On Android APP, the user commands the beginning and the end of the localization period according to his or her wishes. The feature extraction part refers to the process in [4, 14], which is implemented on the smartphone, and the classification is completed on the PC. The LAN module needs to provide a stable Wi-Fi environment to complete interaction support and data transmission among modules. The anchor module contains multiple acoustic anchors, which consists of one main anchor and other acoustic anchors (A, B, C, and D) mentioned above according to their responsibilities. Among them, the main anchor contains the speaker part, the microphone part, and the communication part, which has numerous functions, and the other anchors (A, B, C, and D) only include the speaker part and communication part. As shown in Figure 4, after receiving the localization command, the main anchor first starts to collect the acoustic signal. It broadcasts a predetermined acoustic instruction and completes the signal cutting in one single localization loop according to the time delay gain of acoustic signal. Then, it broadcasts the new acoustic signal after processing. After the overall acquisition is completed, the anchor uploads the collected acoustic signal to the background module. At this time, some important mathematical calculations need to be completed. First of all, the acoustic signal propagation velocity should be calculated judging by the temperature measured in the sites, and acoustic signal sending time difference and propagation time difference should be obtained for each acoustic anchor. Then, the server performs Generalized Cross Correlation (GCC) operation between the acoustic signals collected and template signal to obtain the arrival time difference of acoustic signal. The acoustic signal’s sending interval is fixed, so the server can calculate the acoustic signal sending time difference and then calculate the propagation time difference by solving the equation . Based on the main anchor’s speaker, the distance between the smartphone and the other speaker can also be obtained according to , where . Finishing all the mathematical calculations mentioned above, the current location of the smartphone could be obtained based on a triangulation method. The classification module performs the classification and transmits the results back to the background module, which is one of the most significant parts of this system. In total, the localization process will be completed in 3-5 s if the localization conditions are suitable for the advanced TDOA. Its processing diagram is shown in Figure 4.
The background module includes a database and server, in which the database is not only responsible for storing information and related data during localization but also responsible for cooperating with others to perform necessary operations in time. At the end of all localization periods, the system will automatically clear the files cached locally. MySQL is selected here used for the system database. This module has the following key functions: (1) data pretreatments; (2) time delay estimation; (3) the NLOS identification results obtained by advanced SVM should be transmitted; (4) determine the system’s actual indoor localization process; and (5) calculate the user’s (main anchor’s) coordinate and calibrate and show the localization results on the APP in time. During this loop, the results are transmitted among various parts through the LAN module.
The coordination among the modules is vital. In the ZKLocPro system, the main anchor can receive the instruction sent by the background module after the button is clicked on APP. After receiving the command from the user, the main anchor starts to collect the acoustic signals received within the next 2 s. The obtained signals are sent to the background module once the collection process is completed. After the background module receives the acoustic signal successively sent by each anchor (A, B, C, and D) or the APP module, our system filters the ambient noise of the acoustic signal first. Then, the acoustic signal arrival time is used to calculate the acoustic signal sending time difference between each anchor (A, B, C, and D) and the arrival time difference of the signals, so as to obtain the time difference between the acoustic signals transmitted from each anchor (A, B, C, and D) and the user’s smartphone, that is, the time difference between each anchor and the main anchor.
To sum up, some important illustrations need to be emphasized. If the system is allowed to finish NLOS identification, after the NLOS identification process is completed by the classification module, the system will eliminate no more than one NLOS acoustic signal after cutting, rather the whole wav files. Next, it is expected to find out the global optimal solution according to the Levenberg-Marquardt (LM) algorithm in [22]. Even with the boosting function of piezoelectric tweeter, the sound generated in the indoor localization system will be quite soft, and the loudness is less than 20 dB. The buzz of it sounds like the tapping of your fingernails on your smartphone shell slightly. In total, calculating the user’s current coordinates and sending them to the APP module on time for visual display turn out to be realizable based on advanced TDOA and NLOS identification after processing.
3.2. Improvement of Acoustic Localization Signal
Motivated by [23, 24], our system uses the Linear Frequency Modulated (LFM) signal, which can help distinguish multipath components by a cross-correlation algorithm after a time delay estimation process. The expression of LFM signal used in the system with finite length is as follows:
In equation (1), , means the time length, means the initial phase, is the carrier frequency, means the magnitude of amplitude value, and means the chirp rate.
If the frequency band of acoustic signal is less than 8 kHz, it will have good linear characteristic response, but the frequency band mentioned above is not the proper range used by the indoor localization system. Since the highest acoustic sampling frequency of our smartphone supports 44.1 kHz, the frequency of acoustic localization signal should be less than half of the sampling frequency. Based on the real-world situations and combined with applying scenarios, the ZKLocPro system adopts high-frequency signals above 18 kHz that are insensitive to human ear to reduce noise pollution. In some places where allowable signal frequency range is large enough, such as warehouse or indoor car parking area underground, low-frequency signals can be considered during that stage. The step response of the signal will not only bring frequency leakage but also affect the acoustic signal anchor broadcasting [4, 13], which are not conducive to the NLOS identification process and localization calculation.
To address this challenge, we design and then add a composite window function to complete amplitude modulation and filtering process compared to [4]. The system adopts a new window function consisting of one Blackman window and one rectangular window to modulate the amplitude of the signal more effectively. Meanwhile, the time-frequency characteristics of the localization signal will also be guaranteed [25]. According to their frames, the Blackman window is divided into two parts equally, which are placed on the left and right side of the composite window function. And in the middle part, one rectangular window is set. LFM signal with a frequency band of 17.5 kHz-20.5 kHz is selected as the localization signal, and the sampling rate remains at 44.1 kHz during the periods. Figure 5(a) on the left is the time domain diagram and time-frequency diagram of the modulated acoustic signal after upgrading, and Figure 5(b) on the right is the composed window function used in the system.
(a)
(b)
4. SVM Optimization Based on Swarm Intelligence
4.1. SVM Implementation
The SVM model is very suitable for our indoor localization system [4, 26, 27], as it has strong generalization ability and involves fewer parameters. At this time, it is imperative to use reasonable methods to build our own data set based on the MATLAB toolbox developed by scholar Chih-Jen Lin [28], and relevant codes are completed.
4.2. Elitist Strategy
According to the demands on the system based on SVM, it is necessary to make preliminary preparations from two aspects. On the one hand, although there is no clearly universal fitness function used for acoustic localization signal till now, it is essential to find out the fitness function that can highlight the difference between LOS and NLOS acoustic signals. The value of the fitness function will affect the dynamic adjustment with LOS and NLOS acoustic signal training weight in our system. On the other hand, neither the swarm intelligence optimization strategy nor SVM is an algorithm that must be iterated many times. The fitness function is chosen based on our test in the real-world situation, that is, the inverse of the absolute value of the difference between the predicted value and the actual value. If the fitness function of -th sample is , the actual value of the -th sample is and the predicted value of the -th sample is; it can be recognized as the total mean value. So the corresponding formula is as follows:
Based on equation (2), all samples are arranged in ascending order according to the value of fitness function. There are three batches based on different NLOS situations. For each batch, most samples are taken out on time, and the parts taken out are divided equally into three sections. Then, the reselection and rearrangement of samples should be completed. According to the highly targeted grouping results, the LOS and NLOS acoustic signal classification process is upgraded based on the swarm intelligence optimization strategy, focusing on the processes of sample rearrangement, grouping training, and group selecting. Soon multiple interventions are carried out, respectively, so as to complete the elitist strategy independently in all three batches. The samples’ training weight of the three batches can be adjusted according to the scenarios before the training. In each section, the samples should be selected based on the chosen probability and processes in Figure 6 accordingly. In three batches, there are 4200 acoustic signals used for the training set in total. More details will be given in Section 5.1.
We implement this part on PC, which not only strengthens the classification ability of the classifier for NLOS acoustic signals but also creates a prerequisite for adding the proper decreasing mechanism of inertia weight nonlinear decrement to upgrade the iterative process of novel swarm intelligence algorithm in time. The elitist strategy here is used to ensure that the sample with higher fitness function value has a higher probability of being focused and used. At the same time, the probability of selecting sample with lower fitness function value is not 0, and the redundant samples are returning to the tail section conditionally. The samples that have not been selected during the period directly skip the rearrangement process. In principle, more high-quality samples should be confirmed and selected in time based on their probabilities for each section. Then, less inferior samples can also be selected unfrequently, and it cannot be left unchecked. To sum up, the core process of sample reselection and rearrangement is shown in Figure 6.
The elitist strategy will make the role of higher-quality samples more prominent in the classification part and alleviate this section under some circumstances. However, the convergence speed of the algorithm may be slow without any updating strategy. Therefore, it is necessary to add the inertia weight nonlinear decrement [29].
4.3. Inertia Weight Nonlinear Decrement
In the process of swarm intelligence optimization, with the progress of training and selecting, the fitness function mean value of the population will be flatten gradually in each section. Although the distribution of high-quality samples will become more concentrated in each batch, it is also essential to ensure that few low-quality samples can enter the process of high fitness value grouping under a certain probability. However, with the progress of training and selecting, the algorithm also needs to become convergence and all grouping states will tend to be stable. For the samples with lower fitness function value, the probability of existing grouping state and training weight adjusting processes needs to be gradually reduced, but it cannot change to 0 instantly. Although the iterative process, selection scale, and core process are slightly different in some swarm intelligence algorithms, their main structure is similar. Taking the novel particle swarm optimization algorithm as an example, the core mechanism in the training process is to make the samples with higher fitness survive easier and weaken the training weight of the samples with lower fitness. Although the training weight of the samples with low fitness function is weakened, their influences cannot be completely ignored. If the learning speed corresponds to the -th particle in the iteration, its corresponding weight is , and , respectively, represent the initialization setting value and final value that the weight can be obtained in the iteration process, and is the maximum number of iterations, which can be set as a constant number. At this time, the relationship of weight is as follows:
Equation (3) can be regarded as a summary on the general law of nonlinear decreasing weight adjusting process in advanced swarm intelligence optimization, which can ensure reasonable dynamic optimization and adaptive adjustment. The new dynamic weight obtained above can be brought into the equations with dynamic particle velocity in the system, so as to achieve more significant training. Adjusting the pertinence of selection and the convergence speed of the algorithm as planning turns out to be necessary and ensures its convergence smoothly on time. Compared with the original SVM classifier, the two strategies mentioned above greatly enhance the ability to jump out of the local minimum region in the learning process. The state of the particle at this moment determines whether the following direction conforms to the training trend and then adjusts the particle speed in the next iteration process if necessary. The equations about and its next position in the next unit time can be written as follows:
In equation (4), controls the “self-cognition” part of the learning strategy on the population. Compared with the original algorithm, it is vital to emphasize the “innovation” ability of this particle in a complex NLOS scenario. Its value will reflect the strength of the sample’s ability to adapt to the environmental selection’s direction, and not only the positive value but also the negative value will reflect whether the sample adapts to the system’s learning direction at present. puts more emphasis on the “social ability” of particles in the system. This factor will highlight their acclimation ability and pay more attention to the interaction of particles in the population. The two factors correspond to the adaptability of different data sets used in the system. The sample is regarded as a particle in the -th iteration, the velocity of the -th particle is , and , respectively, represent the best value of the two parameters at this time in SVM, and represents the position at present. Based on the previous description, the speed can be written as follows:
Judging by our test in the real world, it shows that the value of is close to 1, which can be set as 0.95 during initialization, and the initial value of can be set as 0.7. It can be convinced that the classifier has a high probability to break the inherent selection trend and the established grouping states in the early stage of training. Before half of the training, the whole part is not necessarily carried out in the positive direction of the training. With the continuous training, the probability of overturning the existing training trend is gradually reduced and making the sample rearrangement tends to be gentle, but the probability of breaking through the inherent selection strategy cannot be 0 sharply.
The inertia weight nonlinear decrement is used to synergize the learning ability and generalization ability of the classifier for NLOS acoustic signals under high probability. On this basis, the machine learning part of the system can be optimized, so as to adjust the localization process, intensify the LOS and NLOS acoustic signal identification ability in the system, and then improve the positioning accuracy after adjusting.
5. Experiment and Analysis
Motivated by these previous researches, we carry out the following work: first, according to the experiment before in the real world, if the classification part was completed only based on the original SVM and the feature extraction algorithm mentioned in [4, 11, 12, 14], the actual classification accuracy would be less than 80% in the complex NLOS scenario. Compared with NLOS acoustic signal, the eigenvalues of LOS acoustic signal are more easily affected by the location of the acoustic anchors. So the classification of multiple NLOS shielding methods could be tricky.
Coping with special situations for the indoor localization system, we use the new acoustic signal sample constructing scheme and process to construct a new training sample set of our own, especially for NLOS acoustic signal production description in the real-world situation. Secondly, as for the classification module, we utilize an advanced swarm intelligence optimization strategy to improve the classification accuracy. We also use the elitist strategy and take into account the relationship among the algorithm convergence, the sample rearrangement process, and the trend of sample selection in total. They will be the support for adjustments in our indoor localization system.
According to the methods mentioned above, focusing on the NLOS acoustic signals blocked by different obstacles turns out to be critical corresponding to different production processes precisely. There are still some circumstances that the LOS scenario for one anchor in the localization system will suddenly become an NLOS scenario, like in the museum or in some office buildings. During acoustic signal acquisition and production, the irregular switching between LOS and NLOS scenario will be added in order to strengthen its ability to identify NLOS acoustic signals in the real-world situations.
The spectral distribution characteristic graph of acoustic signals in the specific sites will have a great impact on the deployment and schematic design. As for the hall of Guilin smart Industrial Park, some positive changes are needed for the system’s deploying area. As for the normal situations of museums or office buildings, their background noise waterfall is shown in Figure 7.
It is imperative to make sure that there is a huge difference between the frequency band of acoustic localization signal and the environmental noise. It can be seen from Figure 7 that the ambient noise (caused by alarm bell, exhaust fan, air conditioner, footsteps, human speech, etc.) at the hall is almost all distributed at the range of 0 to 10 kHz. On the one hand, the practical range of the human ear is between 16 kHz and 18 kHz for most adults. On the other hand, the acoustic signal frequency used by the indoor localization system is distributed between 17.5 kHz and 20.5 kHz and only a small part of coincidence with the background noise. This kind of acoustic signal can hardly be heard by the human ear under this circumstance, which greatly avoids the noise pollution. This frequency band is also very suitable for the sensitive situation involved in this paper. Obviously, the composite window function mentioned earlier also has positive effects. In this situation, the frequency distribution of localization acoustic signal is shown in Figure 8.
In order to further reduce the NLOS influence, a waterproof and moisture-proof special plastic box in [30] should be used to carry the acoustic anchors. According to the conditions, many multicalibre USB adapters appear to be required. In order to complete the deployment of the system, the USB adapters are used from Shantou Shaowei Company. Then, fix the acoustic anchors below the ceiling, and this deploying way will further reduce the possibility of the NLOS scenario and noise pollution, as shown in Figure 9.
5.1. Improved Acoustic Signal Sample Production
For the acoustic signal in the system, different smartphones are used in the system during the localization periods for the test. The smartphones used are HUAWEI Honor 4 and 7, HUAWEI Nova3, OPPO Realme X, XiaoMi 10, and HUAWEI 20 Pro, respectively, in our system. The OPPO Realme X smartphone is used as the main anchor, and the microphone used is also provided by this smartphone. It can be convinced that for different types of smartphones, it is essential to pay more attention to the coordinates before the experiment due to their different speakers or horns. Some smartphones used in the system may need to be fixed in a special way to prevent unnecessary blocking. Turning them upside down is a proper choice during our experiments. The deployment process is different from the experiment process since the core components and microprocessors are fixed in the special plastic box mentioned above during our deployment. It is supposed to select the appropriate microprocessor and control the orientations to minimize unnecessary NLOS transmission paths as possible.
The test and the sample production experiments are completed in room 7305, teaching building 7, Jinjiling Campus, Guilin University of Electronic Technology. For acoustic signal production and acquisition, the key aspect is the NLOS acoustic signal. Corresponding to different batches, appropriate obstacles are selected to complete shielding, and a pedestrian shielding factor also needs to be added. For dynamic NLOS shielding, a total of five volunteers are used as pedestrian factors, and these five people are different in height and body style. For static NLOS shielding, obstacles of five different materials are also selected as baffles.
In order to complete the dynamic change of LOS and NLOS scenarios, the PC is used to control the generation time of acoustic signal, and the receiver or the microphone array on the other side completes the receiving task. The acoustic source used is the Harman Kardon Aura Studio 360 Degree Omnidirectional System. It supports 220 V voltage and theoretical power of 57 W with effective frequency range of 50 Hz-20 kHz. The receiver microphone is ECM 8000 omnidirectional microphone from Behringer, and the frequency response range is 20 Hz-20 kHz. Omnidirectional microphone is not easy to block directly, so obstacles are set in front of receiving terminal to complete the shielding. It should be noted that even in the LOS scenario, some obstacles can be properly reserved, so that it can better simulate these situations. The LOS acoustic signal experimental scene and the top view of that are shown in Figure 10.
During the production and acquisition process, the same acoustic source and the same type of receiver are used. Both kinds of shielding methods need to be maintained in this process. The first shielding method can be completed by using books, cabinets, composite baffles, and so on to form the second batch of sample set. The second shielding method can be completed by using different team members to form the third batch of the sample set. The two shielding methods and their top view of the scenes, respectively, correspond to Figures 11(a) and 11(b).
(a)
(b)
For LOS acoustic signal, the production principles and acquisition processes are as follows: (1)For each time, three microphones with the same parameters are used to receive the transmitted acoustic signals, so that they are horizontally set on the same line from the ground. The acoustic source used at the transmitting terminal determines the beginning and ending time. Acoustic signals should be sent on time. Each terminal is connected through Wi-Fi and controlled by our own software(2)For LOS acoustic signals, it is obvious that there should be no obstacles between the microphone and the speaker. Also it is necessary to ensure that the microphone used in each experiment and acquisition process is of the same type and height(3)The microphone is located at the same position, and the receiving end is, respectively, arranged at 0.8 m, 1.2 m, and 1.5 m tall from the ground. According to the previous steps, the three speakers send a 16-20 kHz chirp signal, respectively. During these periods, the height for the three receiving ends from the ground needs to be consistent(4)The receiving end can move 1-3 unit distances backward or forward each time (according to the actual situation of the site, unit distance should be an appropriate length between 10 cm and 20 cm, and the value is fixed). After that, multiple receiving ends can repeat steps 1-3 independently to complete all acoustic signal acquisition several times
For NLOS acoustic signal, the production principles and acquisition processes are as follows: (1)For each time, three microphones with the same parameters in the LOS scenario are used to receive the transmitted acoustic signals, which are horizontally set on the same line from the ground. The acoustic source used at the transmitting terminal determines the beginning and ending period. Acoustic signals should be sent on time. Each terminal is connected through Wi-Fi and controlled by our own software(2)During the NLOS scenario in the first method, we use obstacles of different materials to block the path between the microphones and the speakers. The obstacles are close to each microphone but not close to the speaker itself completely. The sudden disappearance of the obstacle shall be properly added. When the NLOS scenario in the second method is performed, the receiving end can use the smartphone speaker or horn to replace the old receiver and the remaining factors are the same as above(3)We adjust the height of all receivers and implement the other parts in the same way as that in the LOS scenario. Keep them separately from each other to avoid omission. The transmission mode of acoustic signal is consistent with that in the LOS scenario above(4)When blocking, ignore the possibility that obstacles can block multiple receivers in each localization loop, and we carry out the other parts in the same was as that in the LOS scenario. Steps 1-3 can be repeated if necessary(5)Different from the first NLOS shielding method mentioned above, the second NLOS shielding method is often affected by the height and shape of our different participants. If there is a practical demand, different experimental participants can be used in different localization loops. Other parts can refer to the processes in the LOS scenarios
Corresponding to the two NLOS shielding methods, the first and second batches focus more on the unmovable objects, which may be more common in some situations. The third batch focuses more on human blocking occlusion, and its probability of occurrence is higher than the first two batches in these sites. The sample training weights of the last two batches can be adjusted or modified according to the actual needs if necessary. On this basis, it will facilitate the classification ability of LOS and NLOS acoustic signals in complex NLOS scenarios comprehensively and then improve the positioning accuracy of indoor localization system after relative processing.
Obviously, as for two NLOS shielding methods mentioned above, they are quite different with each other. Previously, the way of NLOS shielding is relatively simple in the same type of indoor localization system in [4, 11, 13]. And they ignore the difference between rigid body occlusion and human body occlusion, so their training data set does not have the good performance on our classification part. Matching all kinds of NLOS shielding methods in the indoor localization system turns out to be necessary. Judging by the time delay gain distribution in the two cases below, ① is used to represent the first NLOS shielding method and ② is used to represent the second NLOS shielding method; it can be seen that there are great differences between the two kinds of NLOS shielding methods, as shown in Figure 12.
The NLOS shielding effect of acoustic localization signal is related to the distance from the obstacle to the acoustic source and its ability of absorbing the acoustic signal. Obviously, the delay gain distribution will greatly affect the feature extraction algorithm and then affect the classification part in the system. Therefore, it is of great importance to build our own databases involved in both two NLOS shielding methods. In summary, 4200 special acoustic localization signals will be used for the classification part.
5.2. NLOS Identification Test and Analysis
With the help of the elitist strategy and inertia weight nonlinear decrement, the pertinence of classifier training is enhanced, the convergence speed of the algorithm is accelerated, and the classification accuracy is significantly improved. The system can be controlled to turn on or off the NLOS identification process according to its actual needs. The project or classifier mentioned in [4, 9, 12–14] has been tested with our own data set. All the results are summarized in Table 1 (italics indicate the method used in this system).
Thanks to the more accurate NLOS identification process, the system will greatly enhance the indoor positioning accuracy in complex NLOS scenarios. It should be noticed that according to the relative time delay gain or RSS, the NLOS identification effect based on human body shielding is actually stronger than that based on unmovable obstacles.
5.3. NLOS Localization Test and Analysis
We completed the experiment several times in room 7305, teaching building 7, Jinjiling Campus, Guilin University of Electronic Technology. During the test, the indoor temperature is about 12~26°C and the environment noise is about 52 dB. The environmental parameters are similar to the deploying area. Each anchor is mainly composed of four parts: smartphone, audio power amplifier, piezoelectric tweeter, and power supply part. First, after receiving the localization command from the smartphone at the anchor, each anchor sends a predetermined acoustic signal in its localization loop. Then, the acoustic signal is amplified by the audio power amplifier, and it needs to transmit to the acoustic anchor of the system. Power supply parts are also important for the anchors. If it is to be used in some areas (e.g., Japan), the voltage converter needs adding before charging. In order to reduce the low-frequency noise interference, the piezoelectric tweeter needs to be connected with one 3.3 V electrodeless capacitor, which can also block DC, and should be connected with the composite window function module. In order to protect the circuit, a voltage dividing circuit can be added if necessary based on the Arduino module. Meanwhile, these components mentioned above need to be linked on a specific circuit board before deployment. The NTS-334R electronic total station is used for finishing axis calibration before the experiment and the deployment. Physical picture about the acoustic anchor and the electronic total station used are shown in Figures 13(a) and 13(b).
(a)
(b)
As for the acoustic anchors in the experiment scenarios, more details should be given. For the experimental acoustic anchor mentioned in Figure 13(a), the piezoelectric tweeter improves the transmission and reception quality of acoustic localization signal. Otherwise, the receivers or microprocessors may not effectively receive the acoustic signal. The POE part supports the necessary conversion process for audio power amplifier and composite window function part. The USB power supply part is the backbone of the acoustic anchor, and the APP on the Android platform controls the key process. As the core of the acoustic anchor, the APP is developed based on Android Studio to implement all necessary functions. For each APP, it depends on all these four preconditions below: (1)The Android application is implemented based on Android 6.0, and it can no longer run in the Android platform with too low version(2)The Android application must be used in the indoor situations where the reference anchor and all the acoustic anchors are fixed on the specific coordinates. The rectangular plane coordinate system involved shall not be changed in the localization periods(3)The Android application must obtain the effective recording, network, and storage permissions in advance continuously before activation(4)The Android application should be connecting to the network and in the same network environment as other terminals
For the acoustic anchors in the indoor localization system, MVP (Model View Presenter) architecture is used. The view layer is responsible for monitoring the event. It answers to the acoustic anchor and update the UI display in time. As a bridge from the view layer to the model layer, the presenter layer undertakes the function of decoupling. The model layer carries out the key information to this APP, and its functions include data processing, file processing, and algorithm implementation. The architecture of this APP is shown in Figure 14.
Corresponding to Figure 14, the data processing part ① needs to interact with the Android terminal and piezoelectric tweeter to complete acoustic signal transmission and reception, which is also a prerequisite for subsequent data analysis. In order to satisfy the demand of storing, for any indoor localization loops, the conversion process from “.pcm” to “.wav” needs to be added. Referring to the processes and functions in [31] to finish this part is necessary. This makes it easier to process these audio files and save them. For the part of ②, the system will use the processed acoustic signal and significant preconditions to complete the NLOS identification process after the audio file cutting processing. According to the NLOS identification results, if the indoor localization can be completed, the coordinate of the main anchor is obtained through the advanced TDOA and the results are transmitted to the view layer. Only those basic components are used by us to design and implement UI. It has appropriate image display function and low computing pressure. On the promise of the four preconditions mentioned above, it is suitable for most Android smartphones to install.
When the user uses it for the first time, some basic operations for Android APP is required. After the user clicks the software icon, the APP will start the permission detection. After the permissions mentioned above are confirmed and authorized, the APP can be used by users. Then, the user needs to fill in the IP address manually, and next, it begins the service. If the connection is constructed successfully, the login process will be completed. At this time, the user can click “Begin Localization” to start the localization periods and click “End Localization” to end all the services according to their own wishes.
After the ZKLocPro system is set, an effective area is delimited in the measured site with a range of , and this part is marked with a red dotted line in Figure 15. Use the desks with equal height, liftable platforms, or plastic boxes with adjustable height to support them, so that the geometric centre of each anchor is consistent from the ground within a controllable range in the rectangular plane coordinate system. As follows, take from ① anchor to ④ anchor as the positive direction of the axis and from ① anchor to ② anchor as the positive direction of the axis, and the geometric centre of ① anchor is set as the point. The localization loop starts from the first anchor till the fourth anchor, respectively, according to the counterclockwise direction, which is sequentially marked in one localization loop.
Obviously, the NLOS identification process is only aimed at the wav files after cutting, so as to avoid misjudgement caused by the protecting interval. The amplitude peak of kurtosis or skewness can be referred to determine the cutting position of the acoustic signal in a loop preliminarily. As for the overall wav file obtained at the main point, four new wav files’ feature set can be obtained after extracting the process based on the time delay gain at all other anchors. In some situations, the relative gain can also be used. Feature extraction is completed on the Android platform, and the results are processed by PC. After the classification result is obtained, if it can finish the localization task, the final results should be transmitted back to the smartphone held by the user, one complete localization period will be finished, and the next loop may begin if necessary. The coordinates of the main anchor are to be required; the APP commanded by the user controls the periods of the localization process, displays the results obtained after calculation, and marks the target on the map. So far, the entire indoor localization process and the coordinate displaying UI are shown in Figure 15.
There are 8 coordinates used for the experiment, which are (1.5, 2.5), (1.5, 2), (1, 2.5), (1, 2), (1, 1.5), (1.5, 1.5), (1.5, 1), and (1, 1). After optimization, the ZKLocPro system’s error is one-sixth to one-third of the original one; the positioning error of the system will be within 12 cm and satisfy the localization demands of decimetre level. Compared with the old system without NLOS identification or the following adjustments, the positioning accuracy is significantly improved in complex NLOS scenarios. For different NLOS localization methods, the results are shown in Table 2 (italics indicate the method used in this system).
In this paper, the system involves a limited number of the acoustic anchors, and each anchor has a significant impact on the indoor localization system. If the system is located in some areas with obvious topographic relief or in some places with special demands for the acoustic anchors, the localization processes may not be completed due to the inability to finish the deployment reasonably. In this paper, the core of each acoustic anchor is an Android smartphone with normal audio transceiver function during our experiment. In some situations or our deployment sites, an effective microprocessor can be used to replace it, so as to reduce the cost. The GY-1 piezoelectric tweeter used in the acoustic anchor is from Guangzhou, Kuaiyibu Company, with the power of 30 W, which has limitations in transmitting high-frequency acoustic signals. According to the experiment, when the distance between the reference anchor and the localization anchor exceeds 5 m, the acoustic localization signal’s receiving effect will decline, which is not conducive to the indoor localization. The disadvantages mentioned above will become the motivations and the directions of the next-stage optimization in those more complicated situations.
6. Conclusions and Future Works
This paper focuses on the LOS and NLOS acoustic localization signal classification part in the real-world situations. According to the classification results, some useful steps should be added to adjust the main process of the localization system and then improve the positioning accuracy. We design and construct the ZKLocPro system to finish the indoor localization task. In this system, we upgrade SVM based on an advanced swarm intelligence optimization strategy and related updating strategies. Then, we successfully solve the following three problems. (1) Based on the system’s limitation, an effective acoustic signal sample production process is proposed, and the adjusted feature extraction process for our system is implemented. (2) For the case of insufficient discrimination of existing acoustic signal training samples, the elitist strategy is added to energize the training process, so as to make the training more targeted. At the same time, in order to make it better applied to the system, some reasonable schemes, a composite window function, and some interaction processes are given. (3) Aiming at the weakness of SVM itself, on the one hand, on the premise of not bringing too much computational burden, we improve the SVM to obtain higher NLOS identification accuracy. On the other hand, in order to interconnect with the advanced swarm intelligence optimization strategy in the NLOS scenario, supporting APPs are implemented on multiple platforms to construct the ZKLocPro system. Based on these, the classification accuracy is improved by 8%-12% compared with others at least. According to identification result, the system will adjust the localization process for the NLOS scenario in time to improve the positioning accuracy sharply. In total, although the platform is slightly different, these methods and following processes can also make other systems like the Linloc system and so on more effective. In the future, the system will be divided into online and offline parts more often. All two parts will perform their duties better to synergize the LBS.
Obviously, with the help of hardware upgrading and the more powerful swarm intelligence optimization algorithms, the system will develop steadily towards intelligence and integration in the future. For the special situations on the deployment, it is necessary to determine the most proper way to deploy all the equipment and the periodic law of acoustic signal transceiver processing plan according to their needs. In order to provide the more powerful support, 3D map construction, background modelling, and simulation can be finished in 3DsMAX2020 and AutoCAD2020, and the plan on deployment can be determined in advance. This is conducive to the indoor localization system in the real-world situation and will become the key to the same type of the system in the future.
Data Availability
The LOS and NLOS localization acoustic signal data used to support the findings have not been made available because this paper is funded by the National Natural Science Foundation of China under Grant 61961010. This grant is still in the research phase, and all research data is currently restricted to disclosure within the project team.
Conflicts of Interest
The authors declare that there is no conflict of interest regarding the publication of this paper.
Acknowledgments
This research work was supported by the National Natural Science Foundation of China under Grants 62071135 and 61961010; the Key Laboratory Fund of Cognitive Radio and Information Processing, the Guangxi Science and Technology Plan Project under Grant No. AD18281044; and the Director Fund Project of the Key Laboratory of Cognitive Radio and Information Processing, Ministry of Education under Grant Nos. CRKL190104 and CRKL180203. We also appreciate Dr. Yu Hong-gang’s help in grammar and translation.