Abstract
Wireless communications are vulnerable against radio frequency (RF) interference which might be caused either intentionally or unintentionally. A particular subset of wireless networks, Vehicular Ad-hoc NETworks (VANET), which incorporate a series of safety-critical applications, may be a potential target of RF jamming with detrimental safety effects. To ensure secure communications between entities and in order to make the network robust against this type of attacks, an accurate detection scheme must be adopted. In this paper, we introduce a detection scheme that is based on supervised learning. The k-nearest neighbors (KNN) and random forest (RaFo) methods are used, including features, among which one is the metric of the variations of relative speed (VRS) between the jammer and the receiver. VRS is estimated from the combined value of the useful and the jamming signal at the receiver. The KNN-VRS and RaFo-VRS classification algorithms are able to detect various cases of denial-of-service (DoS) RF jamming attacks and differentiate those attacks from cases of interference with very high accuracy.
1. Introduction
A prevalent prediction is that fully autonomous vehicles, capable of self-navigating in unpredictable real-world environments with little human feedback, will flood the global market by 2025 [1]. Autonomous vehicle control imposes very strict security requirements for the wireless communication channels [2] which are used by a fleet of vehicles [3]. Specifically, the connected vehicles use the connected adapted cruise control (CACC) technology, in which the following vehicles learn the lead vehicle’s dynamics via intervehicle communication and through them they determine their movement. However, an RF jamming attack can overload the wireless medium leading to large packet losses. So, the platoons of vehicles can become unsafe and collisions are possible. Moreover, with the intelligent vehicle grid technology, each vehicle becomes a sensor platform absorbing information from the environment or from other vehicles (also called Internet of Vehicles (IoV)). Vehicles also feed each other or infrastructure for assisting in safe navigation and traffic management.
Wireless communications, however, are vulnerable against a wide range of attacks. An attack that is particularly hard to detect in every wireless network is the RF jamming attack [4]. An RF jamming attack reduces the availability of the wireless medium making the successful detection of a jamming attack may be obstructed by several conditions that might occur in an urban environment, such as unintentional interference caused by other wireless nodes, poor link conditions, etc. In a VANET, RF jamming attack detection is even more challenging due to the constant and rapid changes in topology and the high mobility of the vehicles. Detection becomes even harder with the presence of a variety of jammers and unintentional interference sources in the same area. Jamming may affect the communication between vehicles (V2V communication) or the communication between vehicles and roadside units, namely, RSUs (V2R communication).
Over the last few years, there have been several experimental approaches for jamming detection [4–7], some of which suggest the use of machine learning techniques [6, 8]. However, only Puñal et al. [6] examine closely the adoption of machine learning techniques for jamming detection. None of the related works that focused on machine learning-based schemes has investigated the use of the speed of the involved vehicles as an extrafeature for classifying jamming attacks in VANETS. In this work, we show that this is a critical feature and, more specifically, the variations of relative speed (VRS) metric. VRS is used as a new feature for jamming classification in realistic scenarios with a minimum number of assumptions leading to increases in classification accuracy.
The proposed VRS metric, extracted at the application layer [9, 10], is combined with classic physical layer metrics leading to a cross-layer classification scheme. The intuition behind the use of the VRS is the following. In the general case, jamming reduces the receiver signal-to-interference-and-noise ratio (SINR), a problem that can be addressed with classic communication algorithms. However, SINR can be reduced due to unintentional interference, a problem very prevalent in dense populated areas where vehicles operate. Hence, for jamming detection, the actual reason behind the reduction in the SINR and the packet-delivery-ratio (PDR) has to be determined. The proposed VRS metric reveals the behavior of the jammer in relation to the receiver, specifically, the variations of its relative speed. An unintentional source of interference does not exhibit a specific pattern in its relative speed, allowing us thus to effectively differentiate the cases where a malicious intentional source of interference, namely, jammer, moves in ways that intend to disrupt communication. Our extensive results indicate that the proposed scheme can effectively differentiate the case of jamming attack from that of an interfering wireless source.
Accurate detection is also important because these two problems could be addressed differently; that is, in the case of interference, an interference cancellation (IC) scheme [11] is needed, while techniques such as spectral evasion (channel surfing and spatial retreats) scheme can be used for jamming attacks. With the use of more sophisticated techniques for alleviating the problem, the proposed scheme can be used as a first step of a process that aims at keeping alive the wireless communication between a transmitter and receiver, by detecting the exact cause of the wireless interference followed by appropriate actions related to the physical location of the nodes. Such actions could be the de-routing of the malicious vehicle from a specific area or the rerouting of legitimate vehicles towards different areas, free of any RF jamming or interference. Lack of such smart detection mechanisms could lead to incorrect de-routing decisions that may compromise the different objectives of applications that use intervehicle communications (IVC). For this reason, we tested both, a typical form of RF jamming, which is the continuous jamming, and a more smart reactive jamming.
The main contribution of this paper is the introduction of a proactive detection method against potential RF jamming attacks with fairly good detection results. This detection system is also able to differentiate interference from malicious RF jamming. Additionally, it is able to distinguish the unique characteristics of each attack especially when the proposed VRS metric is utilized among the other cross-layer features. The accuracy of the proposed detection method is about or over under different supervised learning testing cases and under realistic values of the relative speed between the jammer and the receiver. This result is significantly improved as compared to other corresponding methods in the literature.
One key application area for our scheme is vehicle platoons in which an exterior or an interior attacker can cause significant instability in the CACC of the vehicle stream [12]. Our classifier could be used as a trustworthy indicator of a jamming attack; thus, the control model of the platoon could change from CACC to noncooperative adaptive cruise control (ACC), relying solely on radar techniques. This control mode switch can be considered as a mitigation technique to the impact of the attack [12]. For the evaluation of our approach, one interference-only scenario and two jamming attack scenarios have been designed and tested.
The rest of this paper is structured as follows. Section 2 provides an overview of related work in the domain of attack detection. Section 3 describes the topology and the channel model of our scenarios. Section 4.1 describes the methodology used for the estimation of the relative speed. Section 4.2 presents the proposed machine learning-based jamming detection system. Section 5 describes the simulation setup. Section 6 presents the experimental results and comparisons. Finally, Section 7 summarizes our findings and concludes our work.
2. Related Work
Several recent works have proposed machine learning-based techniques for attack detection in vehicular ad-hoc networks. Puñal et al. [6] used metrics that include the noise and channel busy ratio (CBR), packet delivery ratio (PDR), maximum inactive time (Max IT), and received signal strength (RSS), to detect attacks with machine learning techniques, and examined the cases of reactive and constant jammers.
Azogu et al. [5] proposed a new mechanism, called the hideaway strategy, according to which all nodes should remain silent while the network is under a jamming attack. Bißmeyer et al. [13] proposed a detection scheme that is based on the verification of vehicle movement data and on the assumption that a certain space will be occupied by only one vehicle at a certain time.
Malebary et al. [14] presented a two-phase jamming detection method that utilized metrics such as the RSS, the packet delivery/send ratio (PDSR), and the packet loss ratio (PLR), as well consistency, checks to distinguish a jamming from a no-jamming situation. In the first phase, which is the initialization phase, the values of the RSS, the packet delivery/send ratio (PDSR), and packet loss ratio (PLR) are calculated by the RSUs in a jammer-free network. Furthermore, a max value for the RSS is obtained for every PDSR value as well as two threshold values, equal to the maximum PDSR and to the minimum PLR, respectively. In the second phase, when a PDSR value is lower than the defined threshold and a PLR value is higher than the respective threshold, a consistency check is conducted to determine whether the low PDSR value is consistent with the RSS value assigned in phase one, thus determining a jamming or no-jamming situation.
The authors in [15] proposed a data mining-based method for real-time detection of radio jamming DoS attacks in IEEE 802.11p V2V communications for platoon of vehicles. The state-of-the-art methods are compared with the proposed method which allows operating under the realistic assumption of random jitter accompanying every cooperative awareness message (CAM) transmission. However, only features from the network layer are utilized. Mokdad et al. [16, 17] proposed a scheme for detecting a jamming attack in vehicular ad-hoc networks that depends on the variations of the PDR. The approach is based on the premise in which only packets that originate from the sender are allowed through the cyclic redundancy check (CRC) and the PDR is equal to the ratio of these packets and the total number of packets received. Puñal et al., in [18], generated a set of jammers and implemented a variety of jamming scenarios, both indoor and outdoor, under different jamming behaviors (constant, reactive, and pilot jamming) in order to address the impact of an RF jammer in VANET communications.
Quyoom et al. [19] presented an RF jamming attack that consists of radio signals maliciously emitted to disrupt legitimate communications. This type of jamming is already known to be a big threat for any type of wireless network. With the rise in safety-critical vehicular wireless applications, this is likely to become a constraining issue for their deployment in the future.
RoselinMary et al. [20] proposed an approach that is based on the detection of malicious and irrelevant packets using the number of broadcast packets per second (frequency) and the velocity of the vehicle that the packets are sent from. This method calculates the frequency, e.g., the number of broadcast packets per second, and the velocity and then starts the detection algorithm. If the frequency and the velocity are both high and above a threshold, then the packets are malicious, whereas if they are between a low and a high threshold value, the packet is real.
A subcategory of related papers dealt with real-time medium access control- (MAC-) based jamming detection method to meet the requirements of safety applications in vehicular networks. These methods operate either under realistic assumption of random jitter accompanying every CAM transmission [21] or the decision of the detector (monitor) depends on the number of nearby vehicles and the number of successful transmissions and failed transmissions [22]. These detection methods can more accurately distinguish the causes of failed transmissions such as contention collisions, interference, and jamming attacks. In [23], the authors proposed a method for DoS attack detection in wireless sensor networks (WSNs). This method is based on the grouping of sensor nodes and the timestamp and the PDR calculated from one node to another one. However, all the above papers focused on simplistic jamming attacks such as “random jamming” or “ON-OFF jamming” without taking into account smarter jammers such as reactive jammer. Lastly, Mowla et al. [24] proposed a federated learning-based on-device jamming attack detection security architecture for flying ad-hoc network (FANET) using the RSSI and PDR features with a fairly good accuracy results in detecting the RF jamming attack. All the aforementioned jamming detection approaches used parameters only from the MAC or the physical layer for training and testing without exploiting upper layer features. Feng and Hua [25] proposed jamming detection schemes based on a variety of machine learning algorithms. They incorporate the information from the physical layer, the MAC layer, and the network layer (such as RSS, carrier sense time, noise, and PDR) for training and testing. Lastly, there are recent works in the literature that use either machine learning [26] or deep learning for a multistage jamming detection scheme in 5G networks: the cloud radio access network (C-RAN) [27]. However, these methods have not been tested on vehicular networks that have special features such as high-speed moving nodes. Only the authors, in [28], adopt a cross-layer approach incorporating also an application layer features for detecting and classifying different types of RF jamming attacks in VANETs. Specifically, the IDS that was proposed in [28] is able to differentiate a RF jamming attack from spoofing attacks in connected autonomous vehicles (CAVs).
Sharanya and Karthikeyan [29] proposed a support vector machine (SVM) algorithm with modified fading memory (MFM) for classifying legitimate and malicious nodes. The proposed classification scheme considers the following critical parameters to classify a node as malicious node, namely, power ratio, signal strength, packet delivery ratio, speed of node, number of packets generated, and transmission power. Their proposed system has two specific phases.
Lastly, Karagiannis and Argyriou [10] proposed an RF jamming attack detection scheme using unsupervised learning with clustering. The novelty of the above paper is that the relative speed metric is utilized between the jammer and the receiver, along with other parameters, in order to differentiate intentional from unintentional jamming as well as identify the unique characteristics of each jamming attack. However, this relative speed metric is assumed to be available without any form of estimation.
In all the previous works that were proposed, machine-learning based schemes, the estimated variations of the relative speed have not been considered as a classification feature. Our proposed system is the first one in the literature that uses the point-to-point RF communication in order to estimate the relative speed metric.
3. System Model
3.1. Topology
Our system topology is represented in Figure 1. In the left part (a), an interference scenario is presented, in which we assume that no jammer is present in the network. This scenario is important in order to be able to evaluate the efficiency of our method in differentiating jamming from interference. The vehicle travels, when, at some point, it passes through an area with significant RF interference that is caused by a RSU. In the right part of this figure (b), a jamming situation is presented. The topology we adopt for this case involves a moving vehicle , which serves as the target of the jammer, another vehicle that is the transmitter of the useful signal, and the jamming vehicle that tries to intervene in the communication between and . The travelling speed of , namely, , is equal to the travelling speed of , namely, . Moreover, we assume the presence of a static object in the area that causes multipath fading from reflections, as it is usually in urban environments. Upon spotting its target, the jammer begins following it and starts jamming either continuously or periodically (in order to stay undetected for as long as possible).
3.2. Rician Fading Model
In our work, we adopt the Rician fading model that is a channel model which includes path loss and also Rayleigh fading [30]. When a signal is transmitted, whether it is a useful signal or a jamming one, this model adds multipath fading in addition to thermal noise. It is assumed that a line-of-sight (LOS) ray and nonline-of-sight (NLOS) ray exist in the area. The combined baseband signal that the receiver receives from the jammer and the transmitter iswherewhere are the Rician fading channel models between transmitter-receiver and jammer-receiver, respectively. This type of channel model includes path loss and also Rayleigh fading. and are complex Gaussian variables capturing the Rayleigh fading between transmitter-receiver and jammer-receiver, and are the symbols that are transmitted from the transmitter and the jammer, respectively, for which the BPSK modulation is used. This modulation scheme is preferred because it achieves lower bit error rate providing a reliable communication between and . Moreover, this modulation scheme is the most robust in a high interference environment. In (2) and (3), is the carrier frequency, is the maximum Doppler shift, and are the transmission power per symbol of the useful and of the jamming signal, respectively, and is the channel noise at time instant . The terms correspond to the distance between the transmitter and the reflected object and between the jammer and the reflected object, respectively.
The terms correspond to the distance between the transmitter and the receiver and between the jammer and the receiver. In (2) and (3), the travel distance of the LOS rays is equal to . On the contrary, the travel distance of the NLOS rays is , respectively. Moreover, is the incidence angle of departure (AOD) between the vector of speed and the signal vector of the transmitter, is the incidence AOD between the vector of speed and the signal vector of the jammer, is the excess delay time for the transmitter and jammer signal ray (that may be caused due to ground reflection), and is the current time instant. For the remainder of this paper, we will use the parameter as the transmitter-receiver complex amplitude associated with the LOS path and the parameter as the jammer-receiver complex amplitude. The above complex amplitude values are known at the receiver.
3.3. System Overview
In our system model, a fixed number of known pilot symbols are sent using the wireless IEEE 802.11p standard [14] over consecutive time instants from the transmitter to the receiver. At the same time, the jammer simultaneously transmits over consecutive time instants’ random jamming symbols to the receiver. Using these pilots, the LOS channel and the NLOS channels between the jammer and the receiver are estimated by the receiver.
The basic idea is to first estimate the relative speed between the jammer and the receiver, exploiting the RF Doppler shift. We use the variations of the estimated relative speed as a new feature in a supervised machine learning algorithm for RF jamming attack detection. Along with the relative speed from the application layer, we use cross-layer data that we obtain from the physical layer, such as the received signal strength indicator (RSSI), the SINR, and the PDR. Two classification algorithms are investigated, namely, the k-nearest neighbors (KNN) and the random forest (RaFo) algorithm, respectively.
3.4. Jamming Scenarios
We assume that the jammer continuously transmits so as to overload the wireless medium conducting a DoS attack [31]. We investigate three different attack scenarios, namely, interference scenario, smart attack scenario, and constant attack scenario, each representing a jamming attack case that could affect a VANET in real life.
In the interference scenario, we assume that no jammer is present in the network. This scenario is useful for evaluating the efficiency of our method in differentiating jamming from interference. The vehicle travels, when, at some point, it passes through an area with significant RF interference that affects the communication with other vehicles or the RSU. The smart attack scenario models an intelligent jammer behavior [32]. This smart jammer is designed to start transmitting in a reactive way upon sensing energy above a certain threshold. We set the latter to as it is empirically determined to be an average threshold between jammer sensitivity and false transmission detection rate [18, 33]. Using this minimum threshold, each ongoing transmission can be detected by the reactive jammer. The standard protocol wireless access in vehicular environment (WAVE) IEEE 802.11p orthogonal frequency-division multiplexing (OFDM) frame format consists of the OFDM PHY layer convergence protocol (PLCP) preamble, PLCP header, MAC header, wave short message protocol (WSMP) header, PLCP service data unit (PSDU), tail bits, and pad bits. In the PLCP preamble field, the preamble consists of ten identical short training symbols and two identical long training symbols. The smart jammer is designed to affect the header of the 802.11p frame sent from to . When the next OFDM signal can be transmitted, there is an idle time of required to set up the next transmission. If the detected energy exceeds the threshold during a certain time span , an ongoing 802.11p transmission is assumed by the jammer. The time interval of the detection is the sum between the idle time and a small value as the detection time to avoid reacting to sporadic noise power peaks. In the case where the detected energy exceeds the threshold during a certain time span, the jammer starts its transmission for a duration of in order to jam a substantial part of the packet header to prevent being decoded by the receiver, as illustrated in Figure 2.
Specifically, a smart jammer starts following the victim vehicle, while transmitting a jamming signal. When the jammer reaches its target at a distance of about , it retreats to a different position in order to stay undetected and transmits in a reactive way as described above. The most common approach in [33] is when the jammer keeps changing its transmission power, thus achieving the same disrupt or thwart in the communication (DoS attack) without the need of changing its distance from the target. With our smart attack, we aim at affecting the communication of the - pair, with the jammer detection being more difficult, pointing out the importance of the proposed VRS metric for the detection accuracy results. For that reason, the smart jammer alters also its position with the aim of staying undetected.
In the constant attack scenario, we study the case of a jammer that follows the receiver while transmitting constantly at a minimum power. When the jammer reaches its target, it begins transmitting constantly with its full power without any intention to stay undetected as in the smart attack scenario.
4. Proposed Detection System Based on Supervised Learning
4.1. Relative Speed Estimation
In this section, we present the basic idea regarding the estimation of the relative speed between the jammer and the victim vehicle. Based on the obtained values, the VRS metric is generated and then used for classification. The relative speed metric as defined in [10] iswhere and are the speed of the jammer and the speed of the receiver, respectively.
From (3), the multipath combined channels are estimated, using a minimum mean square error (MMSE) estimator [9]. By exploiting the Doppler phenomenon for modeling the LOS channel between the jammer and the receiver, we estimate the above-defined relative speed metric, as described in [9]. Note that the jammer estimation method is based on the passive communication between -.
4.2. Proposed Algorithm
To make our detection method robust, apart from using physical and network metrics that were already used in related works, we use the VRS metric that is derived from the application layer and can be efficiently estimated from the RF signals (see Section 4.1). Our method uses this new metric, as an extrafeature in a cross-layer approach, along with other metrics from the physical layer for the classification process. All these metrics are presented in Table 1.
To generate the VRS metric for classification, we make three fundamental assumptions [34]:(1)When the relative speed is equal to zero and remains unchanged, it indicates the existence of a constant jammer that follows the victim vehicle(2)When the relative speed is not equal to zero and remains unchanged, it indicates the absence of a moving jammer as the relative speed is equal to the speed of the receiver and the speed of the jammer is equal to zero(3)When the relative speed is not equal to zero for a period of time and then becomes zero while remaining unchanged, it indicates the existence of a jammer that begins following the target after reaching it
The common characteristic of these assumptions is that the speed of the participating nonmalicious vehicles remains unchanged and is always greater than zero.
However, in a real-life scenario, such as the one that we study, the speed—and as a consequence the relative speed—may not remain constant during the observation period. In other words, if we want to accurately model an urban environment, we have to consider the fact that the vehicles can alter their travelling speed. To handle these real-life situations, while still using the previously presented assumptions, we introduce the Variations of Relative Speed (Algorithm 1) (VRS algorithm).
|
The VRS algorithm detects changes in the relative speed of the training sample. To ensure that the relative speed in the current time instance along with the speed from previous as well as subsequent observations are used along with a series of control flow statements, the algorithm is divided into two main parts; the first considers the case in which the relative speed value is not equal to zero and the second the opposite case, each one with its own logical checks to determine the existence of a threat.
Apart from the estimated relative speed, in order to handle cases of speed alterations, the speed of the receiver has to be examined as well. If is not equal to zero, then either there is no jammer present (and only interference may potentially affect the wireless communication) or there is a jammer that has not yet reached the receiver. To identify in which case we are, we have to examine whether or not there has been a variation in the relative speed compared to a previous time instance.
Observing a variation in the relative speed, however, it is not, by itself, a clear indicator of the presence or absence of a jammer. For that reason, the speed of the receiver is, also, used. The equality between the relative speed () and the speed , while changes, indicates the absence of a jammer, since the speed of the jammer is equal to zero and the speed of the receiver is in fact the relative speed. On the contrary, a difference between and indicates the presence of a jammer that follows the receiver.
On the contrary, if no alteration of the relative speed is observed while the relative speed value is not equal to the speed value, a possible presence of a jammer is registered. This could occur in a situation where the target vehicle would reduce its speed due to an obstacle. Following our assumption, the jammer would, also, decrease its travelling speed, thus keeping the relative speed unchanged but also different from the travelling speed of the receiver. Contrary to the previous, if no alteration in the relative speed value is observed (for the previous and the next measurement), while having , we conclude that a jammer is not following the receiver.
Having examined the case where the observed relative speed value is not equal to zero, we proceed to the opposite case. With , a simplistic form of the proposed algorithm (VRS algorithm) is presented, indicating the existence of a jammer that has reached its target and follows it closely with the same speed. A real-life environment, however, is more complicated. If the travelling speed of the receiver is not equal to zero, while , a jammer has reached the receiver and follows it while disrupting the communications. On the contrary, if the travelling speed is zero (while ), there might be a jammer present that has stopped moving (following the behavior of the target). In that case, we have to examine the previous observation for equality between relative speed and travelling speed as well as the trigger value to determine the situation.
The variables and represent an array of estimated relative speed values and real travelling speed values of the receiver, respectively, M is the number of the available observations upon which the algorithm operates, vrs is an array used to store the classification result (A for attack or NA for not attack) of the current observation, and trigger is a binary variable which indicates the presence of a jammer (value is equal to 1) or its absence (value is equal to 0). The NA and A values are two extreme and distinct values able to differentiate the attack from the no attack cases and guide the classification process.
4.3. Supervised Learning Algorithms
The supervised learning methods that are used in this work are KNN [35] and random forests [36]. Their choice does not affect the efficiency of our algorithm as our proposed feature is not constrained by the type of the supervised learning algorithm that is used. The VRS (Algorithm 1) generates the new metric which is used as an extrafeature for classification.
Both supervised learning techniques are very popular, with the KNN being robust against noisy training data like the ones obtained from a real-life urban environment and random forests being one of the most accurate algorithms, due to the fact that it reduces the chance of overfitting (by averaging several trees, there is a significantly lower chance of overfitting). As it is previously stated, our detection scheme is currently based on offline training that leverages the use of a dataset of collected measurements in order to train the classifier.
5. Simulation Setup
Figures 3(a)–3(c) illustrate the behavior of the jammer by plotting how SINR varies in time for each of the three scenarios, namely, interference scenario, smart attack scenario, and constant attack scenario.
(a)
(b)
(c)
5.1. Supervised-Learning Testing Cases
Apart from the scenarios that we use to evaluate the performance of the overall system, we also created a series of test cases that are presented in Table 2. They allow for a deeper exploration of the proposed method depending on the set of observations that are utilized for both training and testing.
These cases only affect how the training and testing is performed, without any further implications in the scenarios. They are created in such a way so as to provide insight about the importance of using the VRS metric for classification under different circumstances [37]. Specifically, it is evaluated for the cases that use or omit the VRS metric as an extrametric for the classification process. For the sake of completeness, the trained prediction model is also tested using data that were collected under a receiver speed of 25 m/s, that is, under a speed different from the 15 m/s that we trained the prediction model. We also conducted additional experiments using data measurements from the 25 m/s receiver speed range used for training. Finally, the data are normalized prior to their use for training and testing. By normalization, we refer to the process of changing the data so as to belong in the 0-1 range. It should be noted that, in all the other cases than those declared, the data are not normalized prior to their use for training and testing.
5.2. VANET Simulation Assumptions
Regarding the details of our simulation setup, the speed of the vehicles involved in the legitimate communication , the initial distance between the jammer and the pair of -, the distance that separates the receiver from the transmitter throughout the course of the simulation as well as the power of all the transmitted signals , and the reference distance , with which the path loss component is estimated, are presented in Table 3.
The power of all transmitted signals is measured in milliwatts (mW) and is converted in the dBm scale prior to using them in the algorithm. Each signal that is transmitted from both the jammer and the transmitter consists of streams that are 500 bits long. In all scenarios, 1000 packets are transmitted from the transmitter to the receiver. Using a time sample of 0.1 sec, we simulate the system for 100 seconds and obtain 1000 measurements.
We used Veins that combines the Simulation of Urban Mobility (SUMO) and the OMNET++/VEINS [38]. SUMO is adopted as our traffic simulator and OMNET++ is used to simulate the wireless communication. Furthermore, the GEMV (a geometry-based efficient propagation model for V2V) [39] tool was integrated into the VEINS network simulator for a more realistic simulation of the PHY layer [32]. For describing the modeled area, GEMV takes the map of a real area as an input and uses the outlines of vehicles, buildings, and foliage. Based on the outlines of the objects, it forms R-trees. R-tree is a tree data structure in which objects in the field are bound by rectangles and are hierarchically structured based on their location in space. Hence, GEMV employs a simple geometry-based small-scale signal variation model and calculates the additional stochastic signal variation and the number of diffracted and reflected rays based on the information about the surrounding objects. Last, to set up and test our classification algorithms for the RF jamming attacks detection on the previously obtained data, we chose to use the programming language R [40]. Part of the Erlangen city (see the evaluation setup in [41]) is used for conducting the simulations.
6. Evaluation
6.1. Detection System Evaluation Setup
To underline the significance of our proposed system, we implement and analyze the performance of our model under the different cases presented previously. In particular, for each supervised learning testing case presented in Table 2, we execute a simulation which lasts for 300 seconds and is equally split in the three jamming scenarios discussed in Section 3.4 so that the first 100 sec represent the smart attack scenario, the next 100 sec represent the interference scenario, and the last 100 sec represent the constant attack scenario. All the above scenarios are independent from each other and are run at consecutive time instants.
To avoid testing with “previously seen data,” thus leading to biased classification results, we have to ensure that the training and testing sets are completely separated. So, prior to presenting the classification results, we have to define the size of the training and testing sets as well as the total number of observations used, so as to make them more interpretable. The overall simulation utilizes a set of 3000 observations equally split into the three attack scenarios examined. To avoid overfitting (overfitting occurs when the classifier tends to memorize the training set and thus generalize poorly when facing previously unseen data), only 30% of the total number of the observations are used for training, while the remaining 70% are used for testing.
Based on the ratio above, the number of the observations in the training set is 941 (that is, 293 observations from the interference scenario, 319 from the smart attack scenario, and 329 from the constant attack scenario), whereas the number of the observations in the testing set is 2059 (that is, 703 observations from the interference scenario, 685 from the smart attack scenario, and 671 from the constant attack scenario), randomly chosen but almost equally split among the three scenarios in both cases.
To present the classification results, the confusion matrix is used, where the rows represent classification output and the columns represent the ground truth. To evaluate the performance of our detection system in the various scenarios previously described, we use the accuracy of the prediction model. Accuracy is a measure that is obtained from the confusion matrix and is equal to the ratio of all the correctly predicted labels over all the predictions. The correctly predicted labels are the labels of the main diagonal of the confusion matrix. As an example of the above-defined confusion matrix for the accuracy calculation of our prediction model for the Same_KNN case compared to the Same_KNN-VRS case, we present the subsequent confusion matrices for the KNN algorithm (see Table 4).
6.2. Same_KNN-VRS vs. Same_KNN and Same_RaFo-VRS vs. Same_RaFo Case Classification Results
Starting from the first case, the accuracy of the prediction model achieved while using the VRS metric as an extrafeature in the classification process is 82.27% for the KNN and 80.04% for the random forest algorithm.
On the contrary, when omitting the VRS metric, we observe not only a drop in the classification accuracy but also a high confusion between interference and jamming cases. The accuracy of the prediction model is now equal to 79.16% and 76.54% for the KNN and the random forest algorithms, respectively, so the impact of the VRS metric is evident. Apart from the fact that it increases the success rate of the classification (compared to the cases where the VRS metric is omitted) it ensures, almost perfectly, the differentiation between the cases of intentional and unintentional jamming (see Table 5).
6.3. Different_KNN-VRS vs. Different_KNN and Different_RaFo-VRS vs. Different_RaFo Case Classification Results
As stated previously, these cases examine the situation in which training and testing are based on observations that were collected under different speeds. The accuracy achieved while using the VRS metric as an extrafeature in the classification process is equal to 66.97% for KNN and 69.84% for random forest, respectively.
On the contrary, when the VRS metric is not used, the accuracy of the prediction model is reduced to 56% for the KNN and to 55.37% for the random forest algorithm. Figures 4 and 5 provide insight to the results for the random forest, respectively.
The color of the figures indicates the class in which each observation is predicted to belong to. The smart attack scenario is represented by the red and lasts for the first 100 seconds, the interference scenario is represented by the black lasts for the time interval 100–200 seconds, and the constant attack scenario is represented by the green color and lasts for the time time interval between 200 and 300 seconds(as described in Section 5.2). In Figure 4, we explain in more detail the detection process for each scenario:(1)The appearance of black or green colors in the smart attack scenario (0–100 seconds) indicates the misclassification of this scenario with the interference and the constant attack scenarios, respectively. On the contrary, the points with red color indicate a correct detection of this attack scenario.(2)The points with green or red colors in the interference scenario (100–200 seconds) indicate the misclassification of this scenario with the constant attack scenario and the smart attack scenario, respectively. The black color indicates a correctly detected of the interference scenario.(3)Lastly, for the constant attack scenario during the time interval between (200–300 seconds), the presence of points with black or red colors indicates the misclassification with the interference and the smart attack scenarios, respectively. On the contrary, the appearance of the green color indicates a proper detection of the sonstant attack scenario.
Based on the classification results presented above, we can reach an important conclusion. When testing the prediction model with observations from a different speed—compared to the one used in training—we observe an overall reduction in accuracy. Nevertheless, the use of the VRS metric significantly increases prediction accuracy (in both supervised algorithms examined in this paper), while also achieving a clear separation between interference and jamming.
6.4. Norm_KNN-VRS vs. Norm_KNN and Norm_RaFo-VRS vs. Norm_RaFo Case Classification Results
In these two cases, we try to determine whether normalizing the data prior to using them in training and testing affects the classification results, with and without the use of the VRS metric. The accuracy achieved while using the VRS metric is equal to 81.25% for the KNN algorithm and 80.09% for the random forest, with its omission leading to an accuracy equal to 78.1% and 76.4%, respectively. Once more, the use of the VRS metric in the classification process leads to a upturn in the accuracy of the prediction model. In addition to that, if we compare the previous classification results of the Same_KNN and Same_RaFo cases with the respective ones that derive when no normalization is applied to the data prior to their use, we observe that there is no significant increase in accuracy results. Thus, we conclude that a normalization of the measurements is not necessary. It should be noted that in all the previous and the next presented classification results, the data are not normalized prior to their use for training and testing.
6.5. Same_KNN-VRS_25 m/s vs. Same_KNN_25 m/s and Same_RaFo-VRS_25 m/s vs. Same_RaFo_25 m/s Case Classification Results
As already stated, our RF jamming attack detection system is based on offline training, using a dataset of measurements collected under a speed of 15 m/s so as to train the classifier prior to its use for testing. For the sake of completeness, we examine the Same_KNN-VRS and Same_RaFo-VRS and Same_KNN and Same_RaFo cases presented previously using the data measurements from a higher speed at about 25 m/s speed range for training.
For the Same_KNN-VRS_25 m/s and Same_RaFo-VRS_25 m/s cases, the accuracy of the prediction model achieved is equal to 94.46% for the KNN and 94.61% for the random forest algorithm. For the Same_KNN_25 m/s and Same_RaFo_25 m/s cases, on the contrary, the calculated accuracy is equal to 88.68% for the KNN and 89.22% for the random forest algorithm, respectively.
From the classification results presented above, an important observation can be made. There is an increase in classification accuracy when the training is done using data from a higher speed. The higher classification accuracy comes from the fact that the increase in speed adversely influences the effects of jamming. More concretely, in the constant attack scenario, the jammer overtakes the sender-receiver pair faster, in the interference scenario, the sender-receiver pair remains in the jamming area for a shorter period of time, and in the smart attack scenario, the jammer reaches its target at a higher speed, thus the gradual effect of the jamming observed at lower speeds is greatly reduced. All the above lead to a significant increase in the quality of the measurements obtained, hence leading to higher classification accuracy as well as to better distinction between the different types of jammers affecting the communication, as seen in Figure 6, for the KNN algorithm.
We also investigate more thoroughly the effect of the relative speed metric in the detection probability of a RF jamming attack in a multiclass environment with three classes (class of reactive jamming attack, class of continuous jamming attack, and class of interference). In Figure 7, we present the detection probabilities of the proposed model using the KNN algorithm for a range of relative speed . We observe that, in the medium range of values, we achieve a perfect RF jamming detection result. This result is attributed to the specific characteristics of each type RF jamming attack. Specifically, the continuous jammer transmits continuously deteriorating the wireless communication between the transmitter-receiver. On the contrary, the reactive jammer starts its activity only when it retreats to a safe position (close to the receiver). So, for a small range of values, both types of RF jamming attackers (reactive and continuous) have started their attack leading to several misclassification errors between the two corresponding classes. Finally, at higher values over , we have some misclassification errors between the classes of reactive jamming and interference because the relative speed value is approximately equal to the speed of the receiver () when there is no attacker in the area but only a static RSU that interferes the wireless communication between the transmitter-receiver.
In Table 6, we summarize the classification accuracy, exploiting the usage of the proposed VRS metric as an extra feature, achieved while training with measurements from a speed of 15 m/s and a speed of 25 m/s, respectively.
6.6. Result Summary and Comparison with State of the Art
Figure 8 summarizes classification accuracy percentages that are presented above. These are achieved by both the KNN and the random forest algorithms when based only on the features previously used in the literature for jamming attack detection [26], compared to the proposed approaches, KNN-VRS and RaFo-VRS, that use the VRS metric. The VRS metric increases the accuracy of the classifier and ensures almost perfect differentiation between cases of intentional and unintentional jamming. When using the VRS metric while testing with data from the same speed, there is an increase up to about in the classification accuracy. When testing with data of a different speed, the increase in accuracy is even greater up to about .
We also compare the accuracy of the proposed scheme versus recent state-of-the-art work. We compare RF jamming detection methods with the same complexity and without using extrahardware (e.g., multiple antennas at the receiver [42]). For collecting the used jamming detection metrics, we assume only a completely passive scheme that is based on RF communication between the transmitter-receiver under the presence of a jammer in the area.
As we explained earlier, the authors in [24] proposed a federated learning-based on-device jamming attack detection security architecture for FANET. In order to compare our proposed RF jamming detection method with this method (Federated-Nischat-2019), we preprocess the simulated datasets to derive two unbalanced subdatasets. The first subdataset contains a higher percentage of nonjamming instances () and a lower percentage of jamming instances (). We show in Figure 9 that the method (Federated-Nischat-2019) achieves an accuracy of under the ns-3 simulated FANET dataset. This performance is better only for the cases where VRS metric is not used.
We also compare the KNN and the RaFo algorithm for reactive and constant jamming attack detection using the cross-layer combination of metrics proposed by Feng et al. named (feng2018-KNN) and (feng2018-RaFo). We observe in Figure 10 that when the receiver moves at low speeds of , we have the same accuracy. When the receiver increases its speed and it moves with a speed of , our proposed jamming detection scheme using the VRS metric achieves a much better accuracy (an increase of about ) than the competing methods.
Finally, we compare with the work of Lyamin et al. [21], where the authors use historical observation of events in the V2V channel for the jamming detection. The method is evaluated for two jamming models: random and ON-OFF jamming. To represent random jamming in our model, the reactive jammer transmits its jamming signal randomly and independently with a probability when it is triggered. When comparing the jamming detection results of the method [21] with the proposed jamming detection method for a random jammer with , we have a probability of attack detection (true positive rate) at about for the random reactive jammer, while the method in [21] achieves a probability of attack detection at about for the same type of jamming. Additionally, a priori knowledge about a platoon is employed for this method to achieve better detection results. Only when the number of receivers increases to 20 in the form of a platoon of vehicles (also increasing the received observations for the training phase), the method in [21] manages to reach the probability of attack detection that we achieve using our proposed jamming detection method with a single receiver.
From this set of comparative results, the effect of the proposed VRS metric in RF jamming classification is clear. Especially, when the receiver increases its speed to a speed of , the accuracy of the proposed method increases by over . This performance is much higher than the other corresponding methods in the literature.
7. Conclusions
In this paper, we presented a method for detecting a specific type of DoS attack, namely, RF jamming, based on a cross-layer set of features and supervised machine learning. We introduced a novel metric from the application layer, namely, the variations of the relative speed between the jammer and the target. The relative speed is passively estimated from the combined value of the desired and the jamming signal at the target vehicle combined with metrics from the network and physical layer. To evaluate the significance of the proposed metric and its estimation algorithm, we implemented three different scenarios: two with a jammer and one with interference only.
With our work, we introduced a proactive approach against potential RF jamming attacks which is able to differentiate interference from malicious RF jamming. Additionally, it is able to distinguish the unique characteristics of each attack, especially when the offline training is conducted with a higher speed than . Through our evaluation results, we were able to highlight the vital role of the relative speed and its variations, in addition to other metrics obtained from the physical layer and in jamming detection and unintentional jamming cases differentiation, as well as in the overall increase in the prediction accuracy.
As part of our future work, we plan to investigate the efficiency of our idea in complex vehicular networks with a large number of communicating nodes and several attackers. The target of this classification process will be the characterization of the behavior of a node as malicious or as regular node, mainly using the proposed VRS metric. The classification results can be collected and managed from a Trusted Central Authority (TCA) in an area with V2X communication. Having this information, the TCA could reroute vehicles towards more jamming friendly areas.
Data Availability
The data presented in this study are available from the corresponding author upon request. The data are not publicly available due to privacy reasons.
Disclosure
A preliminary version of the article appears on arxiv at https://arxiv.org/abs/1812.11886.
Conflicts of Interest
All authors declare no conflicts of interest.