Abstract
Based on the function and performance testing requirements of Internet of Vehicles, a design scheme for automatic simulation test and verification platform for the Internet of Vehicles based on cellular networks and real network in-loop is proposed. First, the overall structure of the simulation test platform and the specific function of each component are presented. The main and functional components of the real cellular network simulation subsystem are then discussed. Finally, two typical application test scenarios are designed and presented; namely, vehicle remote-startup delay test based on a cloud platform and Internet of Vehicles local performance test of different frequency cell switching in a dynamic scenario. Experimental results verify that the simulation test platform can meet the test verification, performance evaluation, and troubleshooting requirements of the Internet of Vehicles function. It is important to improve the application experience of intelligent connected vehicles and the Internet of Vehicles.
1. Introduction
Internet of Vehicles (IoV) is a technical field where communication technology and intelligent vehicles are deeply integrated. The functional experience of intelligent connected vehicles is being rapidly improved and expanded with the development and application of communication technology [1], ranging from vehicle-mounted bluetooth to end-to-end interconnection with the outside world, to cloud service, eCall emergency rescue call, C-V2X direct communication, and other applications. In particular, the WAN interconnection based on cellular networks makes the intelligent connected vehicles become an important Internet intelligent terminal in daily life [2]. At the same time, the contradiction between the automotive industry, which has higher safety level and stability requirements, and the communication industry, which requires fast development and rapid technology updates and iteration, has gradually become prominent [3]. The focus of the major automotive design manufacturers and research institutes is now concentrated on how to detect, inspect, troubleshoot, and find issues with the function and performance of intelligent connected vehicles. On-board cellular wireless communication function links longer, involve communication chips, communication module, vehicle-mounted network controller, automobile and electronic architecture, RF antenna module, core network operator communication base stations, operators, cloud service providers, and mobile phone client APP. It faces troubleshooting problems and low performance test coverage problems caused by the lack of test equipment species [4]. Enhancing the stability of telematics performance is much more important in times where fifth-generation mobile communication technologies are applied in the automotive industry [5].
Based on the above-mentioned background, it will be challenging to provide customers with high-quality intelligent-networked vehicles and ensure that they can connect to all mobile carriers globally. This issue can be solved by testing intelligent connected devices under real network conditions. Compatibility issues between intelligent connected cars and networks can be found in advance to a large extent after thorough network testing, and boundary determination and traceability can be carried out beforehand to ensure that the vehicles can provide customers with the ultimate user experience after delivery. It has become critically important to create a software development and test platform for 5G only and 4G-5G coexisted vehicle networking in light of the commercial and widespread use of 5G [6]. The establishment of intelligent-networked vehicles platform, Internet of Things, intelligent transportation network, smart power grid, and smart city has become a necessity to explore the creative development mode of intelligent-networked vehicles with multifield linkage. Obviously, the intelligent-networked vehicles have become a hot spot of competition in the automotive industry.
In light of the aforementioned context, various domestic and foreign research and development institutions have developed a range of solutions for simulation test verification, hardware-in-the-loop test system, and Internet of Vehicles testing. He presented the noninvasive Internet of Vehicles APP test automation systems [7]. They achieved input simulation by touchscreen clicks. In addition, image acquisition, processing, and character recognition were realized. In the design of the mobile terminal, the cloud, external interactive simulation environment on the car used the combination of image processing and optical character recognition (OCR) technology to realize the vehicle Internet of Vehicles test validation. The design and development of an automatic driving vehicle in a ring virtual simulation test platform for the vehicle’s intelligent driving by Wang et al. [8] involved the use of a turning drum, cameras, and millimeter wave radar target signal simulation, as well as the integration of a traffic scene virtual simulation test software and a simulation test platform for the intelligent vehicle driving performance test lab. The design concept of vehicle in-loop simulation test provides a reference for vehicle network test. Rod and Schwartz design and develop solutions that can support simultaneous signaling and nonsignaling tests. Based on the nonsignaling test, mobile communication base station functions are simulated to realize the instrument, measure the signaling connection, and support the mobile terminal connection to the network to realize data interaction. At the same time, different levels of monitoring and diagnosis are achieved. The mobile communication terminal is tested and verified in a laboratory setting. In order to test the direct communication technology used in the Internet of Vehicles, Peng proposed a test method based on channel simulation of the vehicle network communication system in-loop [9], researching the LTE-V2X channel situation of direct communication in the real environment. They build the channel simulation function of hardware in-loop test platform directly connected to Internet of Vehicles communication including vehicle-to-vehicle (V2V) [10] and vehicle-to-infrastructure (V2I) [11] function simulation and verification tests [12–14].
Their work has carried out unit tests on functions such as the Internet of Vehicles, vehicle driving, and mobile terminal connection performance, but these functions will be integrated into a system on the vehicle. In the integrated system, a function may be affected by other functions, and the unit test cannot detect faults introduced by other functions. To ensure the stability of the integrated system, we need a system for integration testing. Hence, we have reconstructed these test systems and designed an integrated test system. We proposed a simulation test platform based on real network in-loop. It primarily focuses on the overall architecture of the simulation test platform and the composition and functional principles of each subsystem. The simulation test platform is verified by two typical application cases: vehicle remote-start control delay test and different-frequency cell switching and camping performance test for the vehicle network.
The main contributions of this study are summarized as follows: (1)To drive the vehicle in the real cellular network simulation environment, we propose a real network in-loop simulation test platform by mounting the vehicle longitudinal motion in-loop simulation subsystem of passive free travel on the drum, which can gather the vehicle wheel speed information and transmit it to the virtual traffic scene of test automation management system to realize the actual condition of the vehicle longitudinal motion fusion(2)To simulate the real cellular network environment in a laboratory setting, we designed a real network simulation communication system including a global navigation satellite system (GNSS) simulator, baseband processing unit (BBU), remote-radio frequency unit (RRU), core network, Internet original entrusted manufacturer (OEM) cloud, and firewall(3)We established an automated test auxiliary system to achieve vehicle longitudinal motion in-loop simulation and mobile APP automated test
The remainder of this paper is organized as follows. In Section 2, we show the overall structure of the simulation test platform. The real network simulation communication subsystem and the automated test auxiliary subsystem are then presented in Sections 3 and 4, respectively. In Section 5, platform integration and test process design are described. Two typical cases by our system are discussed in Section 6, and conclusions are finally drawn in Section 7.
2. The Overall Structure of the Simulation Test Platform
The simulation test platform can build the entire chain for IoV development and validation [15]. The platform mainly includes: laboratory mobile cellular network infrastructure components, interaction between inner and outer net interface, complex cellular network communication environment simulator, vehicle longitudinal motion in-loop simulation, mobile APP automation test system, vehicles head unit-automated test system, and platform automation test management software of seven parts. The system block diagram is shown in Figure 1.
Among these systems, laboratory mobile cellular basic components are used to implement a cellular wireless communication network. Through the cellular network, complex communication environmental simulation devices connected to the on-board wireless terminal realize data transmission and business services. At the same time, the laboratory proprietary wireless communication server is directly connected to the outer network through the interaction between inner and outer net interface. It is necessary to connect a channel to a telematics service provider (TSP) to enable access to the vehicle being tested by a remote-control terminal such as a mobile phone. The test management software is connected with each part of the test platform to achieve coordinated control of subsystems and automatic test execution [16, 17].
Based on the description and connection relationship of the subfunctional components, a real network in-loop simulation test platform for the Internet of Vehicles is designed. During the testing process, the vehicle to be measured is fixed to the vehicle longitudinal motion in-loop simulation subsystem of passive free travel on the drum. The movement in-loop simulation subsystem then acquires vehicle wheel speed information and provides feedback to the virtual traffic scene of the test automation management system to achieve the actual condition of the vehicle longitudinal motion fusion, driving the vehicle in real network and real cellular signal simulation environment. The automated test management system enables the online real-time control of the simulation of a complex communication environment in a cellular network, therefore enabling the dynamic and arbitrary simulation of wireless signal intensity transformation, cell switching, and different frequency switching. By extending the interactive interfaces of the internal network with the outside network, and also the automatic test system of the mobile phone, this platform can query and remotely control the remote-control terminal to the vehicle, which can be realized in the laboratory environment. At the same time, the test and verification of the communication function and performance of the IoV terminal can be completed by monitoring the data flow of the communication link. (https://source.android.com/devices/automotive?hl=zh-cn.) (https://www.jdpower.com/business/pressreleases/).
3. Real Network Simulation Communication Subsystem
3.1. Laboratory Mobile Cellular Network Basic Components
The basic components of the laboratory mobile cellular network are the functional core of the private network construction. The main functional modules include the core network, BBU, RRU, and connection auxiliary system. The 5G core network, which includes both the 4G and 5G core networks, is a critical component for the testing of 5G vehicles [18, 19]. The block diagram is shown in Figure 2.
The communication core network adopts Nokia 4G-5G non-standalone (NSA) core network scheme, mainly including 4G/LTE grouped core network mobility management entity (MME), serving gateway/packet data network (SGW/PGW), home subscriber server (HSS), policy and charging rules function (PCRF), 5G NSA, and 5G standalone (SA) functions, with a total throughput of 30 Gb/s for upstream and downstream and 256 access point names and data network names. The mobile terminal user registration, certification, mobility, accessibility, and connection management process can be completed by the communication core network. IP addresses for user terminal distribution can also be obtained in light of the user terminal strategy which was taken from the policy and charging rules function (PCRF) quality of service (QoS) control strategy. It can also implement user data forwarding and routing, a permanent storage and record signing of the 4G area within the user data, such as location information, service data, and account management information, and can provide real-time query and modification of user location information. It can also realize all types of business operations, including location update, call processing, authentication, and supplemental services, to complete user mobility management in the mobile communication network.
BBU and RRU are the core units of the communication base station to realize the function reproduction of the operator’s communication base station and the interaction with the radio frequency connection of the vehicle’s mobile terminal. In view of the current situation of China Mobile communication and the configuration details of the tested vehicles, two cellular mobile communication systems, 5G NSA and LTE FDD&TDD, E-UTRAN, are mainly constructed. According to the actual future demand, more base station environments of communication systems can be expanded to improve the utilization rate and equipment coverage.
Reappearing the cellular communication network environment in the laboratory environment can assist in realizing the transition from solely relying on field tests to laboratory tests, therefore realizing labor and time savings as well as compression cycle efficiency and an expansion of the test range in terms of test manpower, test time, test cycle, and test execution efficiency. At the same time, according to the export vehicle model, wireless communication tests can hardly realize the domestic research and development, production, and test problem. The construction of global frequency coverage, multiple standards, and multiservice end-to-end solutions will enable OEMs to conduct numerous tests for global business operations, while also saving time and travel costs, allowing the necessary work to be completed in a laboratory setting, which will reduce costs and speed up test cycles and product development lead times.
3.2. Inner and Outer Network Interaction Interface Design
The laboratory cellular network communication private network server is connected with TSP remote-service provider platform through a customized firewall, and the designated IP port of the private network is configured and mapped with the port opened by TSP, so that the on-board unit (OBU) can connect to TSP management platform through private network service. The design of the structure also supports the development and testing of TSP security, allowing for the realization of all vehicle remote-communication control functions based on TSP cloud service [20]. Additionally, as the demand for SoC vehicle operation capability increases, the majority of this capability will be deployed on the network edge; it is called mobile edge computing (MEC). This platform can integrate and meet this requirement [21–27].
3.3. Simulation of Complex Communication Environment in Cellular Network
In real traffic and real cellular network communication environments, the distance between the vehicle terminal and each real communication base station or edge RSU gradually changes as the vehicle moves, resulting in a variety of complex communication conditions [28, 29], such as strong and weak RF signals, different frequency, same RF frequency of multiple base stations covered at the same time, no communication signal covered or intermittent communication signal, the base station signal changes during vehicle movement, and the system changes. The unstable performance and abnormal function of vehicle communication are often caused by the complex cellular communication environment.
In this simulation test platform, four 5G-AAU units (5G-NR N78 RRU2 and 5G-NR N1 RRU2) are configured to output 16-channel RF channel signals and 4G-RRU (4G-FDD Band 1 RRU2 and 4G-FDD Band 3 RRU2). Four sets of output 16-channel RF channel signals, that is, the system supports analog 32 channel communication antennas. A simulation device of complex communication environment in a cellular network is designed based on the basic configuration. It is composed of a 32-channel program-controlled attenuation system and a multiplexer, to achieve the independent control and adjustment of 32 channels in the cellular network. The bidirectional-programmed attenuation system has 32 input ports and 32 output ports and is composed of 32 programmed attenuation units. Each unit has an attenuation range of 0 to -127 dB and supports a frequency band of 700-6000 MHz. Each programmed attenuation unit can independently control signal attenuation and adjust the signal energy value of different channels. Testing the signal attenuation of various channels simulates the test scenarios of moving and switching. The number of channels here can be expanded according to the number of base station configurations, such as the 3G RRU base station connected by the dotted line in Figure 3.
4. Automated Test Auxiliary Subsystem
4.1. Vehicle Longitudinal Motion In-Loop Simulation
Vehicle longitudinal motion in-loop simulation system is used to support vehicles running in a laboratory environment, realize the condition of various vehicle speeds and course angle function tests, and collect and record the real-time vehicle distance. Mapped to the simulation environment, it synchronizes the simulation environment state of the vehicle and the actual vehicle running state. The real GNSS RF positioning information is simultaneously output to the Internet of Vehicles module of the tested vehicle through GNSS simulator in order to simulate the vehicle navigation positioning signal [30]. Additionally, with the simulated GNSS signal, the high-precision map function in a vehicle can be tested and evaluated [31]. The block diagram of the vehicle longitudinal motion in-loop simulation system is shown in Figure 4.
A highly accurate rotary encoder is mounted on the vehicle’s drum, and the real state of the vehicle is fed back and recorded through transmission ratio conversion. The real-time collected pulse distance signal is transformed into the current longitudinal velocity and acceleration of the tested vehicle by differentiating. In order to provide vehicle navigation and positioning service, the real-time status of the vehicle is input into the GNSS positioning signal simulator (R&S SMBV100b), and the GNSS-RF positioning signal is output to the shielded room of the tested vehicle [32, 33]. As a result, the GNSS signal can also be played back and represented in this platform, and related simulation modeling will be set up [34, 35].
4.2. Mobile APP Automation Test System
One of the main functions of vehicle remote interconnection is information exchange with a mobile application, enabling useful features like real-time vehicle location query, vehicle remote start, vehicle remote unlock, and vehicle state diagnosis. In the actual test process, the mobile APP of synchronous operation control is required to complete the test and verify the vehicle network connection function by detecting the state and action feedback of the tested vehicle. A snapshot of the test system software is shown in Figure 5.
The mechanical arm, screen touch head, industrial camera, and control program are the main components of the automatic testing system for mobile phones. Additionally, the bus signal (e.g., CAN, LIN, FlexRay, and Ethernet) for vehicle internal communication should also be captured to close the test loop. The test process is explained in the following. After receiving a control command from the distributed mobile-automated test execution equipment, the system awakes and locates the icon through the industrial camera. Then it controls the mechanical arm to click the icon or do a sliding movement on the phone’s screen [36]. Once the command is sent from the phone side, the signal goes through the base station network, then through the core network, and returns back to the vehicle. The value of the relevant bus signal will change after the telematics control unit (TCU) receives the command. When a signal is checked through the BUS, the function and performance can be evaluated. Program-based automatic control for mobile terminal APP and functions is realized to improve the efficiency of the test system and implement unsupervised automated test execution.
4.3. Vehicle Head Unit Automation Test System
The automatic test system is used to monitor and evaluate the state of the vehicle head unit display image, alert symbol, or sound feedback information, and the feedback judgment link of the closed-loop test system is used to carry out an automatic test of the vehicle’s Internet function. The main function modules include vehicle instrument screen information and image acquisition and recognition, audio acquisition and warning sound recognition and classification, and vehicle OBU touch control based on ADB debugging interface. By setting up the phone call OBU and vehicle head unit function in this way, the call function quality and performance will be assessed independently as the controlled signal strength changes [37].
5. Platform Integration and Test Process Design
The open-automated test management platform is developed based on the overall structure of the simulation test platform, which can implement customized scripts based on test scenarios, embed process control and state feedback monitoring of subfunction modules, and implement the real network in-loop simulation test for the Internet of Vehicles. An extremely good point, a midpoint, a bad point, and an extremely bad point are all covered in an automated testing platform based on network environment. Combining the requirements for testing static and moving scenes and developing customized test scripts can realize and trigger network events such as reelection, switch, redirect, OOS, CCO, and NC0. Business traversal through automation test and stress test can be achieved to find and reproduce problems. The test platform primarily consists of a number of functional modules, such as test equipment management, scene design, use case planning, test data resource management, execution monitoring, and output reporting. It can efficiently manage test equipment and resources, and it supports various log output and graphical display functions. The automated test platform is shown as Figure 6.
At the same time, the automated testing software has the ability of graphical speech modeling of a complex wireless communication environment for cellular networks. Set the number of base stations, an algorithm for constructing virtual RF communication environment, location, type, signal strength information, and specify the starting longitude and latitude of the vehicle being tested, as well as the movement track of the actual measured vehicle as it is launched on the drum, and longitudinal motion in-loop simulation equipment will synchronize the movement distance of the vehicles to the simulated RF environment created by automated testing software. The automated testing software controls the appropriate communication channel attenuator based on the vehicle’s position in the virtual communication environment and the base station’s distance from it, thus allowing for the simulation of wireless radio frequency communication environments and the implementation of real network in-loop Internet of Vehicles communication performance testing in the process of dynamic movement. With this platform, many functions in related IoV can be developed and validated, such as digital car key development and testing [38] and cyber security simulation testing, as well as OTA software upgrade and diagnostics technology development and testing [39].
6. Typical Cases
6.1. Vehicle Remote-Start Control Delay Time Test
Aiming at the problem of occasional failure of the connectivity function of the mobile APP, a typical test application example of a vehicle remote-start control delay test was designed to verify the cloud service-related testing capability of the simulation test platform. The information transmission link in the process of vehicle remote startup is depicted in Figure 7.
The enterprise TSP server will receive the pertinent request from the APP when it sends the engine ignition request message over the external operator base station network. The TSP server sends the received request message from the corresponding interface to the core network of the laboratory private network and sends it to the shielding room through BBU and RRU. After the corresponding request data information is sent to the terminal Internet of Vehicles communication service process, the Internet of Vehicles communication terminal chip sends the corresponding data through the antenna following the communication module encapsulation to the Internet of Vehicles terminal software. After attestation decryption process, the corresponding request information is sent through the bus to the engine controller. The engine controller responds to the request signal to start the vehicle and broadcasts the corresponding state on the bus in the car. At the same time, the Internet of Vehicles terminal feeds back the status of the vehicle to the APP on the mobile phone to complete the entire communication execution of the remote-start control of the vehicle.
In this real network in-loop simulation test system, the simulation platform can be on the TSP side, core network side, and inside to monitor the bus according to the timestamp, the status of transmission, and execution of signals. Hence, the development and optimization of the vehicle controller embedded software can take advantage of the efficient test environment. The test environment is provided by the communication base station terminal. When it receives the request signal for the vehicle networking process to complete the transfer from the vehicle bus request, we can compute the signal time delay.
Here, the delay time standard is listed in 5G NSA environment: (1)From IoV module to base station, while 32-byte data are sent, the average delay time between request and reply should be less or equal to 15 ms, and the successful rate should be greater or equal to 98 presents (signal strength greater than -75 dBb, 5G NSA network)(2)From IoV module to core network, while 32-byte data are sent, the average delay time between request and reply should be less or equal to 22 ms, and the successful rate should be greater or equal to 98 presents (signal strength greater than -75 dBb, 5G NSA network). The delay time data from IoV module to core network is collected and analyzed as shown in Figure 8
From the data, the average delay time from terminal to core network is 16.18 ms, and meanwhile all the data are in range. This result shows that the situation is good when the test is executed. The individual chart also shows that the control chart of upper control limit is 21.75 ms while the lower control limit is 10.60 ms; it is a good performance from delay time perspective. Once the delay time data is out of range, action should be taken to improve the system performance.
7. Performance Test of Different Frequency Cell Switching Internet of Vehicles Network Camping
With the aim of addressing the issue of unstable communication quality caused by the change of cellular network coverage in the dynamic driving process of vehicles, a typical test application example, performance test of different frequency cell switching Internet of Vehicles network camping, was designed to demonstrate the simulation test platform’s capacity for simulating complex cellular network communication environments. The principle of local performance test of different frequency cell switching Internet of Vehicles network camping is shown in Figure 9.
Two virtual communication base stations, 4G Band 7 and 4G Band 20, were set and the coverage of the base station spread outward from strong to weak with the base station located at the center. According to the signal strength among the two base stations, we define A, B, and C different areas: the A area has 4G Band 7 signal only, while the C area has 4G Band 20 signal only, and the B area has stronger 4G Band 7 signal than 4G Band 20 signal. The vehicle track is designed to be triggered from area A, and the log data of vehicle terminals are captured. As a result, the exact TCU camping network situation can be known.
In this case, the correct logic of the IoV terminal should give priority to the Band 7 in area A communication network when multiple networks coexist. The IoV terminal should switch network signal while driving from B to C to ensure optimal communication performance.
This scenario could also simulate the vehicle track from 5G n78 driving to 5G n1. The operational status of the IoV module would be monitored at that time, and the test management platform would check if a disconnection occurred during the track. Once an abnormal situation occurs, the logs of both the terminal and the communication module will be captured and analyzed by Qualcomm QXDM tool or wire shark.
Figure 10 shows that when 4G Band 1 signal is stable and located at a good point (>-75 dBm), the communication breaks off and the connection is lost for 10 seconds (c.f. Figure 10, “Detach” indicator). This situation happened every 5 minutes. After several recurrences, the communication module will enter to abnormal mode with a specific error code showing. After analyzing these phenomena, a software bug in MQTT library was identified. After fixing the bug, the situation changed, and these phenomena disappeared. For the communication module issue, the chip setting was incorrect and caused the chip to go into an abnormal status. Both of the two issues are common ones that give customers the impression that their connection is abnormal, cause lag, lose connection, or even terminate a voice call when, in fact, their mobile phone’s signal is strong in the same network environment. The user experience is the most important for customers, and this influences the reputation of the vehicle brand and then impacts sales volume.
The performance of the vehicle network communication is dynamically verified by building an ideal and controllable complex cellular network communication environment. This creates an ideal testing and verification environment for the program optimization and iteration of the vehicle communication terminals and communication modules.
8. Conclusions
In this paper, a design plan for a real network in-loop simulation test platform is presented, which integrates all links from the end, tube, loud, side, and user in Internet of Vehicles link. The paper focuses on the core components of the cellular communication private network in the laboratory and designs and develops related auxiliary equipment for simulating the complex communication environment of the cellular network and conducting vehicle-in-loop automation testing. It also integrates and builds a real network-in-loop simulation test platform for vehicle-in-loop function and performance test. Two typical application cases of different types are used to confirm the reliability and scalability of the platform. In the next step, automatic test scenario library will be designed and developed based on functional specifications and user fault feedback. This will be followed by the establishment of a complete acceptance standard scheme for Internet of Vehicles R&D verification test. In addition, professional testing capabilities toward the promising 6G technology such as digital keys, OTA, and MAPP, and IoV information security will be continuously developed, integrated, and realized [40].
Data Availability
The data used to support the findings of this study have not been made available by the legislation in Wuhan Lotus Cars Co., Ltd.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This work was supported by the National Key Research and Development Program of China (2020YFB1807500), the Science and Technology Innovation 2025 Key Project (Ningbo Key Science and Technology Research Project) (Grant No. 2022Z017), the National Natural Science Foundation of China (62072360, 62001357, 62172438, and 61901367), the Key Research and Development Plan of Shaanxi Province (2021ZDLGY02-09 and 2020JQ-844), the Natural Science Foundation of Guangdong Province of China (2022A1515010988), the Key Project on Artificial Intelligence of Xi’an Science and Technology Plan (2022JH-RGZN-0003), the Xi’an Science and Technology Plan (20RGZN0005), and the Xi’an Key Laboratory of Mobile Edge Computing and Security (201805052-ZD3CG36).