Conflict Judgment and Safety Assessment at Unsignalized Intersections Based on Machine Vision

Wang, Yuqiong; Wang, Liming; Zhu, Ruoyu; Xu, Yi; Jiang, Guoxin; Ma, Xiaotian

doi:https://doi.org/10.1155/2023/6465225

Journal of Advanced Transportation

On this page

Abstract Introduction Conclusion Data Availability Conflicts of Interest Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2023 | Article ID 6465225 | https://doi.org/10.1155/2023/6465225

Conflict Judgment and Safety Assessment at Unsignalized Intersections Based on Machine Vision

Yuqiong Wang,¹Liming Wang,¹Ruoyu Zhu,¹Yi Xu,^1,2Guoxin Jiang,¹and Xiaotian Ma³

Academic Editor: Jinjun Tang

Received01 Nov 2022

Revised18 Jan 2023

Accepted30 Jan 2023

Published17 Feb 2023

Abstract

This article aims to explore an effective method for reducing vehicle collisions at unsignalized intersections. First, a monocular-binocular vision switching system is built to enable machine vision-based detection of obstacle vehicles in the left and right front directions. Then, the motion state and trajectory of each obstacle vehicle are predicted, and the intersection points of the trajectories of the obstacle vehicle and the ego vehicle are calculated. On this basis, a cross-conflict judgment model based on trajectories and collision times and a safety assessment model based on safety distance are established. Finally, the conflict judgment and safety assessment for the obstacle vehicles are simulated. The results of the simulation demonstrate that the monocular-binocular vision switching system proposed in this article can achieve a detection accuracy of 95%, a ranging accuracy of 96%, and a cross-conflict detection accuracy of 97%, while ensuring a maximum detection area, which can meet the requirements of traffic safety assurance at unsignalized intersections.

1. Introduction

Traffic conflict can be expressed as the result of interaction between one traffic participant and other traffic participants or traffic facilities during the course of movement in space [1]. In addition to rear-ending collisions, cross-collisions are a common type of collision. In a cross-collision between two vehicles, there is a certain angle between the motion directions of the two vehicles, so the collision is a kind of side collision. Most of the cross-collisions occur at the right front or left front of each vehicle. For the convenience of expression, all collisions involving a collision between the left and right fronts of the involved vehicles are collectively referred to as cross-collisions in this article, while the traffic conflicts before the cross-collisions are referred to as cross-conflicts.

Commonly used sensors for obstacle detection include millimeter wave radar, laser radar, and camera [2]. Millimeter wave radar has excellent real-time performance [3, 4], but its detection accuracy is low. Laser radar has high detection accuracy, but its price is quite high. Compared to other sensors, the camera has the advantages of high detection speed, good real-time performance, and robust adaptability [5, 6]. Obstacle detection based on machine vision uses methods such as machine learning and feature matching to identify the obstacles in a target region based on the region of interest (ROI) partition and obtains the deep information of the obstacles using a vision ranging model [7–10]. Liu et al. [11] proposed the SSD (single shot multibox detection) model based on the precise positioning method based on the Faster R-CNN candidate area proposal and the fast detection method based on YOLO, improving the end-to-end target detection method. Redon et al. [12] proposed the detection and recognition framework of YOLOv2, optimized the detection accuracy, detection speed, and detection category of the network model, and improved the running speed of the feature extraction network and the convergence speed of the model. Zhao et al. [13] proposed an integrated microscopic traffic flow model to describe human-driven vehicle maneuvers under interactions and verified that the model can accurately describe vehicle passing orders of interacting maneuvers, paths, and speeds against empirical data. Zhao et al. [14] proposed a novel method to model the trajectories of vehicles in two-dimensional space and speed, and the descriptive power, plausibility, and accuracy of the proposed model are investigated by comparing the calculated results under several cases, which can be solved from symmetry or analytically. This article proposes a detection method based on machine vision. Three monocular cameras are used to build a monocular-binocular vision switching system based on an artificial potential field for detecting the obstacle vehicles of the ego vehicle. The Kalman filter algorithm is used to locate, track, and predict the positions of the target vehicles. Monocular and binocular ranging models are established to calculate the distance, speed, direction, and other parameters of obstacle vehicles.

An accurate real-time prediction of vehicle trajectories at unsignalized intersections is a crucial part of traffic collision detection and early warning [15–18]. Formosa et al. proposed a centralized digital architecture that predicts traffic conflicts using deep learning. This model detects lane markings and tracks vehicles according to the images captured by a single front-end camera. The data are combined with traffic variables and calculated surrogate safety measures (SSMs) through the centralized digital architecture to develop a series of deep neural network (DNN) models for predicting traffic conflicts [19]. Huang et al. developed an extended driving risk estimation model considering the conflict between vehicles and pedestrians based on an artificial potential field, which projects the vehicles at the intersection into virtual lanes and measures the conflict risk in dynamic risks using the minimum safety distance between conflicting vehicles in the virtual lane [20]. Rocha et al. developed a framework [21] that can utilize the self-organizing mechanism to group traffic conflicts based on the similar characteristics and contributing factors of traffic accidents. The causes and characteristics of different types of conflict events can be revealed based on the grouping results, allowing the traffic management department to formulate strategies to reduce the risk of collision. Qadri et al. captured road traffic information using a handheld digital camera and extracted vehicle information using a video playback and flow recording device. On this basis, they modeled cross-collision using a multiregression model. They analyzed the types of traffic conflicts at the unsignalized intersections of six intersecting road segments from the perspective of time, accident, and travel speed [22]. Zheng et al. [23] validate the Bayesian hierarchical extreme value model that is developed for estimating crashes from traffic conflicts. The study shows that Bayesian hierarchical extreme value models significantly outperform the on-site models in terms of crash estimation accuracy and precision. This article focuses on unsignalized intersections and only considers the obstacle vehicles traveling in the right front and left front directions of the ego vehicle. A model based on trajectory prediction, collision zone, collision time, and safety distance is established based on machine vision detection to enable vehicle detection and safety assessment at the unsignalized intersection. Thus, an early warning can be given to the driver before the accident actually occurs, allowing the driver to take braking measures in case of an emergency to reduce the occurrence of cross-collision accidents and improve the safety of driving.

2. Vehicle Detection and Tracking in Left and Right front Direction Based on the Monocular-Binocular Vision Switching System

The cross-collision detection for vehicles is to determine whether there is a threat in the left and right front directions of the ego vehicle. There are many detection methods based on machine learning at present. The YOLOv5 model launched by Ultralytics in 2020 is implemented in the ecologically mature PyTorch, which has the advantages of small volume, high precision, and fast speed, and the effect of YOLOv5 has been recognized [24–26]. In this article, the detection of the obstacle vehicles in the left and right front directions of the ego vehicle is based on YOLOv5, and a monocular-binocular vision switching system is developed using an improved artificial potential field.

2.1. Vehicle Detection in Left and Right front Direction Based on the Monocular-Binocular Vision Switching System

A monocular vision ranging model and a binocular vision ranging model are established to identify the vehicles in the left and right front directions of the ego vehicle and calculate the distance , travel speed , and driving direction .

Image information acquisition is the process of converting a three-dimensional object into a two-dimensional plane, which can be simplified as a pinhole camera model (see Figure 1). Assuming that the effective focal length of the camera is , the installation height of the camera is , the pitch angle of the camera is , and the origin of the plane coordinate system (the intersection of the image plane and the optical axis of the camera) is usually set to . The intersection of the obstacle ahead and the road plane is , and the coordinates of point in the image plane coordinate system are . The horizontal distance between point and the camera is obtained through monocular camera ranging:

Binocular stereo vision entails using two cameras to capture two images of the same scene from two perspectives, thus obtaining a disparity image containing three-dimensional information about the scene. By processing the image with the stereo correction technique, the ordinate values of the same point on the two imaging planes can be equalized. Then, the distance information of the object can be calculated after the disparity is obtained. It is assumed that the left imaging plane and the right imaging plane are coplanar, the main optical axes are strictly parallel, and are the left and right focal points of the two cameras, respectively, and the focal lengths () of the two cameras are equal. Assuming that point is an arbitrary feature point in the three-dimensional scene, then, its imaging points in the camera coordinate systems on the left and right imaging planes are and , respectively (the ordinate values of the left and right imaging points and are equal). Taking the left camera as the reference, we can translate (the imaging point of point on the imaging plane of the right camera) to the imaging plane of the left camera to obtain ; thus, is the vision disparity between the two cameras. The depth information can be derived according to the similar triangle principle, as shown in equation (2). The principle of binocular stereo vision ranging is shown in Figure 2, where is the baseline length of the binocular camera:

The obstacle vehicles are identified using the images captured by vision sensors, and their speeds are calculated using the interframe difference method. The frame rate of a general camera is greater than 25f/s; the calculated speed can be closer to the real speed. Since the ego vehicle is moving, the relative speed and absolute speed of the obstacle vehicle in the left and right front directions can be calculated based on the travel speed of the ego vehicle. Assuming that the moment when the obstacle vehicle first appears in the camera’s field of view (FOV) is , the distance between the two vehicles at is , the next moment is , the distance between the two vehicles at is , and the interframe interval of the camera is . Then, the time difference between the two moments and the distance difference between the two vehicles can be calculated using the following formulas:

The relative speed of the two vehicles is

The value of should be determined on the premise of ensuring driving safety and high efficiency, and is taken as a time interval. Plugging it into formula (4), we can calculate the relative speed of the two vehicles. The travel speed of the ego vehicle is obtained by the speed sensor. Thus, the travel speed of the obstacle vehicle is

The driving direction angle of the obstacle vehicle relative to the ego vehicle is calculated based on the and coordinates:

2.2. Monocular-Binocular Vision Switching Strategy Based on Improved Artificial Potential Field

Considering the differences between monocular vision and binocular vision in terms of perception accuracy and range accuracy at different distances, we developed a monocular-binocular vision switching system based on the improved artificial potential field. The monocular-binocular switching strategy is formulated in real time to enable rapid and accurate calculation of obstacle vehicles, as well as effective tracking of obstacle vehicles. When cross-conflicts can be detected effectively, driving safety at unsignalized intersections under various driving conditions can be ensured.

All cross-conflicts are assumed to occur within 135° left and 135° right from the longitudinal axis of the ego vehicle, the detection radius of the camera is r ≤ 55°m (r is the effective ranging depth of the camera), and the . Three cameras are mounted on the ego vehicle to ensure the coverage area of the camera FOV and the effective switching between monocular and binocular cameras. In the FOV of each camera, some parts are only covered by one camera, while others are covered by two cameras (overlap region). In the part of FOV covered by a single camera, the environmental perception is based on monocular vision. In the FOV overlap region of two cameras, the advantages of both monocular vision and binocular vision can be combined for more effective detection [27, 28]. The monocular-binocular vision switching strategy is formulated based on the real-time traffic situation at the intersection. The FOVs of the three cameras are shown in Figure 3.

The artificial potential field method is a kind of virtual force method [29–31]. The premise of establishing a repulsion force field is that the ego vehicle will purposefully avoid obstacle vehicles. However, during the actual driving process, the movements of the ego vehicle and the obstacle vehicle are not linked. Therefore, we improved the artificial potential field method. The intersection area of the gravitational field between the ego vehicle and its target point and the gravitational field between each obstacle vehicle and its target point is identified using the principle of an artificial gravitational field. Then, a real-time monocular-binocular vision switching strategy is developed based on the intersection area. Due to the difficulty of obtaining the target points of obstacle vehicles, the maximum distance that the vehicle can travel within 3 seconds in a straight line is used as the maximum length of the gravitational field, which is set to 50 m in this article according to the speed range of vehicles near an intersection.

Considering the requirements in the real driving environment, an overlap region is defined according to the angle parameter (the driving direction angle of the obstacle vehicle relative to the ego vehicle) and the intersection area of the artificial gravitational fields and , and the monocular-binocular vision switching strategy is formulated in real time according to the overlap region. In the single-camera area, the advantages of monocular vision, such as its small computing workload and high detection efficiency, can be exploited to detect obstacle vehicles, and the gravitational field between the ego vehicle and its target point is established in real time. For the obstacle vehicles that appear in the FOV overlap region of two cameras, the artificial gravitational field between each obstacle vehicle and its target point is established, and the overlap region is determined according to the intersection of and . In the FOV overlap region, the distance and speed of the nearest vehicle are calculated, and whether to use binocular vision is decided according to the size of . Gravity is linearly related to the distance between the two vehicles. The greater the distance , the greater the gravity. When the overlap region , it indicates that the obstacle vehicle has begun to threaten the normal travel of the ego vehicle. At this time, binocular vision is used to identify the obstacle vehicle and calculate its distance and speed. The switching process is shown in Figure 4; the process of determining monocular-binocular vision switching strategy is shown in Figure 5.

The specific process is as follows:(1)The three cameras are mounted on the ego vehicle to acquire the environmental information in their own FOVs. The monocular vision technology based on YOLOv5 is used to identify the obstacle vehicles. We calculate the distance from the target vehicle to the camera by using similar triangles. is calculated according to the operating angle and distance parameter . The driving angle of the obstacle vehicle relative to the ego vehicle is calculated according to the and coordinates of the obstacle vehicle relative to the ego vehicle: An orthogonal coordinate system is established, in which the moving direction of the ego vehicle is taken as the positive direction of the X-axis, and the positive direction of the Y-axis is obtained by rotating the X-axis clockwise by 90°. The acute angle between the moving direction of the centroid of the obstacle vehicle and the Y-axis in a time interval is determined according to the target point of the obstacle vehicle, and the gravitational field between the obstacle vehicle and its target point is determined accordingly. The scene is shown in Figure 6.(2)When , monocular vision technology is used to identify and track the obstacle vehicle, as well as calculate its distance and speed.(3)When , the gravitational field is established, and the gravitational field is established according to the driving direction of the obstacle vehicle. When , monocular vision technology is used to track the obstacle vehicle as well as calculate its speed and distance. When , binocular vision technology is used to track the obstacle vehicle as well as calculate its distance and speed.(4)When the angle parameter or deviates out of the original range, real-time monocular-binocular vision switching is needed to enable high-precision tracking.(5)For a new obstacle vehicle or an obstacle vehicle that reappears in the vision system, we repeat the above four steps.

2.3. Vehicle Motion State Estimation Based on Extended Kalman Filter

The position of the monitored vehicle at a certain moment in the future can be obtained based on a reasonable prediction of the trajectory of the vehicle. This provides a basis for tracking the obstacle vehicle [32–36]. Taking the centroid of the vehicle as the origin of coordinates, we can establish a three-dimensional coordinate system, in which the positive direction of the X-axis is the driving direction of the vehicle. The positive direction of the Y-axis is obtained by rotating the positive direction of the X-axis clockwise by 90°, and the positive direction of the Z-axis is the upward direction perpendicular to the XOY plane, as shown in Figure 7.

State and observation variables are determined using the coordinate, coordinate, longitudinal component of the travel speed, and yaw angle of the driving direction of the vehicle. Assuming that the motion state of the vehicle is and the observation state is , we can obtain the position of the obstacle vehicle by predicting the and coordinates of the obstacle vehicle. We can obtain the state equation (8) of vehicle motion and the observation equation (9) of the vehicle motion system by ignoring the system noise and inputting the longitudinal acceleration and yaw rate of the vehicle calculated by the environmental sensing system as specific quantities:where represents the actual sampling time (i.e., the time interval between two samples) and is the observation noise, assuming that the position calculation error of the system is less than 1 m, the velocity calculation error is less than 2 m/s, and the acceleration calculation error is less than 1.3 m/s².

The matrix of the system can be expressed as

During the actual driving process, the motion state of the obstacle vehicle may be moving straightforward, left turning, or right turning, and it is also possible that turning left or right is followed by moving straightforward. Under the traffic condition of moving straightforward, the motion state is characterized by the regular variation of the position parameter , slight variation of the position parameter , and slight variation of and in direction. Under the traffic condition of left and right turning, obvious variations of the position parameters , , and , as well as , can be observed.

The vehicle is considered to be a particle. After the vehicle motion state is predicted using the extended Kalman filter algorithm, its motion state equation is modified, and the time period length of each prediction is changed in order to obtain the predicted value after a period of ( represents the time difference between the two predictions). The new equation for vehicle motion is

The vehicle motion state equation at the moment iswhere is given a value from 1 to , and , in which is the iteration number of the state equation. We can change the value of to predict the trajectory of the vehicle in a certain period of time.

The process of tracking the detected obstacle vehicle using the Kalman filter algorithm is as follows: First, the detected vehicle is marked by a rectangular frame, and the vehicle’s position in the image is obtained. The center of the rectangular frame is taken as the centroid of the vehicle for marking, and the marked position is the state variable in the tracking process. Then, the algorithm judges whether the vehicles in different images are the same vehicle through feature matching. Finally, the motion state equation is continuously updated in order to determine the predicted position of the vehicle at a certain moment in the future. The tracking result is shown in Figure 8. The vehicle position is indicated by a yellow frame, and the vehicle tracking result is marked with a red frame. The experimental results show that the vehicle tracking method based on the Kalman filter algorithm can achieve a position matching degree of 94%, which satisfies the requirement for tracking vehicles at intersections.

3. Safety Assessment Based on Trajectory Prediction, Collision Zone, and Collision Time

After determining the accurate motion state and trajectory of the obstacle vehicle, the next step is to determine the cross-conflict point between the ego vehicle and the obstacle vehicle. The time-to-collision method is used to detect cross-collisions and assess the obstacle vehicle’s safety. Calculating the time for the ego vehicle and the obstacle vehicle to enter and leave the collision zone, respectively, and judging whether the cross collision occurs according to the W value of the time period of the ego vehicle and the obstacle vehicle passing through the collision zone, the conflict level is determined based on the distance between the two vehicles when one of them reaches the cross-conflict point, and the safety of the obstacle vehicle is evaluated based on the conflict level. The specific process is shown in Figure 9.

3.1. Vehicle Trajectory Crossing Model Based on Motion State

When there is the possibility of a cross-collision between the ego vehicle and the obstacle vehicle, their trajectories will intersect. In cross-conflicts, the X-shaped trajectory intersection is the most common type. An X-shaped trajectory intersection indicates that the smaller included angle after the intersection of the extension lines of the driving directions of two vehicles is an acute angle, i.e., the two lines form an X shape. Perpendicular trajectory intersection is a special case where the included angle is 90°. Suppose is the trajectory intersection angle of the ego vehicle and the obstacle vehicle. In this article, we performed cross-collision detection for X-shaped trajectory intersection conditions and established a safety assessment model (Figure 10).

Suppose both the ego vehicle and the obstacle vehicle move along a straight line in a time period ; the position of the ego vehicle at the moment is taken as the origin of coordinates; the driving direction of the ego vehicle is the positive direction of the X-axis, and the position direction of the Y-axis is obtained by rotating the positive direction of the X-axis clockwise by 90°. At the moment , the position of the ego vehicle is the origin of coordinates . The position of the ego vehicle is and at moments and , respectively. The positions of the obstacle vehicle at the moments and are and , respectively. The heading angle of the obstacle vehicle is the angle between the positive direction of the X-axis and the connecting line of the positions of the obstacle vehicle and ego vehicle at a certain moment. At the moments and , the heading angles of the obstacle vehicle are and , respectively. Moreover, the corresponding distances between the two vehicles are and , respectively. The trajectory intersection point is denoted as . Thus, the positions of the obstacle vehicle at the moments and can be expressed using , , , and , which are and .

The trajectory of the obstacle vehicle can be expressed by a two-point formula:

The component of obstacle speed in the direction is

When = 90°, the perpendicular trajectory intersection model is established like the one that is shown in Figure 11.

(a)

(b)

3.2. Cross-Conflict Judgment Based on Collision Zone and Collision Time

We use an X-shaped trajectory intersection as an example to establish a model of collision zone and collision time for the scenario in which the ego vehicle encounters a single obstacle vehicle. Assuming that the position of the intersection point is , we can substitute and into equation (14) to obtain

The collision zone is a circular area with a radius of , and its center is the intersection point . A cross-conflict will occur if two vehicles appear in the collision zone simultaneously. , , , and are the collision times, which are, respectively, the times it takes for the ego vehicle and the target vehicle to enter and exit the collision zone. Since the size of the vehicle cannot be ignored, the length and width of the vehicle should be taken into account when calculating the radius of the collision zone. The formula for calculating is as follows:

Assuming that the time from the driver’s activation reaction to the moment the braking takes effect is 1 s, the vehicle travels meters in 1 second. Thus, the modified collision zone radius iswhere represents the speed of the ego vehicle and and are the length and width of the vehicle, respectively. The time periods for the ego vehicle and the target vehicle to enter the collision zone are and , respectively. The following inferences can be made:(1)When , that is, or , it indicates that two vehicles appear in the collision zone simultaneously, resulting in cross-conflict. The ego vehicle must make emergency braking if the available reaction time is limited.(2)When T₁₁>T₂₂, it indicates that the obstacle vehicle will cross the collision zone at first. Thus, cross-conflict will not occur.(3)If , it indicates that the ego vehicle will cross the collision zone at first. Thus, cross-conflict will not occur.

Figure 12 is a schematic diagram of the collision zone and time at an intersection.

3.3. Vehicle Safety Assessment Based on the Safety Distance

Assuming that the safety distance (the distance just long enough to avoid a collision if both drivers apply emergency braking) is , we can take as the critical value of the highest conflict level to calculate the safety distance:where represents the real-time speed of the vehicle arriving at the cross-conflict point, is the driver’s reaction time, is the delay time of the braking system, and represents the maximum braking deceleration of the vehicle.

When , cross-conflict between the ego vehicle and the obstacle vehicle will occur. It is assumed that one vehicle arrives at the cross-conflict point first, and the distance between the other vehicle and the cross-conflict point is . In the case of cross-conflict, it is stipulated that the two thresholds are and . When ≤ , the conflict is a level-3 one; when ≤ < , the conflict is a level-2 one; if ≤ , the conflict is a level-1 one.

4. Real Vehicle Verification and Simulation

4.1. Reliability Analysis of Monocular-Binocular Vision Switching System

In this article, a HAVAL H7 vehicle serves as the experimental vehicle, a RERVISION camera was used as a sensor, and a Core (TM) i7-9700 Lenovo notebook was used as a processing unit. The camera’s optical center is 1.73 m above the ground. The procedure for verifying the reliability of the monocular-binocular vision switching strategy is as follows:(1)Three RERVISION cameras are mounted on a HAVAL H7 vehicle, traveling at a certain unsignalized intersection and collecting traffic video, as shown in Figure 13.(2)The improved artificial potential field principle is programmed. The monocular-binocular vision switching system was implemented to process the captured images and calculate the motion states of the obstacle vehicles in real-time. The artificial gravitational field was established to determine the value of W, and monocular-binocular vision switching was implemented based on the values of W and α. The selective monocular-binocular vision switching strategy was executed to process the captured images so as to identify the obstacle vehicle and calculate its distance, speed, driving angle, and other parameters.(3)A nonselective monocular-binocular vision switching system was set up. A program was used to process the captured images so as to identify the obstacle vehicle in the monocular vision area and calculate its distance, speed, driving angle, and other parameters. In the overlap zone, only binocular vision was used for vehicle identification and parameter calculation.(4)The experimental results obtained using the three methods were compared. The reliability of the monocular-binocular vision switching strategy proposed in this article was verified based on the performance evaluation of each method in detection accuracy, ranging accuracy, ranging speed, and camera detection area.

A frame of camera picture was taken every 0.04 s to collect information about obstacle vehicles around the experimental vehicle (RERVISION camera frames are 25 frames per second). The proposed monocular-binocular vision switching strategy was used to identify each obstacle vehicle and calculate its distance, speed, driving angle, and other parameters. The calculated data, including the number, distance, speed, and driving angle of obstacle vehicles, were compared with the data measured by on-board Lidar so as to assess the performance in detection accuracy, ranging accuracy, ranging speed, and camera detection area, thus verifying the feasibility of the proposed monocular-binocular vision switching strategy. Figure 14 shows the construction of the real vehicle experimental platform, Figure 15 demonstrates the result of detecting a vehicle ahead of the ego vehicle, and Table 1 shows the distance calculation results obtained based on monocular and binocular vision.

The ranging accuracy values calculated in the monocular vision mode and the binocular vision mode were assessed using the ranging accuracy results of Lidar as a benchmark. Table 1 demonstrates that the distance calculation method based on monocular vision can reach an average accuracy of 96.18% within 60 m. The method based on binocular vision can achieve an average accuracy rate of 96.63% within 60 m, which can meet the accuracy level required in measuring the distances of moving targets in a real driving environment in real time.

The detection areas of cameras under different vision types were calculated using the formula for fan area calculation based on camera FOV. The detection accuracy and ranging accuracy of the monocular-binocular vision switching system are expressed by the average of the accuracy of monocular vision and the accuracy of binocular vision. Table 2 compares the experimental results of various vision switching strategies, assuming that the actual distance is a, the calculated distance is b, and the ranging accuracy is .

Table 2 demonstrates that the monocular-binocular vision switching system proposed in this article outperforms pure monocular vision in terms of detection accuracy and is superior to pure binocular vision in terms of the detection area. In the performance of distance calculation, the proposed monocular-binocular vision switching system has higher ranging accuracy than pure monocular vision and high ranging speed than pure binocular vision. Overall, the proposed monocular-binocular vision switching system meets the requirements for environment perception in the left and right front directions.

4.2. Simulation of Vehicle Safety Assessment Model Based on Collision Zone

A PreScan simulation experiment was conducted on the safety assessment model based on the collision zone. The parameters of the ego vehicle and the driving environment were set according to the driving data and environmental conditions obtained in a real vehicle experiment, and cross-conflict classification and safety level division were carried out following the procedure below:(1)We set the driving scene as an unsignalized intersection.(2)We set the travel speed and direction of the ego vehicle.(3)We set the initial position, driving direction, travel speed, and initial azimuth of the obstacle vehicle.(4)We obtain the length and width data of the ego vehicle and obstacle vehicle according to their models.(5)We calculate the collision radius using formula (16) and calculate the values of and .(6)We randomly generate any value for a speed between 10 km/h and 60 km/h of the ego vehicle.(7)We set the traffic condition of one obstacle vehicle and the traffic condition of two obstacle vehicles. The initial positions and driving directions of the obstacle vehicles are set randomly, and the travel speed of each obstacle vehicle is set randomly between 10 km/h and 60 km/h.(8)We record the motion states of the ego vehicle and the obstacle vehicle every 0.5 s and judge the safety situation.

In the PreScan software, a driving scene model of unsignalized intersection was developed to facilitate the simulation and analysis of the results of judging conflicts resulting from X-shaped trajectory intersection. The scene simulation is shown in Figure 16. There is only one obstacle vehicle in traffic conditions 1–3, and there are two obstacle vehicles in traffic condition 3. Among them, = 16.7 m and = 5 m. The simulation results are as follows.

4.2.1. Traffic Condition 1

The speed of the ego vehicle is = 60 km/h, the speed of the obstacle vehicle is = 16 km/h, the initial distance from the ego vehicle to the cross-conflict point is set to 120 m, the initial distance from the obstacle vehicle to the cross-conflict point is 52 m, and the initial heading angle of the obstacle vehicle is = 15°. The simulation results are as follows: = 5.9 s, = 8.5 s, = 6.8 s, and = 16.6 s, and the distance variation between the two vehicles during driving is shown in Figure 17. When > , it indicates that the ego vehicle passes through the collision zone before the target vehicle and there is no danger of a collision. As shown in Figure 18, when the ego vehicle arrives at the cross-conflict point at 7.2 s, the distance from the obstacle vehicle to the cross-conflict point is = 13 m (which constitutes a second-class conflict). The ego vehicle is far from the cross-conflict point when the obstacle vehicle arrives at the cross-conflict point at 11.7 s. Thus, this scenario will not be discussed further.

4.2.2. Traffic Condition 2

The speed of the ego vehicle is = 50 km/h, the speed of the obstacle vehicle is = 36 km/h, the distance from the ego vehicle to the cross-conflict point is set to 65 m, the distance from the obstacle vehicle to the cross-conflict point is 60 m, and the initial heading angle of the obstacle vehicle is = 67°. The simulation results are = 4.3 s, = 5.1 s, = 4.4 s, and = 5.5 s, and the radius of the collision zone is = 4.57 m. The distance variation between the two vehicles during driving is shown in Figure 18. If T₁₁ < T₂₁ < T₁₂, a cross-collision between the two vehicles will occur. As shown in Figure 18, when the ego vehicle reaches the cross-conflict point, the obstacle vehicle is 14 m away from the cross-conflict point. Thus, the conflict level of this obstacle vehicle is level 2.

4.2.3. Traffic Condition 3

The speed of the ego vehicle is = 40 km/h, the speed of obstacle vehicle 1 is = 45 km/h, and the speed of obstacle vehicle 2 is = 47 km/h. The distance from the ego vehicle to trajectory intersection point 1 is 50 m, the distance from obstacle vehicle 1 to trajectory intersection point 1 is 80 m, and the initial heading angle of obstacle vehicle 1 is 62°. The distance from the ego vehicle to trajectory intersection point 2 is 60 m, the distance from obstacle vehicle 2 to trajectory intersection point 2 is 50 m, and the initial heading angle of obstacle vehicle 2 is 43°. The simulation results of obstacle vehicle 1 are = 2.5 s, = 6.5 s, = 6.26 s, and = 9.74 s. At time T = 0.24 s, indicating that cross-conflict will occur. It can be seen from Figure 19(a) that when the ego vehicle reaches the intersection point, the distance from obstacle vehicle 1 to the intersection point is = 36 m, which constitutes a level 1 conflict. The simulation results of obstacle vehicle 2 are as follows: = 3.5 s, = 7.4 s, = 2.0 s, and = 5.5 s. At time = 2.0 s, indicating that cross-conflict will occur, and the collision radius is = 4.97 m. Figure 19(b) demonstrates that when obstacle vehicle 2 reaches the intersection point, the distance from the ego vehicle to the intersection point is 18 m, which constitutes a level 1 conflict. Figure 19 demonstrates the variations of the distances between the vehicles during the process of obstacle vehicles 1 and 2 passing the intersection points.

(a)

(b)

The simulation results of the three traffic conditions were compared with the theoretical calculation results. The comparison demonstrated that the cross-conflict judgment and safety assessment model based on trajectory prediction, collision zone, and collision time can effectively identify cross-conflicts and rate the severity of each conflict under the corresponding threshold. PreScan simulation results show that the effective recognition rate of cross conflict based on collision zone and collision time is more than 97%, and the recognition accuracy of conflict level is more than 95%. It shows that the intersection conflict judgment method and safety evaluation model proposed in this article are suitable for low-speed driving of vehicles at unsignalized intersections.

5. Conclusion

Aiming to solve the rarely studied problem of how to ensure traffic safety at unsignalized intersections, we developed a method to detect cross-conflicts between the ego vehicle and vehicles traveling in the left and right direction based on machine vision and a safety assessment model. First, a monocular-binocular vision switching strategy based on an improved artificial potential field method was developed, and a monocular-binocular vision switching system was set up. Then, a single-binocular vision conversion strategy based on the basic principle of artificial potential field is formulated, and a single-binocular vision environment perception system is built to realize the effective detection of obstacle vehicles in front of the vehicle side. In addition, a vehicle motion state estimation model and a vehicle trajectory prediction model based on the extended Kalman filter algorithm were constructed. On this basis, a cross-conflict detection and safety assessment model based on vehicle trajectory, collision zone, and collision time was developed. With a reasonable setting of conflict level thresholds, the method proposed in this article can enable effective safety assessment for the vehicles traveling at low speeds at intersections, provide a reasonable process for judging the conflict level in the traffic environment at intersections, and provide a basis for the takeover of the driving assistance system and for human-computer interaction of autonomous vehicles.

Data Availability

No data were used to support this study.

Conflicts of Interest

There are no conflicts of interest regarding the publication of this paper.

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grant 51905320, China Postdoctoral Science Foundation under Grant 2018M632696, Natural Science Foundation of Shandong Province under Grants ZR2022MF230 and ZR2021QF039, Shandong Province Major Science and Technology Innovation Project under Grant 2019JZZY010911, Shandong Key R&D Plan Project under Grant 2019GGX104066, Experimental Technology Upgrading Project under Grant 2022003, and Shandong Provincial Program of Introducing and Cultivating Talents of Discipline to Universities: Research and Innovation Team of Intelligent Connected Vehicle Technology.

References

L. Zheng, T. Sayed, and M. Essa, “Validating the bivariate extreme value modeling approach for road safety estimation with different traffic conflict indicators,” Accident Analysis & Prevention, vol. 123, pp. 314–323, 2019.
View at: Publisher Site | Google Scholar
N. Long, K. Wang, R. Cheng, W. Hu, and K Yang, “Unifying obstacle detection, recognition, and fusion based on millimeter wave radar and RGB-depth sensors for the visually impaired,” Review of Scientific Instruments, vol. 90, no. 4, Article ID 044102, 2019.
View at: Publisher Site | Google Scholar
X. Zhang, M. Zhou, P. Qiu, Y. Huang, and J. Li, “Radar and vision fusion for the real-time obstacle detection and identification,” Industrial Robot: The International Journal of Robotics Research and Application, vol. 46, no. 3, pp. 391–395, 2019.
View at: Publisher Site | Google Scholar
G. Zhai, C. Wu, and Y. Wang, “Millimeter wave radar target tracking based on adaptive kalman filter,” in Proceedings of the 2018 IEEE Intelligent Vehicles Symposium (IV), pp. 453–458, IEEE, Changshu, China, October 2018.
View at: Publisher Site | Google Scholar
R. Zhang, Y. Yang, W. Wang, L. Zeng, J. Chen, and S. McGrath, “An algorithm for obstacle detection based on YOLO and light filed camera,” in Proceedings of the 2018 12th International Conference on Sensing Technology (ICST), pp. 223–226, IEEE, Limerick, Ireland, December 2018.
View at: Publisher Site | Google Scholar
T. Eppenberger, G. Cesari, M. Dymczyk, R. Siegwart, and R. Dubé, “Leveraging stereo-camera data for real-time dynamic obstacle detection and tracking,” in Proceedings of the 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 10528–10535, IEEE, Las Vegas, NV, USA, October 2020.
View at: Publisher Site | Google Scholar
F. Rosique, P. J. Navarro, C. Fernández, and A Padilla, “A systematic review of perception system and simulators for autonomous vehicles Research,” Sensors, vol. 19, no. 3, p. 648, 2019.
View at: Publisher Site | Google Scholar
W. He, Y. Yu, Y. Yu, Z. Wang, Z. Wan, and Z. Xu, “Design of an unmanned aerial vehicle survey system based on embedded 3D machine vision imaging perception,” Journal of Physics: Conference Series, vol. 2141, no. 1, Article ID 012013, 2021.
View at: Publisher Site | Google Scholar
J. Van Brummelen, M. O’Brien, D. Gruyer, and H. Najjaran, “Autonomous vehicle perception: the technology of today and tomorrow,” Transportation Research Part C: Emerging Technologies, vol. 89, pp. 384–406, 2018.
View at: Publisher Site | Google Scholar
I. Benamer, A. Yahiouche, and A. Ghenai, “Deep learning environment perception and self-tracking for autonomous and connected vehicles,” in Proceedings of the International Conference on Machine Learning for Networking, pp. 305–319, Paris, France, November 2020.
View at: Google Scholar
W. Liu, D. Anguelov, D. Erhan et al., “Ssd: single shot multibox detector,” in European Conference on Computer Vision, pp. 21–37, Springer, Berlin, Germany, 2016.
View at: Google Scholar
J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: unified, real-time object detection,” in Proceedings of the IEEE Conference on Computer vision and pattern recognition, pp. 779–788, Las Vegas, NV, USA, June 2016.
View at: Google Scholar
J. Zhao, V. L. Knoop, and M. Wang, “Microscopic traffic modeling inside intersections: interactions between drivers,” Transportation Science, 2022.
View at: Publisher Site | Google Scholar
J. Zhao, V. L. Knoop, and M. Wang, “Two-dimensional vehicular movement modelling at intersections based on optimal control,” Transportation Research Part B: Methodological, vol. 138, pp. 1–22, 2020.
View at: Publisher Site | Google Scholar
Q. Cao, Z. Zhao, Q. Zeng, Z. Wang, and K. Long, “Real-time vehicle trajectory prediction for traffic conflict detection at unsignalized intersections,” Journal of Advanced Transportation, vol. 2021, Article ID 8453726, 15 pages, 2021.
View at: Publisher Site | Google Scholar
D. Katare and M. El-Sharkawy, “Embedded system enabled vehicle collision detection: an ANN classifier,” in Proceedings of the 2019 IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC), pp. 0284–0289, IEEE, Las Vegas, NV, USA, January 2019.
View at: Publisher Site | Google Scholar
S. Agrawal and S. W. Varade, “Collision detection and avoidance system for vehicle,” in Proceedings of the 2017 2nd International Conference on Communication and Electronics Systems (ICCES), pp. 476-477, IEEE, Coimbatore, India, October 2017.
View at: Publisher Site | Google Scholar
K. P. Chen and P. A. Hsiung, “Vehicle collision prediction under reduced visibility conditions,” Sensors, vol. 18, no. 9, p. 3026, 2018.
View at: Publisher Site | Google Scholar
N. Formosa, M. Quddus, S. Ison, M. Abdel-Aty, and J. Yuan, “Predicting real-time traffic conflicts using deep learning,” Accident Analysis & Prevention, vol. 136, Article ID 105429, 2020.
View at: Publisher Site | Google Scholar
Z. Huang, X. Sun, Z. Chen, T. Liu, and X. Li, “Extended risk estimation and warning model for connected vehicles at unsignalized intersection,” in Proceedings of the 2021 international conference on control, automation and information sciences (ICCAIS), pp. 7–14, IEEE, Xi'an, China, October 2021.
View at: Publisher Site | Google Scholar
M. Rocha, M. Anzanello, F. Caleffi, H. Cybis, and G. Yamashita, “A multivariate-based variable selection framework for clustering traffic conflicts in a brazilian freeway,” Accident Analysis & Prevention, vol. 132, p. 105269, 2019.
View at: Google Scholar
F. Qadri, “Traffic conflict modelling at six leg unsignalized intersection,” in 11th Asia Pacific Transportation and the Environment Conference (APTE 2018), Atlantis Press, Amsterdam, Netherlands, 2019.
View at: Google Scholar
L. Zheng, Y. Huang, T. Sayed, and C Wen, “Validating the Bayesian hierarchical extreme value model for traffic conflict-based crash estimation on freeway segments with site-level factors,” Accident Analysis & Prevention, vol. 159, Article ID 106269, 2021.
View at: Publisher Site | Google Scholar
W. Liu, K. Quijano, and M. M. Crawford, “YOLOv5-Tassel: detecting tassels in RGB UAV imagery with improved YOLOv5 based on transfer learning,” Ieee Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 15, pp. 8085–8094, 2022.
View at: Publisher Site | Google Scholar
Z. D. Zhang, B. Zhang, Z. C. Lan et al., “FINet: an insulator dataset and detection benchmark based on synthetic fog and improved YOLOv5,” IEEE Transactions on Instrumentation and Measurement, vol. 71, pp. 1–8, 2022.
View at: Publisher Site | Google Scholar
C. Li, G. Zhao, D. Gu, and Z. Wang, “Improved lightweight YOLOv5 using attention mechanism for satellite components recognition,” IEEE Sensors Journal, vol. 23, no. 1, pp. 514–526, 2023.
View at: Publisher Site | Google Scholar
N. Aswini and S. V. Uma, “Obstacle avoidance and distance measurement for unmanned aerial vehicles using monocular vision,” International Journal of Electrical and Computer Engineering, vol. 9, no. 5, p. 3504, 2019.
View at: Publisher Site | Google Scholar
Z. Meng, X. Kong, L. Meng, and H. Tomiyama, “Distance measurement and camera calibration based on binocular vision technology,” in Proceedings of the 2018 International Conference on Advanced Mechatronic Systems (ICAMechS), pp. 342–347, IEEE, Zhengzhou, China, August 2018.
View at: Publisher Site | Google Scholar
J. F. Duhé, S. Victor, and P. Melchior, “ContribUtions on artificial potential field method for effective obstacle avoidance,” Fractional Calculus and Applied Analysis, vol. 24, no. 2, pp. 421–446, 2021.
View at: Publisher Site | Google Scholar
S. M. H. Rostami, A. K. Sangaiah, J. Wang, and X. Liu, “Obstacle avoidance of mobile robots using modified artificial potential field algorithm,” EURASIP Journal on Wireless Communications and Networking, vol. 2019, no. 1, pp. 1–19, 2019.
View at: Google Scholar
A. D. Sabiha, M. A. Kamel, E. Said, and W. M. Hussein, “Trajectory generation and tracking control of an autonomous vehicle based on artificial potential field and optimized backstepping controller,” in Proceedings of the 2020 12th International Conference on Electrical Engineering (ICEENG), pp. 423–428, IEEE, Cairo, Egypt, July 2020.
View at: Publisher Site | Google Scholar
A. Dairi, F. Harrou, M. Senouci, and Y Sun, “Unsupervised obstacle detection in driving environments using deep-learning-based stereovision,” Robotics and Autonomous Systems, vol. 100, pp. 287–301, 2018.
View at: Publisher Site | Google Scholar
G. Xue, Z. Li, H. Zhu, and Y. Liu, “Traffic-known urban vehicular route prediction based on partial mobility patterns,” in Proceedings of the 2009 15th International Conference on Parallel and Distributed Systems, pp. 369–375, IEEE, Shenzhen, China, December 2009.
View at: Publisher Site | Google Scholar
J. Wu, J. Zhang, B. Nie, Y. Liu, and X He, “Adaptive control of PMSM servo system for steering-by-wire system with disturbances observation,” IEEE Transactions on Transportation Electrification, vol. 8, no. 2, pp. 2015–2028, 2022.
View at: Publisher Site | Google Scholar
N. Deo and M. M. Trivedi, “Convolutional social pooling for vehicle trajectory prediction,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1468–1476, Salt Lake City, UT, USA, 2018.
View at: Google Scholar
X. Chen, Z. Wang, Q. Hua, W. L. Shang, Q. Luo, and K. Yu, “AI-empowered speed extraction via port-like videos for vehicular trajectory analysis,” IEEE Transactions on Intelligent Transportation Systems, 2022.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2023 Yuqiong Wang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies