Abstract

This paper proposes an accurate and robust inter-vehicle distance estimation method using a stereo high-speed vision. The framework involves two phases: a tracking phase, in which a preceding vehicle is accurately and stably tracked by a novel visual tracking algorithm optimized for stereo high-speed vision; and a distance estimation phase, in which multiple distance estimations by scaling are aggregated to provide an accurate and robust estimation with a specially designed technique, patch multiplexing, to perform the aggregation efficiently and effectively. By using a high-speed laser rangefinder to compare not only distance but also velocity and acceleration using a Kalman filter, we showed that the accuracy of inter-vehicle distance estimation can be favorably compared with the laser rangefinder.

How to track the front vehicle?

To measure the distance to the front vehicle, we need to identify the front vehicle in both cameras. The initial bounding box is estimated independently in both cameras using a deep learning-based object detection method. Assuming that the front vehicle is in the same lane, matching is performed to identify the bounding box of the front vehicle on both cameras. As shown in the figure, this bounding box is then tracked using a method, called Alternating MOSSE (A-MOSSE), which leverages stereo high-speed vision. In this method, both cameras share the same correlation filter and update it alternately, resulting in robust, high-speed, and accurate vehicle tracking as shown in the upfront video. It is possible to measure the distance to the front vehicle using triangulation, but as we will see later, it is not accurate enough to measure the velocity and acceleration.

How to measure the distance accurately and robustly?

We propose to use scale-based ranging to achieve higher accuracy. To explain the principle, we consider a situation where a camera mounted on an ego vehicle is measuring the distance to the blue car in front, as shown in the figure below. At a certain time, the distance to the blue car is estimated to be 20m by, for example, triangulation. After that, when the blue car moves away to 40 meters ahead, the apparent size of the car should be reduced by half. Scale-based ranging applies this principle to estimate the distance to the car by estimating the scaling rate of the appearance of the car. The apparent scaling rate of the car is estimated by using a method based on phase-only correlation. This method achieves highly accurate and robust estimation even under intensive vibration. In the proposed method, we aggregate multiple distance measurements by scale-based ranging to achieve higher estimation accuracy. Aggregation involves robust averaging, which reduces the variance of the measurement through statistical processing and efficiently removes outliers from the estimations. However, simply performing Aggregation will increase computational cost.

As a remedy to this problem, we proposed Patch Multiplexing. Suppose a car in front of us is gradually approaching, the size of the bounding box will gradually increase. In this case, the scale of the truck at the current frame with respect to the past frames is calculated for the number of frames to be aggregated, and the amount of calculation increases proportionally. In this study, we discretize the size of the patches used for scale calculation so that the results of previous calculations can be reused. However, this discretization is not always enough to achieve a sufficient number of aggregations. To solve this problem, we propose patch multiplexing, which pre-computes patches in keyframes one size larger and one size smaller. This allows for efficient aggregation by doing the necessary processing for scale calculation as much as possible in advance. Although this will increase the amount of computation for past frames to be used for aggregation, it will not affect the real-time performance because the processing for keyframes can be asynchronously executed.

Results

We demonsrated the proposed system in three different scenes. In the first scene, we were following a truck on the highway, which is the most common situation in truck platooning. We can see the bounding box of the truck and its distance from the ego-vehicle. The estimated relative velocity and acceleration are also displayed in the speedometers below. Although the front truck was slightly tilted when approaching a curve, we can see that the bounding box is pretty stable and the distance is estimated accurately. We demonstrated the proposed system in three different scenes. In the first scene, we were following a truck on the highway, which is the most common situation in truck platooning. We can see the bounding box of the truck and its distance from the ego vehicle. The estimated relative velocity and acceleration are also displayed in the speedometers below. Although the front truck was slightly tilted when approaching a curve, we can see that the bounding box is pretty stable and the distance is estimated accurately.

In the second scene (left), a parked truck that was more than 100m away was approached at about 10m by sudden acceleration and braking. As we can see, the distance, velocity, and acceleration can be estimated stably without losing track even in a situation where relative velocity and acceleration are quite high. In the last scene (right), both the front vehicle and the ego-vehicle were suffering intensive vibration and the slope was changing in the middle of the scene. Even in this challenging situation, we can see that the front vehicle can be stably tracked, and accurate distance estimation can be achieved. For further analysis, please refer to our paper.

YouTube

Citation

@inproceedings{hirano21msa,
  title={Multiple Scale Aggregation with Patch Multiplexing for High-speed Inter-vehicle Distance Estimation},
  author={Masahiro Hirano and Yuji Yamakawa and Taku Senoo and Norimasa Kishi and Masatoshi Ishikawa},
  booktitle={Proceedings of the IEEE Intelligent Vehicles Symposium (IV)},
  year={2021}
}