Learning notes 23-- basic theory of multi-sensor information fusion (Part I)

This blog series includes 6 A column , Respectively ：《 Overview of autopilot Technology 》、《 Technical foundation of autopilot vehicle platform 》、《 Autopilot positioning technology 》、《 Self driving vehicle environment perception 》、《 Decision and control of autonomous driving vehicle 》、《 Design and application of automatic driving system 》, The author is not an expert in the field of automatic driving , Just a little white on the road of exploring automatic driving , This series has not been read , It is also thinking while reading and summarizing , Welcome to friends , Please give your suggestions in the comments area , Help the author pick out the mistakes , thank you ！
This column is about 《 Self driving vehicle environment perception 》 Book notes

---

---

1. Basic theory of multisensor information fusion

1.1 An introduction to

• Automatic driving , Multiple sensors are required to cooperate with each other , Together, they form the perception system of autonomous vehicle ;
• In the process of multi-sensor information fusion , The following key problems need to be solved ：
  • data alignment . Since the data observed by each sensor is within its own reference frame , Before fusing this information , They must be swapped into the same space-time frame ; The rounding error caused by space-time registration must be compensated accordingly ;
  • Uncertainty of sensor observation data . Due to the uncertainty of the working environment of the sensor , This results in noise components in the observation data , In the process of fusion, it is necessary to reduce the uncertainty of these information to the greatest extent ;
  • Data Association . The problem of data association is widespread , It is necessary to solve the correlation problem in the time domain of a single sensor , And the correlation problem in multi-sensor spatial domain , Thus, data from the same target source can be determined ;
  • Incompleteness 、 Inconsistent and false data . In multi-sensor information fusion system , There are sometimes multiple interpretations of the measured data received by the sensor , It is called data incompleteness ; Multisensor data often give inconsistent or even contradictory explanations to the observation environment ;
• Requirements for on-board system ：
  • Unified synchronization clock , Ensure the time consistency and correctness of sensor information ;
  • Accurate multi-sensor calibration , Ensure the spatial consistency of different sensor information at the same time ;

1.2 Basic theory of multisensor information fusion

1.2.1 Overview of multisensor information fusion

• Sensor data fusion is an information processing method for a system using multiple sensors , It can take advantage of the combination of multiple sensors , Eliminate the limitations of a single sensor ;
• Integrate the data resources provided by multiple sensors of the same or different kind distributed in different locations , Use computer technology to analyze it , Complement each other , Achieve the best synergy , Obtain consistent interpretation and description of the observed object , Improve the fault tolerance of the system , So as to improve the system decision-making 、 planning 、 The rapidity and correctness of the response , Make the system obtain more sufficient information ;
• Advantages of using multi-sensor fusion technology ：
  • Improve the accuracy of system perception . Multiple sensors are combined and complementary , Avoid the limitations of a single sensor , Maximize the advantages of each sensor , It can simultaneously obtain a variety of different feature information of the detected object , Reduce environmental 、 Noise and other disturbances ;
  • Increase the perception dimension of the system , Improve the reliability and robustness of the system . Multisensor fusion can bring a certain degree of information redundancy , Even if one of the sensors fails , The system can still continue to work normally within a certain range , It has high fault tolerance , Increase the reliability and confidence of system decision-making ;
  • Enhance the ability to use the environment . The information collected by multi-sensor fusion technology has obvious feature complementarity , A wider coverage of space and time , It makes up for the semantic uncertainty of spatial resolution and environment of a single sensor .
  • Effectively reduce costs . Fusion can realize multiple inexpensive sensors to replace expensive sensor devices , Reduce the cost budget on the basis of guaranteed performance ;
• Sensor fusion process ：
  1. Multiple sensors work independently to obtain observation data ;
  2. For each sensor data (RGB Images 、 Point cloud data, etc ) Pre treatment ;
  3. Feature extraction of processing data 、 Transformation , And carry out pattern recognition processing , Get the description information of the observation object ;
  4. Data association is carried out in the data fusion center according to certain criteria ;
  5. The data of each sensor is fused using an algorithm that is sufficiently optimized , Obtain a consistent description and interpretation of the observed object .